courier imap - Page 22 - Developer IT

Can nginx be an mail proxy for a backend server that does not accept cleartext logins?

- by 84104

Can Nginx be an mail proxy for a backend server that does not accept cleartext logins? Preferably I'd like to know what directive to include so that it will invoke STARTTLS/STLS, but communication via IMAPS or POP3S is sufficient. relevant(?) section of nginx.conf mail { auth_http localhost:80/mailproxy/auth.php; proxy on; ssl_prefer_server_ciphers on; ssl_protocols TLSv1 SSLv3; ssl_ciphers HIGH:!ADH:!MD5:@STRENGTH; ssl_session_cache shared:TLSSL:16m; ssl_session_timeout 10m; ssl_certificate /etc/ssl/private/hostname.crt; ssl_certificate_key /etc/ssl/private/hostname.key; imap_capabilities "IMAP4rev1" "UIDPLUS"; server { protocol imap; listen 143; starttls on; } server { protocol imap; listen 993; ssl on; } pop3_capabilities "TOP" "USER"; server { protocol pop3; listen 110; starttls on; pop3_auth plain; } server { protocol pop3; listen 995; ssl on; pop3_auth plain; } }

Read the article

Dovecot starting and running, but not listening on any port

- by Dženis Macanovic

Among others things I'm in charge of a Debian GNU/Linux (Wheezy) DomU for the mail services of the company i work for. Yesterday one HDD that was used for this particular server has died. After installing Debian again, Dovecot decided to no longer listen on any ports (checked with netstat -l). Other services (like Postfix and MySQL) work without problems. dovecot -n: # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-3-amd64 x86_64 Debian wheezy/sid ext3 auth_mechanisms = plain login disable_plaintext_auth = no first_valid_uid = 150 last_valid_uid = 150 mail_gid = mail mail_location = maildir:/var/vmail/%d/%n mail_uid = vmail namespace inbox { inbox = yes location = prefix = } pass db { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = mail mode = 0666 user = vmail } } service imap-login { inet_listener imaps { port = 993 ssl = yes } } service pop3-login { inet_listener pop3s { port = 995 ssl = yes } } ssl_cert = </etc/ssl/private/mail.crt ssl_key = </etc/ssl/private/mail.key userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol imap { mail_max_userip_connections = 25 } UID 150 is vmail (I double checked file permissions). I didn't install Dovecot from source, but via apt from the official Debian US mirror. There are no messages concerning Dovecot in /var/log/syslog except for: Oct 21 06:36:29 server dovecot: master: Dovecot v2.1.7 starting up (core dumps disabled) Any ideas?

Read the article

What are the typical methods used to scale up/out email storage servers?

- by nareshov

Hi, What I've tried: I have two email storage architectures. Old and new. Old: courier-imapds on several (18+) 1TB-storage servers. If one of them show signs of running out of disk space, we migrate a few email accounts to another server. the servers don't have replicas. no backups either. New: dovecot2 on a single huge server with 16TB (SATA) storage and a few SSDs we store fresh mails on the SSDs and run a doveadm purge to move mails older than a day to the SATA disks there is an identical server which has a max-15min-old rsync backup from the primary server higher-ups/management wanted to pack in as much storage as possible per server in order to minimise the cost of SSDs per server the rsync'ing is done because GlusterFS wasn't replicating well under that high small/random-IO. scaling out was expected to be done with provisioning another pair of such huge servers on facing disk-crunch issues like in the old architecture, manual moving of email accounts would be done. Concerns/doubts: I'm not convinced with the synchronously-replicated filesystem idea works well for heavy random/small-IO. GlusterFS isn't working for us yet, I'm not sure if there's another filesystem out there for this use case. The idea was to keep identical pairs and use DNS round-robin for email delivery and IMAP/POP3 access. And if one the servers went down for whatever reasons (planned/unplanned), we'd move the IP to the other server in the pair. In filesystems like Lustre, I get the advantage of a single namespace whereby I do not have to worry about manually migrating accounts around and updating MAILHOME paths and other metadata/data. Questions: What are the typical methods used to scale up/out with the traditional software (courier-imapd / dovecot)? Do traditional software that store on a locally mounted filesystem pose a roadblock to scale out with minimal "problems"? Does one have to re-write (parts of) these to work with an object-storage of some sort - such as OpenStack object storage?

Read the article

Is there a learnable filter in Thunderbird for non-spam messages (as in Opera Mail)?

- by Debilski

One feature I like very much about Opera Mail is that you can have learnable filters for any purpose. So not only can you filter spam messages but also messages that your friends sent you or info mails from web platforms without having to enter each and every mail address you want to filter. It actually works quite satisfying and you can combine it with string filtering, too. It does a few mistakes in the beginning but then improves quickly after you have removed some of the false positives. However, there are a couple of drawbacks with the Opera Mail module. The filters are only ‘virtual’. So, with IMAP there is no easy way of mirroring the filtered structure back on the server and when I’m using webmail I’ll see the whole unfiltered mess in the inbox folder. Opera’s not using the OS X address book — and neither does it use ldap (which is not too important for me at the moment). So, I’m not specifically looking for a Thunderbird solution here, a way to fix things in Opera or Apple Mail should do as well. Or some other E-Mail program I don’t even know about yet. (So, to be clear, I’d like to have: OS X Address Book integration, Learnable filters for any type of filtering, Ability to push filtered folder structure to the IMAP server) But I thought like, if it is possible then most probably there would be an add-on for Thunderbird available. Any ideas?

Read the article

Email Proxy Ideas

- by jtnire

Hi Everyone, I wish to host some managed email servers for some customers. Each customer will have their own email server which will be an all-in-one virtual machine running postfix, dovecot and some webmail suite. Even though each customer will have their own server, I do not wish to give each email server it's own public facing IP. I wish to avail the use of proxy servers so all customers use the same public IP. As for the "smtp-in" from the public internet, this isn't a problem as I can set up many mx servers (using postfix) which will store-and-forward the mail to the correct server (using transport maps). As for the IMAP access from the customer, I was thinking of using perdition which is an IMAP proxy - I believe that this will suit my needs. I am confused however on what to use for the "smtp-out" proxy. The customers will have to authenticate with their receptive email server, however they will have to go via a proxy of some sort as they won't have direct access to their server instance. It probably can't be a store-and-forward proxy either. Does anyone have any idea on what I could use here? Many Thanks

Read the article

C# acting weird when reading in values from a file to an array

- by Whitey

This is the structure of my file: 1111111111111111111111111 2222222222222222222222222 3333333333333333333333333 4444444444444444444444444 5555555555555555555555555 6666666666666666666666666 7777777777777777777777777 8888888888888888888888888 9999999999999999999999999 0000000000000000000000000 0000000000000000000000000 0000000000000000000000000 0000000000000000000000000 0000000000000000000000000 And this is the code I'm using to read it into an array: using (StreamReader reader = new StreamReader(mapPath)) { string line; for (int i = 0; i < iMapHeight; i++) { if ((line = reader.ReadLine()) != null) { for (int j = 0; j < iMapWidth; j++) { iMap[i, j] = line[j]; } } } } I have done some debugging, and line[j] correctly iterates through each character in the currently read line. The problem lies with iMap[i, j]. After this block of code executes, this is the contents of iMap: - iMap {int[14, 25]} int[,] [0, 0] 49 int [0, 1] 49 int [0, 2] 49 int [0, 3] 49 int [0, 4] 49 int [0, 5] 49 int [0, 6] 49 int [0, 7] 49 int [0, 8] 49 int [0, 9] 49 int [0, 10] 49 int [0, 11] 49 int [0, 12] 49 int [0, 13] 49 int [0, 14] 49 int [0, 15] 49 int [0, 16] 49 int [0, 17] 49 int [0, 18] 49 int [0, 19] 49 int [0, 20] 49 int [0, 21] 49 int [0, 22] 49 int [0, 23] 49 int [0, 24] 49 int [1, 0] 50 int [1, 1] 50 int [1, 2] 50 int [1, 3] 50 int [1, 4] 50 int [1, 5] 50 int [1, 6] 50 int [1, 7] 50 int [1, 8] 50 int [1, 9] 50 int [1, 10] 50 int [1, 11] 50 int [1, 12] 50 int [1, 13] 50 int [1, 14] 50 int [1, 15] 50 int [1, 16] 50 int [1, 17] 50 int [1, 18] 50 int [1, 19] 50 int [1, 20] 50 int [1, 21] 50 int [1, 22] 50 int [1, 23] 50 int [1, 24] 50 int [2, 0] 51 int [2, 1] 51 int [2, 2] 51 int [2, 3] 51 int [2, 4] 51 int [2, 5] 51 int [2, 6] 51 int [2, 7] 51 int [2, 8] 51 int [2, 9] 51 int [2, 10] 51 int [2, 11] 51 int [2, 12] 51 int [2, 13] 51 int [2, 14] 51 int [2, 15] 51 int [2, 16] 51 int [2, 17] 51 int [2, 18] 51 int [2, 19] 51 int [2, 20] 51 int [2, 21] 51 int [2, 22] 51 int [2, 23] 51 int [2, 24] 51 int [3, 0] 52 int [3, 1] 52 int [3, 2] 52 int [3, 3] 52 int [3, 4] 52 int [3, 5] 52 int [3, 6] 52 int [3, 7] 52 int [3, 8] 52 int [3, 9] 52 int [3, 10] 52 int [3, 11] 52 int [3, 12] 52 int [3, 13] 52 int [3, 14] 52 int [3, 15] 52 int [3, 16] 52 int [3, 17] 52 int [3, 18] 52 int [3, 19] 52 int [3, 20] 52 int [3, 21] 52 int [3, 22] 52 int [3, 23] 52 int [3, 24] 52 int [4, 0] 53 int [4, 1] 53 int [4, 2] 53 int [4, 3] 53 int [4, 4] 53 int [4, 5] 53 int [4, 6] 53 int [4, 7] 53 int [4, 8] 53 int [4, 9] 53 int [4, 10] 53 int [4, 11] 53 int [4, 12] 53 int [4, 13] 53 int [4, 14] 53 int [4, 15] 53 int [4, 16] 53 int [4, 17] 53 int [4, 18] 53 int [4, 19] 53 int [4, 20] 53 int [4, 21] 53 int [4, 22] 53 int [4, 23] 53 int [4, 24] 53 int [5, 0] 54 int [5, 1] 54 int [5, 2] 54 int [5, 3] 54 int [5, 4] 54 int [5, 5] 54 int [5, 6] 54 int [5, 7] 54 int [5, 8] 54 int [5, 9] 54 int [5, 10] 54 int [5, 11] 54 int [5, 12] 54 int [5, 13] 54 int [5, 14] 54 int [5, 15] 54 int [5, 16] 54 int [5, 17] 54 int [5, 18] 54 int [5, 19] 54 int [5, 20] 54 int [5, 21] 54 int [5, 22] 54 int [5, 23] 54 int [5, 24] 54 int [6, 0] 55 int [6, 1] 55 int [6, 2] 55 int [6, 3] 55 int [6, 4] 55 int [6, 5] 55 int [6, 6] 55 int [6, 7] 55 int [6, 8] 55 int [6, 9] 55 int [6, 10] 55 int [6, 11] 55 int [6, 12] 55 int [6, 13] 55 int [6, 14] 55 int [6, 15] 55 int [6, 16] 55 int [6, 17] 55 int [6, 18] 55 int [6, 19] 55 int [6, 20] 55 int [6, 21] 55 int [6, 22] 55 int [6, 23] 55 int [6, 24] 55 int [7, 0] 56 int [7, 1] 56 int [7, 2] 56 int [7, 3] 56 int [7, 4] 56 int [7, 5] 56 int [7, 6] 56 int [7, 7] 56 int [7, 8] 56 int [7, 9] 56 int [7, 10] 56 int [7, 11] 56 int [7, 12] 56 int [7, 13] 56 int [7, 14] 56 int [7, 15] 56 int [7, 16] 56 int [7, 17] 56 int [7, 18] 56 int [7, 19] 56 int [7, 20] 56 int [7, 21] 56 int [7, 22] 56 int [7, 23] 56 int [7, 24] 56 int [8, 0] 57 int [8, 1] 57 int [8, 2] 57 int [8, 3] 57 int [8, 4] 57 int [8, 5] 57 int [8, 6] 57 int [8, 7] 57 int [8, 8] 57 int [8, 9] 57 int [8, 10] 57 int [8, 11] 57 int [8, 12] 57 int [8, 13] 57 int [8, 14] 57 int [8, 15] 57 int [8, 16] 57 int [8, 17] 57 int [8, 18] 57 int [8, 19] 57 int [8, 20] 57 int [8, 21] 57 int [8, 22] 57 int [8, 23] 57 int [8, 24] 57 int [9, 0] 48 int [9, 1] 48 int [9, 2] 48 int [9, 3] 48 int [9, 4] 48 int [9, 5] 48 int [9, 6] 48 int [9, 7] 48 int [9, 8] 48 int [9, 9] 48 int [9, 10] 48 int [9, 11] 48 int [9, 12] 48 int [9, 13] 48 int [9, 14] 48 int [9, 15] 48 int [9, 16] 48 int [9, 17] 48 int [9, 18] 48 int [9, 19] 48 int [9, 20] 48 int [9, 21] 48 int [9, 22] 48 int [9, 23] 48 int [9, 24] 48 int [10, 0] 48 int [10, 1] 48 int [10, 2] 48 int [10, 3] 48 int [10, 4] 48 int [10, 5] 48 int [10, 6] 48 int [10, 7] 48 int [10, 8] 48 int [10, 9] 48 int [10, 10] 48 int [10, 11] 48 int [10, 12] 48 int [10, 13] 48 int [10, 14] 48 int [10, 15] 48 int [10, 16] 48 int [10, 17] 48 int [10, 18] 48 int [10, 19] 48 int [10, 20] 48 int [10, 21] 48 int [10, 22] 48 int [10, 23] 48 int [10, 24] 48 int [11, 0] 48 int [11, 1] 48 int [11, 2] 48 int [11, 3] 48 int [11, 4] 48 int [11, 5] 48 int [11, 6] 48 int [11, 7] 48 int [11, 8] 48 int [11, 9] 48 int [11, 10] 48 int [11, 11] 48 int [11, 12] 48 int [11, 13] 48 int [11, 14] 48 int [11, 15] 48 int [11, 16] 48 int [11, 17] 48 int [11, 18] 48 int [11, 19] 48 int [11, 20] 48 int [11, 21] 48 int [11, 22] 48 int [11, 23] 48 int [11, 24] 48 int [12, 0] 48 int [12, 1] 48 int [12, 2] 48 int [12, 3] 48 int [12, 4] 48 int [12, 5] 48 int [12, 6] 48 int [12, 7] 48 int [12, 8] 48 int [12, 9] 48 int [12, 10] 48 int [12, 11] 48 int [12, 12] 48 int [12, 13] 48 int [12, 14] 48 int [12, 15] 48 int [12, 16] 48 int [12, 17] 48 int [12, 18] 48 int [12, 19] 48 int [12, 20] 48 int [12, 21] 48 int [12, 22] 48 int [12, 23] 48 int [12, 24] 48 int [13, 0] 48 int [13, 1] 48 int [13, 2] 48 int [13, 3] 48 int [13, 4] 48 int [13, 5] 48 int [13, 6] 48 int [13, 7] 48 int [13, 8] 48 int [13, 9] 48 int [13, 10] 48 int [13, 11] 48 int [13, 12] 48 int [13, 13] 48 int [13, 14] 48 int [13, 15] 48 int [13, 16] 48 int [13, 17] 48 int [13, 18] 48 int [13, 19] 48 int [13, 20] 48 int [13, 21] 48 int [13, 22] 48 int [13, 23] 48 int [13, 24] 48 int Sorry for the lame formatting, but it's huge :P I have no idea where it's getting these values from, does anyone have an explanation? Thanks :)

Read the article

value types in the vm

- by john.rose

value types in the vm p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 14.0px Times} p.p2 {margin: 0.0px 0.0px 14.0px 0.0px; font: 14.0px Times} p.p3 {margin: 0.0px 0.0px 12.0px 0.0px; font: 14.0px Times} p.p4 {margin: 0.0px 0.0px 15.0px 0.0px; font: 14.0px Times} p.p5 {margin: 0.0px 0.0px 0.0px 0.0px; font: 14.0px Courier} p.p6 {margin: 0.0px 0.0px 0.0px 0.0px; font: 14.0px Courier; min-height: 17.0px} p.p7 {margin: 0.0px 0.0px 0.0px 0.0px; font: 14.0px Times; min-height: 18.0px} p.p8 {margin: 0.0px 0.0px 0.0px 36.0px; text-indent: -36.0px; font: 14.0px Times; min-height: 18.0px} p.p9 {margin: 0.0px 0.0px 12.0px 0.0px; font: 14.0px Times; min-height: 18.0px} p.p10 {margin: 0.0px 0.0px 12.0px 0.0px; font: 14.0px Times; color: #000000} li.li1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 14.0px Times} li.li7 {margin: 0.0px 0.0px 0.0px 0.0px; font: 14.0px Times; min-height: 18.0px} span.s1 {font: 14.0px Courier} span.s2 {color: #000000} span.s3 {font: 14.0px Courier; color: #000000} ol.ol1 {list-style-type: decimal} Or, enduring values for a changing world. Introduction A value type is a data type which, generally speaking, is designed for being passed by value in and out of methods, and stored by value in data structures. The only value types which the Java language directly supports are the eight primitive types. Java indirectly and approximately supports value types, if they are implemented in terms of classes. For example, both Integer and String may be viewed as value types, especially if their usage is restricted to avoid operations appropriate to Object. In this note, we propose a definition of value types in terms of a design pattern for Java classes, accompanied by a set of usage restrictions. We also sketch the relation of such value types to tuple types (which are a JVM-level notion), and point out JVM optimizations that can apply to value types. This note is a thought experiment to extend the JVM’s performance model in support of value types. The demonstration has two phases. Initially the extension can simply use design patterns, within the current bytecode architecture, and in today’s Java language. But if the performance model is to be realized in practice, it will probably require new JVM bytecode features, changes to the Java language, or both. We will look at a few possibilities for these new features. An Axiom of Value In the context of the JVM, a value type is a data type equipped with construction, assignment, and equality operations, and a set of typed components, such that, whenever two variables of the value type produce equal corresponding values for their components, the values of the two variables cannot be distinguished by any JVM operation. Here are some corollaries: A value type is immutable, since otherwise a copy could be constructed and the original could be modified in one of its components, allowing the copies to be distinguished. Changing the component of a value type requires construction of a new value. The equals and hashCode operations are strictly component-wise. If a value type is represented by a JVM reference, that reference cannot be successfully synchronized on, and cannot be usefully compared for reference equality. A value type can be viewed in terms of what it doesn’t do. We can say that a value type omits all value-unsafe operations, which could violate the constraints on value types. These operations, which are ordinarily allowed for Java object types, are pointer equality comparison (the acmp instruction), synchronization (the monitor instructions), all the wait and notify methods of class Object, and non-trivial finalize methods. The clone method is also value-unsafe, although for value types it could be treated as the identity function. Finally, and most importantly, any side effect on an object (however visible) also counts as an value-unsafe operation. A value type may have methods, but such methods must not change the components of the value. It is reasonable and useful to define methods like toString, equals, and hashCode on value types, and also methods which are specifically valuable to users of the value type. Representations of Value Value types have two natural representations in the JVM, unboxed and boxed. An unboxed value consists of the components, as simple variables. For example, the complex number x=(1+2i), in rectangular coordinate form, may be represented in unboxed form by the following pair of variables: /*Complex x = Complex.valueOf(1.0, 2.0):*/ double x_re = 1.0, x_im = 2.0; These variables might be locals, parameters, or fields. Their association as components of a single value is not defined to the JVM. Here is a sample computation which computes the norm of the difference between two complex numbers: double distance(/*Complex x:*/ double x_re, double x_im, /*Complex y:*/ double y_re, double y_im) { /*Complex z = x.minus(y):*/ double z_re = x_re - y_re, z_im = x_im - y_im; /*return z.abs():*/ return Math.sqrt(z_re*z_re + z_im*z_im); } A boxed representation groups component values under a single object reference. The reference is to a ‘wrapper class’ that carries the component values in its fields. (A primitive type can naturally be equated with a trivial value type with just one component of that type. In that view, the wrapper class Integer can serve as a boxed representation of value type int.) The unboxed representation of complex numbers is practical for many uses, but it fails to cover several major use cases: return values, array elements, and generic APIs. The two components of a complex number cannot be directly returned from a Java function, since Java does not support multiple return values. The same story applies to array elements: Java has no ’array of structs’ feature. (Double-length arrays are a possible workaround for complex numbers, but not for value types with heterogeneous components.) By generic APIs I mean both those which use generic types, like Arrays.asList and those which have special case support for primitive types, like String.valueOf and PrintStream.println. Those APIs do not support unboxed values, and offer some problems to boxed values. Any ’real’ JVM type should have a story for returns, arrays, and API interoperability. The basic problem here is that value types fall between primitive types and object types. Value types are clearly more complex than primitive types, and object types are slightly too complicated. Objects are a little bit dangerous to use as value carriers, since object references can be compared for pointer equality, and can be synchronized on. Also, as many Java programmers have observed, there is often a performance cost to using wrapper objects, even on modern JVMs. Even so, wrapper classes are a good starting point for talking about value types. If there were a set of structural rules and restrictions which would prevent value-unsafe operations on value types, wrapper classes would provide a good notation for defining value types. This note attempts to define such rules and restrictions. Let’s Start Coding Now it is time to look at some real code. Here is a definition, written in Java, of a complex number value type. @ValueSafe public final class Complex implements java.io.Serializable { // immutable component structure: public final double re, im; private Complex(double re, double im) { this.re = re; this.im = im; } // interoperability methods: public String toString() { return "Complex("+re+","+im+")"; } public List<Double> asList() { return Arrays.asList(re, im); } public boolean equals(Complex c) { return re == c.re && im == c.im; } public boolean equals(@ValueSafe Object x) { return x instanceof Complex && equals((Complex) x); } public int hashCode() { return 31*Double.valueOf(re).hashCode() + Double.valueOf(im).hashCode(); } // factory methods: public static Complex valueOf(double re, double im) { return new Complex(re, im); } public Complex changeRe(double re2) { return valueOf(re2, im); } public Complex changeIm(double im2) { return valueOf(re, im2); } public static Complex cast(@ValueSafe Object x) { return x == null ? ZERO : (Complex) x; } // utility methods and constants: public Complex plus(Complex c) { return new Complex(re+c.re, im+c.im); } public Complex minus(Complex c) { return new Complex(re-c.re, im-c.im); } public double abs() { return Math.sqrt(re*re + im*im); } public static final Complex PI = valueOf(Math.PI, 0.0); public static final Complex ZERO = valueOf(0.0, 0.0); } This is not a minimal definition, because it includes some utility methods and other optional parts. The essential elements are as follows: The class is marked as a value type with an annotation. The class is final, because it does not make sense to create subclasses of value types. The fields of the class are all non-private and final. (I.e., the type is immutable and structurally transparent.) From the supertype Object, all public non-final methods are overridden. The constructor is private. Beyond these bare essentials, we can observe the following features in this example, which are likely to be typical of all value types: One or more factory methods are responsible for value creation, including a component-wise valueOf method. There are utility methods for complex arithmetic and instance creation, such as plus and changeIm. There are static utility constants, such as PI. The type is serializable, using the default mechanisms. There are methods for converting to and from dynamically typed references, such as asList and cast. The Rules In order to use value types properly, the programmer must avoid value-unsafe operations. A helpful Java compiler should issue errors (or at least warnings) for code which provably applies value-unsafe operations, and should issue warnings for code which might be correct but does not provably avoid value-unsafe operations. No such compilers exist today, but to simplify our account here, we will pretend that they do exist. A value-safe type is any class, interface, or type parameter marked with the @ValueSafe annotation, or any subtype of a value-safe type. If a value-safe class is marked final, it is in fact a value type. All other value-safe classes must be abstract. The non-static fields of a value class must be non-public and final, and all its constructors must be private. Under the above rules, a standard interface could be helpful to define value types like Complex. Here is an example: @ValueSafe public interface ValueType extends java.io.Serializable { // All methods listed here must get redefined. // Definitions must be value-safe, which means // they may depend on component values only. List<? extends Object> asList(); int hashCode(); boolean equals(@ValueSafe Object c); String toString(); } //@ValueSafe inherited from supertype: public final class Complex implements ValueType { … The main advantage of such a conventional interface is that (unlike an annotation) it is reified in the runtime type system. It could appear as an element type or parameter bound, for facilities which are designed to work on value types only. More broadly, it might assist the JVM to perform dynamic enforcement of the rules for value types. Besides types, the annotation @ValueSafe can mark fields, parameters, local variables, and methods. (This is redundant when the type is also value-safe, but may be useful when the type is Object or another supertype of a value type.) Working forward from these annotations, an expression E is defined as value-safe if it satisfies one or more of the following: The type of E is a value-safe type. E names a field, parameter, or local variable whose declaration is marked @ValueSafe. E is a call to a method whose declaration is marked @ValueSafe. E is an assignment to a value-safe variable, field reference, or array reference. E is a cast to a value-safe type from a value-safe expression. E is a conditional expression E0 ? E1 : E2, and both E1 and E2 are value-safe. Assignments to value-safe expressions and initializations of value-safe names must take their values from value-safe expressions. A value-safe expression may not be the subject of a value-unsafe operation. In particular, it cannot be synchronized on, nor can it be compared with the “==” operator, not even with a null or with another value-safe type. In a program where all of these rules are followed, no value-type value will be subject to a value-unsafe operation. Thus, the prime axiom of value types will be satisfied, that no two value type will be distinguishable as long as their component values are equal. More Code To illustrate these rules, here are some usage examples for Complex: Complex pi = Complex.valueOf(Math.PI, 0); Complex zero = pi.changeRe(0); //zero = pi; zero.re = 0; ValueType vtype = pi; @SuppressWarnings("value-unsafe") Object obj = pi; @ValueSafe Object obj2 = pi; obj2 = new Object(); // ok List<Complex> clist = new ArrayList<Complex>(); clist.add(pi); // (ok assuming List.add param is @ValueSafe) List<ValueType> vlist = new ArrayList<ValueType>(); vlist.add(pi); // (ok) List<Object> olist = new ArrayList<Object>(); olist.add(pi); // warning: "value-unsafe" boolean z = pi.equals(zero); boolean z1 = (pi == zero); // error: reference comparison on value type boolean z2 = (pi == null); // error: reference comparison on value type boolean z3 = (pi == obj2); // error: reference comparison on value type synchronized (pi) { } // error: synch of value, unpredictable result synchronized (obj2) { } // unpredictable result Complex qq = pi; qq = null; // possible NPE; warning: “null-unsafe" qq = (Complex) obj; // warning: “null-unsafe" qq = Complex.cast(obj); // OK @SuppressWarnings("null-unsafe") Complex empty = null; // possible NPE qq = empty; // possible NPE (null pollution) The Payoffs It follows from this that either the JVM or the java compiler can replace boxed value-type values with unboxed ones, without affecting normal computations. Fields and variables of value types can be split into their unboxed components. Non-static methods on value types can be transformed into static methods which take the components as value parameters. Some common questions arise around this point in any discussion of value types. Why burden the programmer with all these extra rules? Why not detect programs automagically and perform unboxing transparently? The answer is that it is easy to break the rules accidently unless they are agreed to by the programmer and enforced. Automatic unboxing optimizations are tantalizing but (so far) unreachable ideal. In the current state of the art, it is possible exhibit benchmarks in which automatic unboxing provides the desired effects, but it is not possible to provide a JVM with a performance model that assures the programmer when unboxing will occur. This is why I’m writing this note, to enlist help from, and provide assurances to, the programmer. Basically, I’m shooting for a good set of user-supplied “pragmas” to frame the desired optimization. Again, the important thing is that the unboxing must be done reliably, or else programmers will have no reason to work with the extra complexity of the value-safety rules. There must be a reasonably stable performance model, wherein using a value type has approximately the same performance characteristics as writing the unboxed components as separate Java variables. There are some rough corners to the present scheme. Since Java fields and array elements are initialized to null, value-type computations which incorporate uninitialized variables can produce null pointer exceptions. One workaround for this is to require such variables to be null-tested, and the result replaced with a suitable all-zero value of the value type. That is what the “cast” method does above. Generically typed APIs like List<T> will continue to manipulate boxed values always, at least until we figure out how to do reification of generic type instances. Use of such APIs will elicit warnings until their type parameters (and/or relevant members) are annotated or typed as value-safe. Retrofitting List<T> is likely to expose flaws in the present scheme, which we will need to engineer around. Here are a couple of first approaches: public interface java.util.List<@ValueSafe T> extends Collection<T> { … public interface java.util.List<T extends Object|ValueType> extends Collection<T> { … (The second approach would require disjunctive types, in which value-safety is “contagious” from the constituent types.) With more transformations, the return value types of methods can also be unboxed. This may require significant bytecode-level transformations, and would work best in the presence of a bytecode representation for multiple value groups, which I have proposed elsewhere under the title “Tuples in the VM”. But for starters, the JVM can apply this transformation under the covers, to internally compiled methods. This would give a way to express multiple return values and structured return values, which is a significant pain-point for Java programmers, especially those who work with low-level structure types favored by modern vector and graphics processors. The lack of multiple return values has a strong distorting effect on many Java APIs. Even if the JVM fails to unbox a value, there is still potential benefit to the value type. Clustered computing systems something have copy operations (serialization or something similar) which apply implicitly to command operands. When copying JVM objects, it is extremely helpful to know when an object’s identity is important or not. If an object reference is a copied operand, the system may have to create a proxy handle which points back to the original object, so that side effects are visible. Proxies must be managed carefully, and this can be expensive. On the other hand, value types are exactly those types which a JVM can “copy and forget” with no downside. Array types are crucial to bulk data interfaces. (As data sizes and rates increase, bulk data becomes more important than scalar data, so arrays are definitely accompanying us into the future of computing.) Value types are very helpful for adding structure to bulk data, so a successful value type mechanism will make it easier for us to express richer forms of bulk data. Unboxing arrays (i.e., arrays containing unboxed values) will provide better cache and memory density, and more direct data movement within clustered or heterogeneous computing systems. They require the deepest transformations, relative to today’s JVM. There is an impedance mismatch between value-type arrays and Java’s covariant array typing, so compromises will need to be struck with existing Java semantics. It is probably worth the effort, since arrays of unboxed value types are inherently more memory-efficient than standard Java arrays, which rely on dependent pointer chains. It may be sufficient to extend the “value-safe” concept to array declarations, and allow low-level transformations to change value-safe array declarations from the standard boxed form into an unboxed tuple-based form. Such value-safe arrays would not be convertible to Object[] arrays. Certain connection points, such as Arrays.copyOf and System.arraycopy might need additional input/output combinations, to allow smooth conversion between arrays with boxed and unboxed elements. Alternatively, the correct solution may have to wait until we have enough reification of generic types, and enough operator overloading, to enable an overhaul of Java arrays. Implicit Method Definitions The example of class Complex above may be unattractively complex. I believe most or all of the elements of the example class are required by the logic of value types. If this is true, a programmer who writes a value type will have to write lots of error-prone boilerplate code. On the other hand, I think nearly all of the code (except for the domain-specific parts like plus and minus) can be implicitly generated. Java has a rule for implicitly defining a class’s constructor, if no it defines no constructors explicitly. Likewise, there are rules for providing default access modifiers for interface members. Because of the highly regular structure of value types, it might be reasonable to perform similar implicit transformations on value types. Here’s an example of a “highly implicit” definition of a complex number type: public class Complex implements ValueType { // implicitly final public double re, im; // implicitly public final //implicit methods are defined elementwise from te fields: // toString, asList, equals(2), hashCode, valueOf, cast //optionally, explicit methods (plus, abs, etc.) would go here } In other words, with the right defaults, a simple value type definition can be a one-liner. The observant reader will have noticed the similarities (and suitable differences) between the explicit methods above and the corresponding methods for List<T>. Another way to abbreviate such a class would be to make an annotation the primary trigger of the functionality, and to add the interface(s) implicitly: public @ValueType class Complex { … // implicitly final, implements ValueType (But to me it seems better to communicate the “magic” via an interface, even if it is rooted in an annotation.) Implicitly Defined Value Types So far we have been working with nominal value types, which is to say that the sequence of typed components is associated with a name and additional methods that convey the intention of the programmer. A simple ordered pair of floating point numbers can be variously interpreted as (to name a few possibilities) a rectangular or polar complex number or Cartesian point. The name and the methods convey the intended meaning. But what if we need a truly simple ordered pair of floating point numbers, without any further conceptual baggage? Perhaps we are writing a method (like “divideAndRemainder”) which naturally returns a pair of numbers instead of a single number. Wrapping the pair of numbers in a nominal type (like “QuotientAndRemainder”) makes as little sense as wrapping a single return value in a nominal type (like “Quotient”). What we need here are structural value types commonly known as tuples. For the present discussion, let us assign a conventional, JVM-friendly name to tuples, roughly as follows: public class java.lang.tuple.$DD extends java.lang.tuple.Tuple { double $1, $2; } Here the component names are fixed and all the required methods are defined implicitly. The supertype is an abstract class which has suitable shared declarations. The name itself mentions a JVM-style method parameter descriptor, which may be “cracked” to determine the number and types of the component fields. The odd thing about such a tuple type (and structural types in general) is it must be instantiated lazily, in response to linkage requests from one or more classes that need it. The JVM and/or its class loaders must be prepared to spin a tuple type on demand, given a simple name reference, $xyz, where the xyz is cracked into a series of component types. (Specifics of naming and name mangling need some tasteful engineering.) Tuples also seem to demand, even more than nominal types, some support from the language. (This is probably because notations for non-nominal types work best as combinations of punctuation and type names, rather than named constructors like Function3 or Tuple2.) At a minimum, languages with tuples usually (I think) have some sort of simple bracket notation for creating tuples, and a corresponding pattern-matching syntax (or “destructuring bind”) for taking tuples apart, at least when they are parameter lists. Designing such a syntax is no simple thing, because it ought to play well with nominal value types, and also with pre-existing Java features, such as method parameter lists, implicit conversions, generic types, and reflection. That is a task for another day. Other Use Cases Besides complex numbers and simple tuples there are many use cases for value types. Many tuple-like types have natural value-type representations. These include rational numbers, point locations and pixel colors, and various kinds of dates and addresses. Other types have a variable-length ‘tail’ of internal values. The most common example of this is String, which is (mathematically) a sequence of UTF-16 character values. Similarly, bit vectors, multiple-precision numbers, and polynomials are composed of sequences of values. Such types include, in their representation, a reference to a variable-sized data structure (often an array) which (somehow) represents the sequence of values. The value type may also include ’header’ information. Variable-sized values often have a length distribution which favors short lengths. In that case, the design of the value type can make the first few values in the sequence be direct ’header’ fields of the value type. In the common case where the header is enough to represent the whole value, the tail can be a shared null value, or even just a null reference. Note that the tail need not be an immutable object, as long as the header type encapsulates it well enough. This is the case with String, where the tail is a mutable (but never mutated) character array. Field types and their order must be a globally visible part of the API. The structure of the value type must be transparent enough to have a globally consistent unboxed representation, so that all callers and callees agree about the type and order of components that appear as parameters, return types, and array elements. This is a trade-off between efficiency and encapsulation, which is forced on us when we remove an indirection enjoyed by boxed representations. A JVM-only transformation would not care about such visibility, but a bytecode transformation would need to take care that (say) the components of complex numbers would not get swapped after a redefinition of Complex and a partial recompile. Perhaps constant pool references to value types need to declare the field order as assumed by each API user. This brings up the delicate status of private fields in a value type. It must always be possible to load, store, and copy value types as coordinated groups, and the JVM performs those movements by moving individual scalar values between locals and stack. If a component field is not public, what is to prevent hostile code from plucking it out of the tuple using a rogue aload or astore instruction? Nothing but the verifier, so we may need to give it more smarts, so that it treats value types as inseparable groups of stack slots or locals (something like long or double). My initial thought was to make the fields always public, which would make the security problem moot. But public is not always the right answer; consider the case of String, where the underlying mutable character array must be encapsulated to prevent security holes. I believe we can win back both sides of the tradeoff, by training the verifier never to split up the components in an unboxed value. Just as the verifier encapsulates the two halves of a 64-bit primitive, it can encapsulate the the header and body of an unboxed String, so that no code other than that of class String itself can take apart the values. Similar to String, we could build an efficient multi-precision decimal type along these lines: public final class DecimalValue extends ValueType { protected final long header; protected private final BigInteger digits; public DecimalValue valueOf(int value, int scale) { assert(scale >= 0); return new DecimalValue(((long)value << 32) + scale, null); } public DecimalValue valueOf(long value, int scale) { if (value == (int) value) return valueOf((int)value, scale); return new DecimalValue(-scale, new BigInteger(value)); } } Values of this type would be passed between methods as two machine words. Small values (those with a significand which fits into 32 bits) would be represented without any heap data at all, unless the DecimalValue itself were boxed. (Note the tension between encapsulation and unboxing in this case. It would be better if the header and digits fields were private, but depending on where the unboxing information must “leak”, it is probably safer to make a public revelation of the internal structure.) Note that, although an array of Complex can be faked with a double-length array of double, there is no easy way to fake an array of unboxed DecimalValues. (Either an array of boxed values or a transposed pair of homogeneous arrays would be reasonable fallbacks, in a current JVM.) Getting the full benefit of unboxing and arrays will require some new JVM magic. Although the JVM emphasizes portability, system dependent code will benefit from using machine-level types larger than 64 bits. For example, the back end of a linear algebra package might benefit from value types like Float4 which map to stock vector types. This is probably only worthwhile if the unboxing arrays can be packed with such values. More Daydreams A more finely-divided design for dynamic enforcement of value safety could feature separate marker interfaces for each invariant. An empty marker interface Unsynchronizable could cause suitable exceptions for monitor instructions on objects in marked classes. More radically, a Interchangeable marker interface could cause JVM primitives that are sensitive to object identity to raise exceptions; the strangest result would be that the acmp instruction would have to be specified as raising an exception. @ValueSafe public interface ValueType extends java.io.Serializable, Unsynchronizable, Interchangeable { … public class Complex implements ValueType { // inherits Serializable, Unsynchronizable, Interchangeable, @ValueSafe … It seems possible that Integer and the other wrapper types could be retro-fitted as value-safe types. This is a major change, since wrapper objects would be unsynchronizable and their references interchangeable. It is likely that code which violates value-safety for wrapper types exists but is uncommon. It is less plausible to retro-fit String, since the prominent operation String.intern is often used with value-unsafe code. We should also reconsider the distinction between boxed and unboxed values in code. The design presented above obscures that distinction. As another thought experiment, we could imagine making a first class distinction in the type system between boxed and unboxed representations. Since only primitive types are named with a lower-case initial letter, we could define that the capitalized version of a value type name always refers to the boxed representation, while the initial lower-case variant always refers to boxed. For example: complex pi = complex.valueOf(Math.PI, 0); Complex boxPi = pi; // convert to boxed myList.add(boxPi); complex z = myList.get(0); // unbox Such a convention could perhaps absorb the current difference between int and Integer, double and Double. It might also allow the programmer to express a helpful distinction among array types. As said above, array types are crucial to bulk data interfaces, but are limited in the JVM. Extending arrays beyond the present limitations is worth thinking about; for example, the Maxine JVM implementation has a hybrid object/array type. Something like this which can also accommodate value type components seems worthwhile. On the other hand, does it make sense for value types to contain short arrays? And why should random-access arrays be the end of our design process, when bulk data is often sequentially accessed, and it might make sense to have heterogeneous streams of data as the natural “jumbo” data structure. These considerations must wait for another day and another note. More Work It seems to me that a good sequence for introducing such value types would be as follows: Add the value-safety restrictions to an experimental version of javac. Code some sample applications with value types, including Complex and DecimalValue. Create an experimental JVM which internally unboxes value types but does not require new bytecodes to do so. Ensure the feasibility of the performance model for the sample applications. Add tuple-like bytecodes (with or without generic type reification) to a major revision of the JVM, and teach the Java compiler to switch in the new bytecodes without code changes. A staggered roll-out like this would decouple language changes from bytecode changes, which is always a convenient thing. A similar investigation should be applied (concurrently) to array types. In this case, it seems to me that the starting point is in the JVM: Add an experimental unboxing array data structure to a production JVM, perhaps along the lines of Maxine hybrids. No bytecode or language support is required at first; everything can be done with encapsulated unsafe operations and/or method handles. Create an experimental JVM which internally unboxes value types but does not require new bytecodes to do so. Ensure the feasibility of the performance model for the sample applications. Add tuple-like bytecodes (with or without generic type reification) to a major revision of the JVM, and teach the Java compiler to switch in the new bytecodes without code changes. That’s enough musing me for now. Back to work!

Read the article

Unusually high memory usage on a CentOS VPS with 512 guaranteed RAM

- by Andrei Bârsan

I'm working on a medium-sized web application written in PHP that's running on a VPS with 512mb ram. The webapp hasn't been officially launched yet, so there isn't too much traffic going on, just me and a few other people working on it. There is another slightly smaller webapp also hosted on this machine, among 4-5 other small static sites. We are running Centos 5 32-bit & cPanel/WHM. This is the result of running ps aux and, as you can see, it's not using 100% of the RAM. However, on the hypanel overview, it's always shown as using aroun 500MB ram, just for running apache, mysql, and the lowest-memory-footprint versions of the mail server, ftp server etc. -bash-3.2# ps aux USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 1 0.0 0.0 2156 664 ? Ss 12:08 0:00 init [3] root 1123 0.0 0.0 2260 548 ? S<s 12:08 0:00 /sbin/udevd -d root 1462 0.0 0.0 1812 568 ? Ss 12:08 0:00 syslogd -m 0 named 1496 0.0 0.0 3808 820 ? Ss 12:08 0:00 nsd named 1497 0.0 0.0 10672 756 ? S 12:08 0:00 nsd named 1499 0.0 0.0 3880 584 ? S 12:08 0:00 nsd root 1514 0.0 0.1 7240 1064 ? Ss 12:08 0:00 /usr/sbin/sshd root 1522 0.0 0.0 2832 832 ? Ss 12:08 0:00 xinetd -stayalive -pidfile /var/run/xinetd.pid root 1534 0.0 0.1 3712 1328 ? S 12:08 0:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql - mysql 1667 0.0 2.9 225680 30884 ? Sl 12:08 0:00 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql - mailnull 1766 0.0 0.1 9352 1100 ? Ss 12:08 0:00 /usr/sbin/exim -bd -q60m root 1797 0.0 0.0 2156 708 ? Ss 12:08 0:00 /usr/sbin/dovecot root 1798 0.0 0.0 2632 1012 ? S 12:08 0:00 dovecot-auth root 1816 0.0 3.0 38580 32456 ? Ss 12:08 0:01 /usr/local/bin/spamd -d --allowed-ips=127.0.0.1 --pidfi root 1839 0.0 1.6 63200 17496 ? Ss 12:08 0:00 /usr/local/apache/bin/httpd -k start -DSSL root 1846 0.0 0.1 5416 1468 ? Ss 12:08 0:00 pure-ftpd (SERVER) root 1848 0.0 0.1 6212 1244 ? S 12:08 0:00 /usr/sbin/pure-authd -s /var/run/ftpd.sock -r /usr/sbin root 1856 0.0 0.1 4492 1112 ? Ss 12:08 0:00 crond root 1864 0.0 0.0 2356 428 ? Ss 12:08 0:00 /usr/sbin/atd dovecot 1927 0.0 0.1 5196 1952 ? S 12:08 0:00 pop3-login dovecot 1928 0.0 0.1 5196 1948 ? S 12:08 0:00 pop3-login dovecot 1929 0.0 0.1 5316 2012 ? S 12:08 0:00 imap-login dovecot 1930 0.0 0.2 5416 2228 ? S 12:08 0:00 imap-login root 1939 0.0 0.1 3936 1964 ? S 12:08 0:00 cPhulkd - processor root 1963 0.0 0.8 15876 8564 ? S 12:08 0:00 cpsrvd (SSL) - waiting for connections root 1966 0.0 0.7 15172 7748 ? S 12:08 0:00 cpdavd - accepting connections on 2077 and 2078 root 1990 0.0 0.2 5008 3136 ? S 12:08 0:00 queueprocd - wait to process a task root 2017 0.0 2.9 38580 31020 ? S 12:08 0:00 spamd child root 2018 0.0 0.5 8904 5636 ? S 12:08 0:00 /usr/bin/perl /usr/local/cpanel/bin/leechprotect nobody 2021 0.0 3.2 66512 33724 ? S 12:08 0:00 /usr/local/apache/bin/httpd -k start -DSSL nobody 2022 0.0 3.1 67812 33024 ? S 12:08 0:00 /usr/local/apache/bin/httpd -k start -DSSL nobody 2024 0.0 1.9 64364 20680 ? S 12:08 0:00 /usr/local/apache/bin/httpd -k start -DSSL root 2027 0.0 0.4 9000 4540 ? S 12:08 0:00 tailwatchd root 2032 0.0 0.1 4176 1836 ? SN 12:08 0:00 cpanellogd - sleeping for logs nobody 3096 0.0 1.9 64572 20264 ? S 12:09 0:00 /usr/local/apache/bin/httpd -k start -DSSL nobody 3097 0.0 2.8 66008 30136 ? S 12:09 0:00 /usr/local/apache/bin/httpd -k start -DSSL nobody 3098 0.0 2.8 65704 29752 ? S 12:09 0:00 /usr/local/apache/bin/httpd -k start -DSSL nobody 3099 0.0 3.1 67260 32816 ? S 12:09 0:00 /usr/local/apache/bin/httpd -k start -DSSL andrei 3448 0.0 0.1 3204 1632 ? S 12:50 0:00 imap nobody 3537 0.0 1.9 64308 20108 ? S 13:01 0:00 /usr/local/apache/bin/httpd -k start -DSSL nobody 3614 0.0 1.9 64576 20628 ? S 13:10 0:00 /usr/local/apache/bin/httpd -k start -DSSL nobody 3615 0.0 1.3 63200 14672 ? S 13:10 0:00 /usr/local/apache/bin/httpd -k start -DSSL root 3626 0.0 0.2 10232 2964 ? Rs 13:14 0:00 sshd: root@pts/0 root 3648 0.0 0.1 3844 1600 pts/0 Ss 13:14 0:00 -bash root 3826 0.0 0.0 2532 908 pts/0 R+ 13:21 0:00 ps aux Lately, without any significant changes to the configuration, the memory usage started peaking and going over 512, causing the virtual server to kill apache, basically murdering our site in the process. Do you have any idea if this is normal and more resources should be acquired? I don't think... since there isn't too much data or traffic online yet.

Read the article

How to design a high-level application protocol for metadata syncing between devices and server?

- by Jaanus

I am looking for guidance on how to best think about designing a high-level application protocol to sync metadata between end-user devices and a server. My goal: the user can interact with the application data on any device, or on the web. The purpose of this protocol is to communicate changes made on one endpoint to other endpoints through the server, and ensure all devices maintain a consistent picture of the application data. If user makes changes on one device or on the web, the protocol will push data to the central repository, from where other devices can pull it. Some other design thoughts: I call it "metadata syncing" because the payloads will be quite small, in the form of object IDs and small metadata about those ID-s. When client endpoints retrieve new metadata over this protocol, they will fetch actual object data from an external source based on this metadata. Fetching the "real" object data is out of scope, I'm only talking about metadata syncing here. Using HTTP for transport and JSON for payload container. The question is basically about how to best design the JSON payload schema. I want this to be easy to implement and maintain on the web and across desktop and mobile devices. The best approach feels to be simple timer- or event-based HTTP request/response without any persistent channels. Also, you should not have a PhD to read it, and I want my spec to fit on 2 pages, not 200. Authentication and security are out of scope for this question: assume that the requests are secure and authenticated. The goal is eventual consistency of data on devices, it is not entirely realtime. For example, user can make changes on one device while being offline. When going online again, user would perform "sync" operation to push local changes and retrieve remote changes. Having said that, the protocol should support both of these modes of operation: Starting from scratch on a device, should be able to pull the whole metadata picture "sync as you go". When looking at the data on two devices side by side and making changes, should be easy to push those changes as short individual messages which the other device can receive near-realtime (subject to when it decides to contact server for sync). As a concrete example, you can think of Dropbox (it is not what I'm working on, but it helps to understand the model): on a range of devices, the user can manage a files and folders—move them around, create new ones, remove old ones etc. And in my context the "metadata" would be the file and folder structure, but not the actual file contents. And metadata fields would be something like file/folder name and time of modification (all devices should see the same time of modification). Another example is IMAP. I have not read the protocol, but my goals (minus actual message bodies) are the same. Feels like there are two grand approaches how this is done: transactional messages. Each change in the system is expressed as delta and endpoints communicate with those deltas. Example: DVCS changesets. REST: communicating the object graph as a whole or in part, without worrying so much about the individual atomic changes. What I would like in the answers: Is there anything important I left out above? Constraints, goals? What is some good background reading on this? (I realize this is what many computer science courses talk about at great length and detail... I am hoping to short-circuit it by looking at some crash course or nuggets.) What are some good examples of such protocols that I could model after, or even use out of box? (I mention Dropbox and IMAP above... I should probably read the IMAP RFC.)

Read the article

Parallelism in .NET – Part 20, Using Task with Existing APIs

- by Reed

Although the Task class provides a huge amount of flexibility for handling asynchronous actions, the .NET Framework still contains a large number of APIs that are based on the previous asynchronous programming model. While Task and Task<T> provide a much nicer syntax as well as extending the flexibility, allowing features such as continuations based on multiple tasks, the existing APIs don’t directly support this workflow. There is a method in the TaskFactory class which can be used to adapt the existing APIs to the new Task class: TaskFactory.FromAsync. This method provides a way to convert from the BeginOperation/EndOperation method pair syntax common through .NET Framework directly to a Task<T> containing the results of the operation in the task’s Result parameter. While this method does exist, it unfortunately comes at a cost – the method overloads are far from simple to decipher, and the resulting code is not always as easily understood as newer code based directly on the Task class. For example, a single call to handle WebRequest.BeginGetResponse/EndGetReponse, one of the easiest “pairs” of methods to use, looks like the following: var task = Task.Factory.FromAsync<WebResponse>( request.BeginGetResponse, request.EndGetResponse, null); .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } The compiler is unfortunately unable to infer the correct type, and, as a result, the WebReponse must be explicitly mentioned in the method call. As a result, I typically recommend wrapping this into an extension method to ease use. For example, I would place the above in an extension method like: public static class WebRequestExtensions { public static Task<WebResponse> GetReponseAsync(this WebRequest request) { return Task.Factory.FromAsync<WebResponse>( request.BeginGetResponse, request.EndGetResponse, null); } } This dramatically simplifies usage. For example, if we wanted to asynchronously check to see if this blog supported XHTML 1.0, and report that in a text box to the user, we could do: var webRequest = WebRequest.Create("http://www.reedcopsey.com"); webRequest.GetReponseAsync().ContinueWith(t => { using (var sr = new StreamReader(t.Result.GetResponseStream())) { string str = sr.ReadLine();; this.textBox1.Text = string.Format("Page at {0} supports XHTML 1.0: {1}", t.Result.ResponseUri, str.Contains("XHTML 1.0")); } }, TaskScheduler.FromCurrentSynchronizationContext()); By using a continuation with a TaskScheduler based on the current synchronization context, we can keep this request asynchronous, check based on the first line of the response string, and report the results back on our UI directly.

Read the article

PowerShell Control over Nikon D3000 Camera

My wife got me a Nikon D3000 camera for Christmas last year, and Im loving it but still trying to wrap my head around some of its features. For instance, when you plug it into a computer via USB, it doesnt show up as a drive like most cameras Ive used to, but rather it shows up as Computer\D3000. After a bit of research, Ive learned that this is because it implements the MTP/PTP protocol, and thus doesnt actually let Windows mount the cameras storage as a drive letter. Nikon describes the use of the MTP and PTP protocols in their cameras here. What Im really trying to do is gain access to the cameras file system via PowerShell. Ive been using a very handy PowerShell script to pull pictures off of my cameras and organize them into folders by date. Id love to be able to do the same thing with my Nikon D3000, but so far I havent been able to figure out how to get access to the files in PowerShell. If you know, Id appreciate any links/tips you can provide. All I could find is a shareware product called PTPdrive, which Im not prepared to shell out money for (yet). (and yes you can do much the same thing with Windows 7s Import Pictures and Videos wizard, which is pretty good too) However, in my searching, I did find some really cool stuff you can do with PowerShell and one of these cameras, like actually taking pictures via PowerShell commands. Credit for this goes to James ONeill and Mark Wilson. Heres what I was able to do: Taking Pictures via PowerShell with D3000 First, connect your camera, turn it on, and launch PowerShell. Execute the following commands to see what commands your device supports. $dialog = New-Object -ComObject "WIA.CommonDialog" $device = $dialog.ShowSelectDevice() $device.Commands You should see something like this: Now, to take a picture, simply point your camera at something and then execute this command: $device.ExecuteCommand("{AF933CAC-ACAD-11D2-A093-00C04F72DC3C}") .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } Imagine my surprise when this actually took a picture (with auto-focus): Imagine what you could do with a camera completely under the control of your computer Time-lapse photography would be pretty simple, for instance, with a very simple loop that takes a picture and then sleeps for a minute (or whatever time period). Hooked up to a laptop for portability (and an A/C power supply), this would be pretty trivial to implement. I may have to give it a shot and report back. Did you know that DotNetSlackers also publishes .net articles written by top known .net Authors? We already have over 80 articles in several categories including Silverlight. Take a look: here.

Read the article

Parallelism in .NET – Part 19, TaskContinuationOptions

- by Reed

My introduction to Task continuations demonstrates continuations on the Task class. In addition, I’ve shown how continuations allow handling of multiple tasks in a clean, concise manner. Continuations can also be used to handle exceptional situations using a clean, simple syntax. In addition to standard Task continuations , the Task class provides some options for filtering continuations automatically. This is handled via the TaskContinationOptions enumeration, which provides hints to the TaskScheduler that it should only continue based on the operation of the antecedent task. This is especially useful when dealing with exceptions. For example, we can extend the sample from our earlier continuation discussion to include support for handling exceptions thrown by the Factorize method: // Get a copy of the UI-thread task scheduler up front to use later var uiScheduler = TaskScheduler.FromCurrentSynchronizationContext(); // Start our task var factorize = Task.Factory.StartNew( () => { int primeFactor1 = 0; int primeFactor2 = 0; bool result = Factorize(10298312, ref primeFactor1, ref primeFactor2); return new { Result = result, Factor1 = primeFactor1, Factor2 = primeFactor2 }; }); // When we succeed, report the results to the UI factorize.ContinueWith(task => textBox1.Text = string.Format("{0}/{1} [Succeeded {2}]", task.Result.Factor1, task.Result.Factor2, task.Result.Result), CancellationToken.None, TaskContinuationOptions.NotOnFaulted, uiScheduler); // When we have an exception, report it factorize.ContinueWith(task => textBox1.Text = string.Format("Error: {0}", task.Exception.Message), CancellationToken.None, TaskContinuationOptions.OnlyOnFaulted, uiScheduler); .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } The above code works by using a combination of features. First, we schedule our task, the same way as in the previous example. However, in this case, we use a different overload of Task.ContinueWith which allows us to specify both a specific TaskScheduler (in order to have your continuation run on the UI’s synchronization context) as well as a TaskContinuationOption. In the first continuation, we tell the continuation that we only want it to run when there was not an exception by specifying TaskContinuationOptions.NotOnFaulted. When our factorize task completes successfully, this continuation will automatically run on the UI thread, and provide the appropriate feedback. However, if the factorize task has an exception – for example, if the Factorize method throws an exception due to an improper input value, the second continuation will run. This occurs due to the specification of TaskContinuationOptions.OnlyOnFaulted in the options. In this case, we’ll report the error received to the user. We can use TaskContinuationOptions to filter our continuations by whether or not an exception occurred and whether or not a task was cancelled. This allows us to handle many situations, and is especially useful when trying to maintain a valid application state without ever blocking the user interface. The same concepts can be extended even further, and allow you to chain together many tasks based on the success of the previous ones. Continuations can even be used to create a state machine with full error handling, all without blocking the user interface thread.

Read the article

Generic Sorting using C# and Lambda Expression

- by Haitham Khedre

Download : GenericSortTester.zip I worked in this class from long time and I think it is a nice piece of code that I need to share , it might help other people searching for the same concept. this will help you to sort any collection easily without needing to write special code for each data type , however if you need special ordering you still can do it , leave a comment and I will see if I need to write another article to cover the other cases. I attached also a fully working example to make you able to see how do you will use that . public static class GenericSorter { public static IOrderedEnumerable<T> Sort<T>(IEnumerable<T> toSort, Dictionary<string, SortingOrder> sortOptions) { IOrderedEnumerable<T> orderedList = null; foreach (KeyValuePair<string, SortingOrder> entry in sortOptions) { if (orderedList != null) { if (entry.Value == SortingOrder.Ascending) { orderedList = orderedList.ApplyOrder<T>(entry.Key, "ThenBy"); } else { orderedList = orderedList.ApplyOrder<T>(entry.Key,"ThenByDescending"); } } else { if (entry.Value == SortingOrder.Ascending) { orderedList = toSort.ApplyOrder<T>(entry.Key, "OrderBy"); } else { orderedList = toSort.ApplyOrder<T>(entry.Key, "OrderByDescending"); } } } return orderedList; } private static IOrderedEnumerable<T> ApplyOrder<T> (this IEnumerable<T> source, string property, string methodName) { ParameterExpression param = Expression.Parameter(typeof(T), "x"); Expression expr = param; foreach (string prop in property.Split('.')) { expr = Expression.PropertyOrField(expr, prop); } Type delegateType = typeof(Func<,>).MakeGenericType(typeof(T), expr.Type); LambdaExpression lambda = Expression.Lambda(delegateType, expr, param); MethodInfo mi = typeof(Enumerable).GetMethods().Single( method => method.Name == methodName && method.IsGenericMethodDefinition && method.GetGenericArguments().Length == 2 && method.GetParameters().Length == 2) .MakeGenericMethod(typeof(T), expr.Type); return (IOrderedEnumerable<T>)mi.Invoke (null, new object[] { source, lambda.Compile() }); } } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; }

Read the article

Implicit and Explicit implementations for Multiple Interface inheritance

Following C#.NET demo explains you all the scenarios for implementation of Interface methods to classes. There are two ways you can implement a interface method to a class. 1. Implicit Implementation 2. Explicit Implementation. Please go though the sample. using System; namespace ImpExpTest { class Program { static void Main(string[] args) { C o3 = new C(); Console.WriteLine(o3.fu()); I1 o1 = new C(); Console.WriteLine(o1.fu()); I2 o2 = new C(); Console.WriteLine(o2.fu()); var o4 = new C(); //var is considered as C Console.WriteLine(o4.fu()); var o5 = (I1)new C(); //var is considered as I1 Console.WriteLine(o5.fu()); var o6 = (I2)new C(); //var is considered as I2 Console.WriteLine(o6.fu()); D o7 = new D(); Console.WriteLine(o7.fu()); I1 o8 = new D(); Console.WriteLine(o8.fu()); I2 o9 = new D(); Console.WriteLine(o9.fu()); } } interface I1 { string fu(); } interface I2 { string fu(); } class C : I1, I2 { #region Imicitly Defined I1 Members public string fu() { return "Hello C"; } #endregion Imicitly Defined I1 Members #region Explicitly Defined I1 Members string I1.fu() { return "Hello from I1"; } #endregion Explicitly Defined I1 Members #region Explicitly Defined I2 Members string I2.fu() { return "Hello from I2"; } #endregion Explicitly Defined I2 Members } class D : C { #region Imicitly Defined I1 Members public string fu() { return "Hello from D"; } #endregion Imicitly Defined I1 Members } }.csharpcode, .csharpcode pre{ font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/}.csharpcode pre { margin: 0em; }.csharpcode .rem { color: #008000; }.csharpcode .kwrd { color: #0000ff; }.csharpcode .str { color: #006080; }.csharpcode .op { color: #0000c0; }.csharpcode .preproc { color: #cc6633; }.csharpcode .asp { background-color: #ffff00; }.csharpcode .html { color: #800000; }.csharpcode .attr { color: #ff0000; }.csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em;}.csharpcode .lnum { color: #606060; }Output:-Hello C Hello from I1 Hello from I2 Hello C Hello from I1 Hello from I2 Hello from D Hello from I1 Hello from I2 span.fullpost {display:none;}

Read the article

Creating shapes on the fly

- by Bertrand Le Roy

Most Orchard shapes get created from part drivers, but they are a lot more versatile than that. They can actually be created from pretty much anywhere, including from templates. One example can be found in the Layout.cshtml file of the ThemeMachine theme: WorkContext.Layout.Footer .Add(New.BadgeOfHonor(), "5"); .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } What this is really doing is create a new shape called BadgeOfHonor and injecting it into the Footer global zone (that has not yet been defined, which in itself is quite awesome) with an ordering rank of "5". We can actually come up with something simpler, if we want to render the shape inline instead of sending it into a zone: @Display(New.BadgeOfHonor()) Now let's try something a little more elaborate and create a new shape for displaying a date and time: @Display(New.DateTime(date: DateTime.Now, format: "d/M/yyyy")) For the moment, this throws a "Shape type DateTime not found" exception because the system has no clue how to render a shape called "DateTime" yet. The BadgeOfHonor shape above was rendering something because there is a template for it in the theme: Themes/ThethemeMachine/Views/BadgeOfHonor.cshtml. We need to provide a template for our new shape to get rendered. Let's add a DateTime.cshtml file into our theme's Views folder in order to make the exception go away: Hi, I'm a date time shape. Now we're just missing one thing. Instead of displaying some static text, which is not very interesting, we can display the actual time that got passed into the shape's dynamic constructor. Those parameters will get added to the template's Model, so they are easy to retrieve: @(((DateTime)Model.date).ToString(Model.format)) Now that may remind you a little of WebForm's user controls. That's a fair comparison, except that these shapes are much more flexible (you can add properties on the fly as necessary), and that the actual rendering is decoupled from the "control". For example, any theme can override the template for a shape, you can use alternates, wrappers, etc. Most importantly, there is no lifecycle and protocol abstraction like there was in WebForms. I think this is a real improvement over previous attempts at similar things.

Read the article

Serving up a RSS feed in MVC using WCF Syndication

- by brian_ritchie

With .NET 3.5, Microsoft added the SyndicationFeed class to WCF for generating ATOM 1.0 & RSS 2.0 feeds. In .NET 3.5, it lives in System.ServiceModel.Web but was moved into System.ServiceModel in .NET 4.0. Here's some sample code on constructing a feed: .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: Consolas, "Courier New", Courier, Monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } 1: SyndicationFeed feed = new SyndicationFeed(title, description, new Uri(link)); 2: feed.Categories.Add(new SyndicationCategory(category)); 3: feed.Copyright = new TextSyndicationContent(copyright); 4: feed.Language = "en-us"; 5: feed.Copyright = new TextSyndicationContent(DateTime.Now.Year + " " + ownerName); 6: feed.ImageUrl = new Uri(imageUrl); 7: feed.LastUpdatedTime = DateTime.Now; 8: feed.Authors.Add(new SyndicationPerson() { Name = ownerName, Email = ownerEmail }); 9: 10: var feedItems = new List<SyndicationItem>(); 11: foreach (var item in Items) 12: { 13: var sItem = new SyndicationItem(item.title, null, new Uri(link)); 14: sItem.Summary = new TextSyndicationContent(item.summary); 15: sItem.Id = item.id; 16: if (item.publishedDate != null) 17: sItem.PublishDate = (DateTimeOffset)item.publishedDate; 18: sItem.Links.Add(new SyndicationLink() { Title = item.title, Uri = new Uri(link), Length = item.size, MediaType = item.mediaType }); 19: feedItems.Add(sItem); 20: } 21: feed.Items = feedItems; Then, we create a custom ContentResult to serialize the feed & stream it to the client: 1: public class SyndicationFeedResult : ContentResult 2: { 3: public SyndicationFeedResult(SyndicationFeed feed) 4: : base() 5: { 6: using (var memstream = new MemoryStream()) 7: using (var writer = new XmlTextWriter(memstream, System.Text.UTF8Encoding.UTF8)) 8: { 9: feed.SaveAsRss20(writer); 10: writer.Flush(); 11: memstream.Position = 0; 12: Content = new StreamReader(memstream).ReadToEnd(); 13: ContentType = "application/rss+xml" ; 14: } 15: } 16: } Finally, we wire it up through the controller: 1: public class RssController : Controller 2: { 3: public SyndicationFeedResult Feed() 4: { 5: var feed = new SyndicationFeed(); 6: // populate feed... 7: return new SyndicationFeedResult(feed); 8: } 9: } In the next post, I'll discuss how to add iTunes markup to the feed to publish it on iTunes as a Podcast.

Read the article

Drawing transparent glyphs on the HTML canvas

- by Bertrand Le Roy

The HTML canvas has a set of methods, createImageData and putImageData, that look like they will enable you to draw transparent shapes pixel by pixel. The data structures that you manipulate with these methods are pseudo-arrays of pixels, with four bytes per pixel. One byte for red, one for green, one for blue and one for alpha. This alpha byte makes one believe that you are going to be able to manage transparency, but that’s a lie. Here is a little script that attempts to overlay a simple generated pattern on top of a uniform background: var wrong = document.getElementById("wrong").getContext("2d"); wrong.fillStyle = "#ffd42a"; wrong.fillRect(0, 0, 64, 64); var overlay = wrong.createImageData(32, 32), data = overlay.data; fill(data); wrong.putImageData(overlay, 16, 16); .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } where the fill method is setting the pixels in the lower-left half of the overlay to opaque red, and the rest to transparent black. And here’s how it renders: As you can see, the transparency byte was completely ignored. Or was it? in fact, what happens is more subtle. What happens is that the pixels from the image data, including their alpha byte, replaced the existing pixels of the canvas. So the alpha byte is not lost, it’s just that it wasn’t used by putImageData to combine the new pixels with the existing ones. This is in fact a clue to how to write a putImageData that works: we can first dump that image data into an intermediary canvas, and then compose that temporary canvas onto our main canvas. The method that we can use for this composition is drawImage, which works not only with image objects, but also with canvas objects. var right = document.getElementById("right").getContext("2d"); right.fillStyle = "#ffd42a"; right.fillRect(0, 0, 64, 64); var overlay = wrong.createImageData(32, 32), data = overlay.data; fill(data); var overlayCanvas = document.createElement("canvas"); overlayCanvas.width = overlayCanvas.height = 32; overlayCanvas.getContext("2d").putImageData(overlay, 0, 0); right.drawImage(overlayCanvas, 16, 16); And there is is, a version of putImageData that works like it should always have:

Read the article

Using Teleriks new LINQ implementation to connect to MySQL

Last week Telerik released a new LINQ implementation that is simple to use and produces domain models very fast. Built on top of the enterprise grade OpenAccess ORM, you can connect to any database that OpenAccess can connect to such as: SQL Server, MySQL, Oracle, SQL Azure, VistaDB, etc. Today I will show you how to build a domain model using MySQL as your back end. To get started, you have to download MySQL 5.x and the MySQL Workbench and also, as my colleague Alexander Filipov at Telerik reminded me, make sure you install the MySQL .NET Connector, which is available here. I like to use Northwind, ok it gives me the warm and fuzzies, so I ran a script to produce Northwind on my MySQL server. There are many ways you can get Northwind on your MySQL database, here is a helpful blog to get your started. I also manipulated the first record to indicate that I am in MySQL and gave a look via the MySQL Workbench. Ok, time to build our model! Start up the Domain Model wizard by right clicking on the project in Visual Studio (I have a Web project) and select Add|New Item and choose Telerik OpenAccess Domain Model from the new item list. When the wizard comes up, choose MySQL as your back end and enter in the name of your saved MySQL connection. If you dont have a saved MySQL connection set up in Visual Studio, click on New Connection and enter in the proper connection information. *Note, this is where you need to have the MySQL .NET connector installed. After you set your connection to the MySQL database server, you have to choose which tables to include in your model. Just for fun, I will choose all of them. Give your model a name, like NorthwindEntities and click finish. That is it. Now lets consume the model with ASP .net. I created a simple page that also has a GridView on it. On my page load I wrote this code, by now it should look very familiar, a simple LINQ query filtering customers by country (Germany) and binding the results to the grid. 1: protected void Page_Load(object sender, EventArgs e) 2: { 3: if (!IsPostBack) 4: { 5: //a reference to the data context 6: NorthwindEntities dat = new NorthwindEntities(); 7: //LINQ Statement 8: var result = from c in dat.Customers 9: where c.Country == "Germany" 10: select c; 11: //Databinding to the Gridview 12: GridView1.DataSource = result; 13: GridView1.DataBind(); 14: } 15: } .csharpcode, .csharpcode pre{ font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/}.csharpcode pre { margin: 0em; }.csharpcode .rem { color: #008000; }.csharpcode .kwrd { color: #0000ff; }.csharpcode .str { color: #006080; }.csharpcode .op { color: #0000c0; }.csharpcode .preproc { color: #cc6633; }.csharpcode .asp { background-color: #ffff00; }.csharpcode .html { color: #800000; }.csharpcode .attr { color: #ff0000; }.csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em;}.csharpcode .lnum { color: #606060; } F5 produces the following. Tomorrow Ill show how to take the same model and create an Astoria/OData data feed. Technorati Tags: MySQL Did you know that DotNetSlackers also publishes .net articles written by top known .net Authors? We already have over 80 articles in several categories including Silverlight. Take a look: here.

Read the article

Overriding the Pager rendering in Orchard

- by Bertrand Le Roy

The Pager shape that is used in Orchard to render pagination is one of those shapes that are built in code rather than in a Razor template. This can make it a little more confusing to override, but nothing is impossible. If we look at the Pager method in CoreShapes, here is what we see: [Shape] public IHtmlString Pager(dynamic Shape, dynamic Display) { Shape.Metadata.Alternates.Clear(); Shape.Metadata.Type = "Pager_Links"; return Display(Shape); } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } The Shape attribute signals a shape method. All it does is remove all alternates that may exist and replace the type of the shape with “Pager_Links”. In turn, this shape method is rather large and complicated, but it renders as a set of smaller shapes: a List with a “pager” class, and under that Pager_First, Pager_Previous, Pager_Gap, for each page a Pager_Link or a Pager_Current, then Pager_Gap, Pager_Next and Pager_Last. Each of these shapes can be displayed or not depending on the properties of the pager. Each can also be overridden with a Razor template. This can be done by dropping a file into the Views folder of your theme. For example, if you want the current page to appear between square braces, you could drop this Pager-CurrentPage.cshtml into your views folder: <span>[@Model.Value]</span> This overrides the original shape method, which was this: [Shape] public IHtmlString Pager_CurrentPage(HtmlHelper Html, dynamic Display, object Value) { var tagBuilder = new TagBuilder("span"); tagBuilder.InnerHtml = Html.Encode(Value is string ? (string)Value : Display(Value)); return MvcHtmlString.Create(tagBuilder.ToString()); } And here is what it would look like: Now what if we want to completely hide the pager if there is only one page? Well, the easiest way to do that is to override the Pager shape by dropping the following into the Views folder of your theme: @{ if (Model.TotalItemCount > Model.PageSize) { Model.Metadata.Alternates.Clear(); Model.Metadata.Type = "Pager_Links"; @Display(Model) } } And that’s it. The code in this template just adds a check for the number of items to display (in a template, Model is the shape) and only displays the Pager_Links shape if it knows that there’s going to be more than one page.

Read the article

Writing an unthemed view while still using Orchard shapes and helpers

- by Bertrand Le Roy

This quick tip will show how you can write a custom view for a custom controller action in Orchard that does not use the current theme, but that still retains the ability to use shapes, as well as zones, Script and Style helpers. The controller action, first, needs to opt out of theming: [Themed(false)] public ActionResult Index() {} .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } Then, we still want to use a shape as the view model, because Clay is so awesome: private readonly dynamic _shapeFactory; public MyController(IShapeFactory shapeFactory) { _shapeFactory = shapeFactory; } [Themed(false)] public ActionResult Index() { return View(_shapeFactory.MyShapeName( Foo: 42, Bar: "baz" )); } As you can see, we injected a shape factory, and that enables us to build our shape from our action and inject that into the view as the model. Finally, in the view (that would in Views/MyController/Index.cshtml here), just use helpers as usual. The only gotcha is that you need to use “Layout” in order to declare zones, and that two of those zones, Head and Tail, are mandatory for the top and bottom scripts and stylesheets to be injected properly. Names are important here. @{ Style.Include("somestylesheet.css"); Script.Require("jQuery"); Script.Include("somescript.js"); using(Script.Foot()) { <script type="text/javascript"> $(function () { // Do stuff }) </script> } } <!DOCTYPE html> <html> <head> <title>My unthemed page</title> @Display(Layout.Head) </head> <body> <h1>My unthemed page</h1> <div>@Model.Foo is the answer.</div> </body> @Display(Layout.Tail) </html> Note that if you define your own zones using @Display(Layout.SomeZone) in your view, you can perfectly well send additional shapes to them from your controller action, if you injected an instance of IWorkContextAccessor: _workContextAccessor.GetContext().Layout .SomeZone.Add(_shapeFactory.SomeOtherShape()); Of course, you’ll need to write a SomeOtherShape.cshtml template for that shape but I think this is pretty neat.

Read the article

Azure Web Sites FTP credentials

- by Bertrand Le Roy

A quick tip for all you new enthusiastic users of the amazing new Azure. I struggled for a few minutes finding this, so I thought I’d share. The Azure dashboard doesn’t seem to give easy access to your FTP credentials, and they are not the login and password you use everywhere else. What Azure does give you though is a Publish Profile that you can download: This is a plain XML file that should look something like this: <publishData> <publishProfile profileName="nameofyoursite - Web Deploy" publishMethod="MSDeploy" publishUrl="waws-prod-blu-001.publish.azurewebsites.windows.net:443" msdeploySite="nameofyoursite" userName="$NameOfYourSite" userPWD="sOmeCrYPTicL00kIngStr1nG" destinationAppUrl="http://nameofyoursite.azurewebsites.net" SQLServerDBConnectionString="" mySQLDBConnectionString="" hostingProviderForumLink="" controlPanelLink="http://windows.azure.com"> <databases/> </publishProfile> <publishProfile profileName="nameofyoursite - FTP" publishMethod="FTP" publishUrl="ftp://waws-prod-blu-001.ftp.azurewebsites.windows.net/site/wwwroot" ftpPassiveMode="True" userName="nameofyoursite\$nameofyoursite" userPWD="sOmeCrYPTicL00kIngStr1nG" destinationAppUrl="http://nameofyoursite.azurewebsites.net" SQLServerDBConnectionString="" mySQLDBConnectionString="" hostingProviderForumLink="" controlPanelLink="http://windows.azure.com"> <databases/> </publishProfile> </publishData> .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } I’ve highlighted the FTP server name, user name and password. This is what you need to use in Filezilla or whatever you use to access your site remotely. Notice how the password looks encrypted. Well, it’s not really encrypted in fact. This is your password in clear text. It’s just crypto-random gibberish, which is the best kind of password. UPDATE: About 2 minutes after I posted that, David Ebbo mentioned to me on Twitter that if you've configured publishing credentials (for Git typically) those will work too. Don't forget to include the full user name though, which should be of the form nameofthesite\username. The password is the one you defined. That’s it. Enjoy.

Read the article

C# 5 Async, Part 3: Preparing Existing code For Await

- by Reed

While the Visual Studio Async CTP provides a fantastic model for asynchronous programming, it requires code to be implemented in terms of Task and Task<T>. The CTP adds support for Task-based asynchrony to the .NET Framework methods, and promises to have these implemented directly in the framework in the future. However, existing code outside the framework will need to be converted to using the Task class prior to being usable via the CTP. Wrapping existing asynchronous code into a Task or Task<T> is, thankfully, fairly straightforward. There are two main approaches to this. Code written using the Asynchronous Programming Model (APM) is very easy to convert to using Task<T>. The TaskFactory class provides the tools to directly convert APM code into a method returning a Task<T>. This is done via the FromAsync method. This method takes the BeginOperation and EndOperation methods, as well as any parameters and state objects as arguments, and returns a Task<T> directly. For example, we could easily convert the WebRequest BeginGetResponse and EndGetResponse methods into a method which returns a Task<WebResponse> via: Task<WebResponse> task = Task.Factory .FromAsync<WebResponse>( request.BeginGetResponse, request.EndGetResponse, null); .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } Event-based Asynchronous Pattern (EAP) code can also be wrapped into a Task<T>, though this requires a bit more effort than the one line of code above. This is handled via the TaskCompletionSource<T> class. MSDN provides a detailed example of using this to wrap an EAP operation into a method returning Task<T>. It demonstrates handling cancellation and exception handling as well as the basic operation of the asynchronous method itself. The basic form of this operation is typically: Task<YourResult> GetResultAsync() { var tcs = new TaskCompletionSource<YourResult>(); // Handle the event, and setup the task results... this.GetResultCompleted += (o,e) => { if (e.Error != null) tcs.TrySetException(e.Error); else if (e.Cancelled) tcs.TrySetCanceled(); else tcs.TrySetResult(e.Result); }; // Call the asynchronous method this.GetResult(); // Return the task from the TaskCompletionSource return tcs.Task; } We can easily use these methods to wrap our own code into a method that returns a Task<T>. Existing libraries which cannot be edited can be extended via Extension methods. The CTP uses this technique to add appropriate methods throughout the framework. The suggested naming for these methods is to define these methods as “Task<YourResult> YourClass.YourOperationAsync(…)”. However, this naming often conflicts with the default naming of the EAP. If this is the case, the CTP has standardized on using “Task<YourResult> YourClass.YourOperationTaskAsync(…)”. Once we’ve wrapped all of our existing code into operations that return Task<T>, we can begin investigating how the Async CTP can be used with our own code.

Read the article

ConcurrentDictionary<TKey,TValue> used with Lazy<T>

- by Reed

In a recent thread on the MSDN forum for the TPL, Stephen Toub suggested mixing ConcurrentDictionary<T,U> with Lazy<T>. This provides a fantastic model for creating a thread safe dictionary of values where the construction of the value type is expensive. This is an incredibly useful pattern for many operations, such as value caches. The ConcurrentDictionary<TKey, TValue> class was added in .NET 4, and provides a thread-safe, lock free collection of key value pairs. While this is a fantastic replacement for Dictionary<TKey, TValue>, it has a potential flaw when used with values where construction of the value class is expensive. The typical way this is used is to call a method such as GetOrAdd to fetch or add a value to the dictionary. It handles all of the thread safety for you, but as a result, if two threads call this simultaneously, two instances of TValue can easily be constructed. If TValue is very expensive to construct, or worse, has side effects if constructed too often, this is less than desirable. While you can easily work around this with locking, Stephen Toub provided a very clever alternative – using Lazy<TValue> as the value in the dictionary instead. This looks like the following. Instead of calling: MyValue value = dictionary.GetOrAdd( key, () => new MyValue(key)); .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } We would instead use a ConcurrentDictionary<TKey, Lazy<TValue>>, and write: MyValue value = dictionary.GetOrAdd( key, () => new Lazy<MyValue>( () => new MyValue(key))) .Value; This simple change dramatically changes how the operation works. Now, if two threads call this simultaneously, instead of constructing two MyValue instances, we construct two Lazy<MyValue> instances. However, the Lazy<T> class is very cheap to construct. Unlike “MyValue”, we can safely afford to construct this twice and “throw away” one of the instances. We then call Lazy<T>.Value at the end to fetch our “MyValue” instance. At this point, GetOrAdd will always return the same instance of Lazy<MyValue>. Since Lazy<T> doesn’t construct the MyValue instance until requested, the actual MyClass instance returned is only constructed once.

Read the article

Hazelcast Distributed Executor Service KeyOwner

- by János Veres

I have problem understanding the concept of Hazelcast Distributed Execution. It is said to be able to perform the execution on the owner instance of a specific key. From Documentation: <T> Future<T> submitToKeyOwner(Callable<T> task, Object key) Submits task to owner of the specified key and returns a Future representing that task. Parameters: task - task key - key Returns: a Future representing pending completion of the task I believe that I'm not alone to have a cluster built with multiple maps which might actually use the same key for different purposes, holding different objects (e.g. something along the following setup): IMap<String, ObjectTypeA> firstMap = HazelcastInstance.getMap("firstMap"); IMap<String, ObjectTypeA_AppendixClass> secondMap = HazelcastInstance.getMap("secondMap"); To me it seems quite confusing what documentation says about the owner of a key. My real frustration is that I don't know WHICH - in which map - key does it refer to? The documentation also gives a "demo" of this approach: import com.hazelcast.core.Member; import com.hazelcast.core.Hazelcast; import com.hazelcast.core.IExecutorService; import java.util.concurrent.Callable; import java.util.concurrent.Future; import java.util.Set; import com.hazelcast.config.Config; public void echoOnTheMemberOwningTheKey(String input, Object key) throws Exception { Callable<String> task = new Echo(input); HazelcastInstance hz = Hazelcast.newHazelcastInstance(); IExecutorService executorService = hz.getExecutorService("default"); Future<String> future = executorService.submitToKeyOwner(task, key); String echoResult = future.get(); } Here's a link to the documentation site: Hazelcast MultiHTML Documentation 3.0 - Distributed Execution Did any of you guys figure out in the past what key does it want?

Read the article

Unordered list appears higher than div

- by LordArmadillo

I am trying to create a simple page, which I normally can without any trouble. However, the unordered list appears higher up the page than the div. I do know a solution for this, however I cannot remember it. The page goes like this: <style> ul.menu { list-style-type: none; margin:150; margin-bottom:30px; padding:0; display: block; width: 708px; margin-left: auto; margin-right: auto; } li.menu { float:left; display: block; width: 118px; } a.menu:link,a.menu:visited { color:black; display:block; border:1px solid black; background-color:#FFFF00; width:118px; text-align:center; text-decoration:none; font-family:"Courier New", Courier, monospace } a.menu:hover { background-color:#FFBB11; } </style> <div id="head"> My images here & such</div> <ul class="menu"> <li class="menu"> <a class="menu">Main</a> </li> </ul> Normally I welcome suggestions to change my code, but I have a deadline today! So, quick & simple help would be appreciated! Thanks! LordArmadillo

Search Results

Search found 1007 results on 41 pages for 'courier imap'.

Page 22/41 | < Previous Page | 18 19 20 21 22 23 24 25 26 27 28 29 | Next Page >

- by 84104

- by Dženis Macanovic

- by nareshov

- by Debilski

- by jtnire

- by Whitey

- by john.rose

- by Andrei Bârsan

- by Jaanus

- by Reed

- by Reed

- by Haitham Khedre

- by Bertrand Le Roy

- by brian_ritchie

- by Bertrand Le Roy

- by Bertrand Le Roy

- by Bertrand Le Roy

- by Bertrand Le Roy

- by Reed

- by Reed

- by János Veres

- by LordArmadillo

< Previous Page | 18 19 20 21 22 23 24 25 26 27 28 29 | Next Page >