reverse thinking - Page 449

Silverlight 2.0 - Can't get the text wrapping behaviour that I want

- by Anthony

I am having trouble getting Silverlight 2.0 to lay out text exactly how I want. I want text with line breaks and embedded links, with wrapping, like HTML text in a web page. Here's the closest that I have come: <UserControl x:Class="FlowPanelTest.Page" xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation" xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" xmlns:Controls="clr-namespace:Microsoft.Windows.Controls;assembly=Microsoft.Windows.Controls" Width="250" Height="300"> <Border BorderBrush="Black" BorderThickness="2" > <Controls:WrapPanel> <TextBlock x:Name="tb1" TextWrapping="Wrap">Short text. </TextBlock> <TextBlock x:Name="tb2" TextWrapping="Wrap">A bit of text. </TextBlock> <TextBlock x:Name="tb3" TextWrapping="Wrap">About half of a line of text.</TextBlock> <TextBlock x:Name="tb4" TextWrapping="Wrap">More than half a line of longer text.</TextBlock> <TextBlock x:Name="tb5" TextWrapping="Wrap">More than one line of text, so it will wrap onto the following line.</TextBlock> </Controls:WrapPanel> </Border> </UserControl> But the issue is that although the text blocks tb1 and tb2 will go onto the same line because there is room enough for them completely, tb3 onwards will not start on the same line as the previous block, even though it will wrap onto following lines. I want each text block to start where the previous one ends, on the same line. I want to put click event handlers on some of the text. I also want paragraph breaks. Essentially I'm trying to work around the lack of FlowDocument and Hyperlink controls in Silverlight 2.0's subset of XAML. To answer the questions posed in the answers: Why not use runs for the non-clickable text? If I just use individual TextBlocks only on the clickable text, then those bits of text will still suffer from the wrapping problem illustrated above. And the TextBlock just before the link, and the TextBlock just after. Essentially all of it. It doesn't look like I have many opportunities for putting multiple runs in the same TextBlock. Dividing the links from the other text with RegExs and loops is not the issue at all, the issue is display layout. Why not put each word in an individual TextBlock in a WrapPanel Aside from being an ugly hack, this does not play at all well with linebreaks - the layout is incorrect. It would also make the underline style of linked text into a broken line. Here's an example with each word in its own TextBlock. Try running it, note that the linebreak isn't shown in the right place at all. <UserControl x:Class="SilverlightApplication2.Page" xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation" xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" xmlns:Controls="clr-namespace:Microsoft.Windows.Controls;assembly=Microsoft.Windows.Controls" Width="300" Height="300"> <Controls:WrapPanel> <TextBlock TextWrapping="Wrap">Short1 </TextBlock> <TextBlock TextWrapping="Wrap">Longer1 </TextBlock> <TextBlock TextWrapping="Wrap">Longerest1 </TextBlock> <TextBlock TextWrapping="Wrap"> <Run>Break</Run> <LineBreak></LineBreak> </TextBlock> <TextBlock TextWrapping="Wrap">Short2</TextBlock> <TextBlock TextWrapping="Wrap">Longer2</TextBlock> <TextBlock TextWrapping="Wrap">Longerest2</TextBlock> <TextBlock TextWrapping="Wrap">Short3</TextBlock> <TextBlock TextWrapping="Wrap">Longer3</TextBlock> <TextBlock TextWrapping="Wrap">Longerest3</TextBlock> </Controls:WrapPanel> </UserControl> What about The LinkLabelControl as here and here. It has the same problems as the approach above, since it's much the same. Try running the sample, and make the link text longer and longer until it wraps. Note that the link starts on a new line, which it shouldn't. Make the link text even longer, so that the link text is longer than a line. Note that it doesn't wrap at all, it cuts off. This control doesn't handle line breaks and paragraph breaks either. Why not put the text all in runs, detect clicks on the containing TextBlock and work out which run was clicked Runs do not have mouse events, but the containing TextBlock does. I can't find a way to check if the run is under the mouse (IsMouseOver is not present in SilverLight) or to find the bounding geometry of the run (no clip property). There is VisualTreeHelper.FindElementsInHostCoordinates() The code below uses VisualTreeHelper.FindElementsInHostCoordinates to get the controls under the click. The output lists the TextBlock but not the Run, since a Run is not a UiElement. private void theText_MouseLeftButtonDown(object sender, System.Windows.Input.MouseButtonEventArgs e) { // get the elements under the click UIElement uiElementSender = sender as UIElement; Point clickPos = e.GetPosition(uiElementSender); var UiElementsUnderClick = VisualTreeHelper.FindElementsInHostCoordinates(clickPos, uiElementSender); // show the controls string outputText = ""; foreach (var uiElement in UiElementsUnderClick) { outputText += uiElement.GetType().ToString() + "\n"; } this.outText.Text = outputText; } Use an empty text block with a margin to space following content onto a following line I'm still thinking about this one. How do you calculate the right width for a line-breaking block to force following content onto the following line? Too short and the following content will still be on the same line, at the right. Too long and the "linebreak" will be on the following line, with content after it. You would have to resize the breaks when the control is resized. Some of the code for this is: TextBlock lineBreak = new TextBlock(); lineBreak.TextWrapping = TextWrapping.Wrap; lineBreak.Text = " "; // need adaptive width lineBreak.Margin = new Thickness(0, 0, 200, 0);

Read the article

value types in the vm

- by john.rose

value types in the vm p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 14.0px Times} p.p2 {margin: 0.0px 0.0px 14.0px 0.0px; font: 14.0px Times} p.p3 {margin: 0.0px 0.0px 12.0px 0.0px; font: 14.0px Times} p.p4 {margin: 0.0px 0.0px 15.0px 0.0px; font: 14.0px Times} p.p5 {margin: 0.0px 0.0px 0.0px 0.0px; font: 14.0px Courier} p.p6 {margin: 0.0px 0.0px 0.0px 0.0px; font: 14.0px Courier; min-height: 17.0px} p.p7 {margin: 0.0px 0.0px 0.0px 0.0px; font: 14.0px Times; min-height: 18.0px} p.p8 {margin: 0.0px 0.0px 0.0px 36.0px; text-indent: -36.0px; font: 14.0px Times; min-height: 18.0px} p.p9 {margin: 0.0px 0.0px 12.0px 0.0px; font: 14.0px Times; min-height: 18.0px} p.p10 {margin: 0.0px 0.0px 12.0px 0.0px; font: 14.0px Times; color: #000000} li.li1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 14.0px Times} li.li7 {margin: 0.0px 0.0px 0.0px 0.0px; font: 14.0px Times; min-height: 18.0px} span.s1 {font: 14.0px Courier} span.s2 {color: #000000} span.s3 {font: 14.0px Courier; color: #000000} ol.ol1 {list-style-type: decimal} Or, enduring values for a changing world. Introduction A value type is a data type which, generally speaking, is designed for being passed by value in and out of methods, and stored by value in data structures. The only value types which the Java language directly supports are the eight primitive types. Java indirectly and approximately supports value types, if they are implemented in terms of classes. For example, both Integer and String may be viewed as value types, especially if their usage is restricted to avoid operations appropriate to Object. In this note, we propose a definition of value types in terms of a design pattern for Java classes, accompanied by a set of usage restrictions. We also sketch the relation of such value types to tuple types (which are a JVM-level notion), and point out JVM optimizations that can apply to value types. This note is a thought experiment to extend the JVM’s performance model in support of value types. The demonstration has two phases. Initially the extension can simply use design patterns, within the current bytecode architecture, and in today’s Java language. But if the performance model is to be realized in practice, it will probably require new JVM bytecode features, changes to the Java language, or both. We will look at a few possibilities for these new features. An Axiom of Value In the context of the JVM, a value type is a data type equipped with construction, assignment, and equality operations, and a set of typed components, such that, whenever two variables of the value type produce equal corresponding values for their components, the values of the two variables cannot be distinguished by any JVM operation. Here are some corollaries: A value type is immutable, since otherwise a copy could be constructed and the original could be modified in one of its components, allowing the copies to be distinguished. Changing the component of a value type requires construction of a new value. The equals and hashCode operations are strictly component-wise. If a value type is represented by a JVM reference, that reference cannot be successfully synchronized on, and cannot be usefully compared for reference equality. A value type can be viewed in terms of what it doesn’t do. We can say that a value type omits all value-unsafe operations, which could violate the constraints on value types. These operations, which are ordinarily allowed for Java object types, are pointer equality comparison (the acmp instruction), synchronization (the monitor instructions), all the wait and notify methods of class Object, and non-trivial finalize methods. The clone method is also value-unsafe, although for value types it could be treated as the identity function. Finally, and most importantly, any side effect on an object (however visible) also counts as an value-unsafe operation. A value type may have methods, but such methods must not change the components of the value. It is reasonable and useful to define methods like toString, equals, and hashCode on value types, and also methods which are specifically valuable to users of the value type. Representations of Value Value types have two natural representations in the JVM, unboxed and boxed. An unboxed value consists of the components, as simple variables. For example, the complex number x=(1+2i), in rectangular coordinate form, may be represented in unboxed form by the following pair of variables: /*Complex x = Complex.valueOf(1.0, 2.0):*/ double x_re = 1.0, x_im = 2.0; These variables might be locals, parameters, or fields. Their association as components of a single value is not defined to the JVM. Here is a sample computation which computes the norm of the difference between two complex numbers: double distance(/*Complex x:*/ double x_re, double x_im, /*Complex y:*/ double y_re, double y_im) { /*Complex z = x.minus(y):*/ double z_re = x_re - y_re, z_im = x_im - y_im; /*return z.abs():*/ return Math.sqrt(z_re*z_re + z_im*z_im); } A boxed representation groups component values under a single object reference. The reference is to a ‘wrapper class’ that carries the component values in its fields. (A primitive type can naturally be equated with a trivial value type with just one component of that type. In that view, the wrapper class Integer can serve as a boxed representation of value type int.) The unboxed representation of complex numbers is practical for many uses, but it fails to cover several major use cases: return values, array elements, and generic APIs. The two components of a complex number cannot be directly returned from a Java function, since Java does not support multiple return values. The same story applies to array elements: Java has no ’array of structs’ feature. (Double-length arrays are a possible workaround for complex numbers, but not for value types with heterogeneous components.) By generic APIs I mean both those which use generic types, like Arrays.asList and those which have special case support for primitive types, like String.valueOf and PrintStream.println. Those APIs do not support unboxed values, and offer some problems to boxed values. Any ’real’ JVM type should have a story for returns, arrays, and API interoperability. The basic problem here is that value types fall between primitive types and object types. Value types are clearly more complex than primitive types, and object types are slightly too complicated. Objects are a little bit dangerous to use as value carriers, since object references can be compared for pointer equality, and can be synchronized on. Also, as many Java programmers have observed, there is often a performance cost to using wrapper objects, even on modern JVMs. Even so, wrapper classes are a good starting point for talking about value types. If there were a set of structural rules and restrictions which would prevent value-unsafe operations on value types, wrapper classes would provide a good notation for defining value types. This note attempts to define such rules and restrictions. Let’s Start Coding Now it is time to look at some real code. Here is a definition, written in Java, of a complex number value type. @ValueSafe public final class Complex implements java.io.Serializable { // immutable component structure: public final double re, im; private Complex(double re, double im) { this.re = re; this.im = im; } // interoperability methods: public String toString() { return "Complex("+re+","+im+")"; } public List<Double> asList() { return Arrays.asList(re, im); } public boolean equals(Complex c) { return re == c.re && im == c.im; } public boolean equals(@ValueSafe Object x) { return x instanceof Complex && equals((Complex) x); } public int hashCode() { return 31*Double.valueOf(re).hashCode() + Double.valueOf(im).hashCode(); } // factory methods: public static Complex valueOf(double re, double im) { return new Complex(re, im); } public Complex changeRe(double re2) { return valueOf(re2, im); } public Complex changeIm(double im2) { return valueOf(re, im2); } public static Complex cast(@ValueSafe Object x) { return x == null ? ZERO : (Complex) x; } // utility methods and constants: public Complex plus(Complex c) { return new Complex(re+c.re, im+c.im); } public Complex minus(Complex c) { return new Complex(re-c.re, im-c.im); } public double abs() { return Math.sqrt(re*re + im*im); } public static final Complex PI = valueOf(Math.PI, 0.0); public static final Complex ZERO = valueOf(0.0, 0.0); } This is not a minimal definition, because it includes some utility methods and other optional parts. The essential elements are as follows: The class is marked as a value type with an annotation. The class is final, because it does not make sense to create subclasses of value types. The fields of the class are all non-private and final. (I.e., the type is immutable and structurally transparent.) From the supertype Object, all public non-final methods are overridden. The constructor is private. Beyond these bare essentials, we can observe the following features in this example, which are likely to be typical of all value types: One or more factory methods are responsible for value creation, including a component-wise valueOf method. There are utility methods for complex arithmetic and instance creation, such as plus and changeIm. There are static utility constants, such as PI. The type is serializable, using the default mechanisms. There are methods for converting to and from dynamically typed references, such as asList and cast. The Rules In order to use value types properly, the programmer must avoid value-unsafe operations. A helpful Java compiler should issue errors (or at least warnings) for code which provably applies value-unsafe operations, and should issue warnings for code which might be correct but does not provably avoid value-unsafe operations. No such compilers exist today, but to simplify our account here, we will pretend that they do exist. A value-safe type is any class, interface, or type parameter marked with the @ValueSafe annotation, or any subtype of a value-safe type. If a value-safe class is marked final, it is in fact a value type. All other value-safe classes must be abstract. The non-static fields of a value class must be non-public and final, and all its constructors must be private. Under the above rules, a standard interface could be helpful to define value types like Complex. Here is an example: @ValueSafe public interface ValueType extends java.io.Serializable { // All methods listed here must get redefined. // Definitions must be value-safe, which means // they may depend on component values only. List<? extends Object> asList(); int hashCode(); boolean equals(@ValueSafe Object c); String toString(); } //@ValueSafe inherited from supertype: public final class Complex implements ValueType { … The main advantage of such a conventional interface is that (unlike an annotation) it is reified in the runtime type system. It could appear as an element type or parameter bound, for facilities which are designed to work on value types only. More broadly, it might assist the JVM to perform dynamic enforcement of the rules for value types. Besides types, the annotation @ValueSafe can mark fields, parameters, local variables, and methods. (This is redundant when the type is also value-safe, but may be useful when the type is Object or another supertype of a value type.) Working forward from these annotations, an expression E is defined as value-safe if it satisfies one or more of the following: The type of E is a value-safe type. E names a field, parameter, or local variable whose declaration is marked @ValueSafe. E is a call to a method whose declaration is marked @ValueSafe. E is an assignment to a value-safe variable, field reference, or array reference. E is a cast to a value-safe type from a value-safe expression. E is a conditional expression E0 ? E1 : E2, and both E1 and E2 are value-safe. Assignments to value-safe expressions and initializations of value-safe names must take their values from value-safe expressions. A value-safe expression may not be the subject of a value-unsafe operation. In particular, it cannot be synchronized on, nor can it be compared with the “==” operator, not even with a null or with another value-safe type. In a program where all of these rules are followed, no value-type value will be subject to a value-unsafe operation. Thus, the prime axiom of value types will be satisfied, that no two value type will be distinguishable as long as their component values are equal. More Code To illustrate these rules, here are some usage examples for Complex: Complex pi = Complex.valueOf(Math.PI, 0); Complex zero = pi.changeRe(0); //zero = pi; zero.re = 0; ValueType vtype = pi; @SuppressWarnings("value-unsafe") Object obj = pi; @ValueSafe Object obj2 = pi; obj2 = new Object(); // ok List<Complex> clist = new ArrayList<Complex>(); clist.add(pi); // (ok assuming List.add param is @ValueSafe) List<ValueType> vlist = new ArrayList<ValueType>(); vlist.add(pi); // (ok) List<Object> olist = new ArrayList<Object>(); olist.add(pi); // warning: "value-unsafe" boolean z = pi.equals(zero); boolean z1 = (pi == zero); // error: reference comparison on value type boolean z2 = (pi == null); // error: reference comparison on value type boolean z3 = (pi == obj2); // error: reference comparison on value type synchronized (pi) { } // error: synch of value, unpredictable result synchronized (obj2) { } // unpredictable result Complex qq = pi; qq = null; // possible NPE; warning: “null-unsafe" qq = (Complex) obj; // warning: “null-unsafe" qq = Complex.cast(obj); // OK @SuppressWarnings("null-unsafe") Complex empty = null; // possible NPE qq = empty; // possible NPE (null pollution) The Payoffs It follows from this that either the JVM or the java compiler can replace boxed value-type values with unboxed ones, without affecting normal computations. Fields and variables of value types can be split into their unboxed components. Non-static methods on value types can be transformed into static methods which take the components as value parameters. Some common questions arise around this point in any discussion of value types. Why burden the programmer with all these extra rules? Why not detect programs automagically and perform unboxing transparently? The answer is that it is easy to break the rules accidently unless they are agreed to by the programmer and enforced. Automatic unboxing optimizations are tantalizing but (so far) unreachable ideal. In the current state of the art, it is possible exhibit benchmarks in which automatic unboxing provides the desired effects, but it is not possible to provide a JVM with a performance model that assures the programmer when unboxing will occur. This is why I’m writing this note, to enlist help from, and provide assurances to, the programmer. Basically, I’m shooting for a good set of user-supplied “pragmas” to frame the desired optimization. Again, the important thing is that the unboxing must be done reliably, or else programmers will have no reason to work with the extra complexity of the value-safety rules. There must be a reasonably stable performance model, wherein using a value type has approximately the same performance characteristics as writing the unboxed components as separate Java variables. There are some rough corners to the present scheme. Since Java fields and array elements are initialized to null, value-type computations which incorporate uninitialized variables can produce null pointer exceptions. One workaround for this is to require such variables to be null-tested, and the result replaced with a suitable all-zero value of the value type. That is what the “cast” method does above. Generically typed APIs like List<T> will continue to manipulate boxed values always, at least until we figure out how to do reification of generic type instances. Use of such APIs will elicit warnings until their type parameters (and/or relevant members) are annotated or typed as value-safe. Retrofitting List<T> is likely to expose flaws in the present scheme, which we will need to engineer around. Here are a couple of first approaches: public interface java.util.List<@ValueSafe T> extends Collection<T> { … public interface java.util.List<T extends Object|ValueType> extends Collection<T> { … (The second approach would require disjunctive types, in which value-safety is “contagious” from the constituent types.) With more transformations, the return value types of methods can also be unboxed. This may require significant bytecode-level transformations, and would work best in the presence of a bytecode representation for multiple value groups, which I have proposed elsewhere under the title “Tuples in the VM”. But for starters, the JVM can apply this transformation under the covers, to internally compiled methods. This would give a way to express multiple return values and structured return values, which is a significant pain-point for Java programmers, especially those who work with low-level structure types favored by modern vector and graphics processors. The lack of multiple return values has a strong distorting effect on many Java APIs. Even if the JVM fails to unbox a value, there is still potential benefit to the value type. Clustered computing systems something have copy operations (serialization or something similar) which apply implicitly to command operands. When copying JVM objects, it is extremely helpful to know when an object’s identity is important or not. If an object reference is a copied operand, the system may have to create a proxy handle which points back to the original object, so that side effects are visible. Proxies must be managed carefully, and this can be expensive. On the other hand, value types are exactly those types which a JVM can “copy and forget” with no downside. Array types are crucial to bulk data interfaces. (As data sizes and rates increase, bulk data becomes more important than scalar data, so arrays are definitely accompanying us into the future of computing.) Value types are very helpful for adding structure to bulk data, so a successful value type mechanism will make it easier for us to express richer forms of bulk data. Unboxing arrays (i.e., arrays containing unboxed values) will provide better cache and memory density, and more direct data movement within clustered or heterogeneous computing systems. They require the deepest transformations, relative to today’s JVM. There is an impedance mismatch between value-type arrays and Java’s covariant array typing, so compromises will need to be struck with existing Java semantics. It is probably worth the effort, since arrays of unboxed value types are inherently more memory-efficient than standard Java arrays, which rely on dependent pointer chains. It may be sufficient to extend the “value-safe” concept to array declarations, and allow low-level transformations to change value-safe array declarations from the standard boxed form into an unboxed tuple-based form. Such value-safe arrays would not be convertible to Object[] arrays. Certain connection points, such as Arrays.copyOf and System.arraycopy might need additional input/output combinations, to allow smooth conversion between arrays with boxed and unboxed elements. Alternatively, the correct solution may have to wait until we have enough reification of generic types, and enough operator overloading, to enable an overhaul of Java arrays. Implicit Method Definitions The example of class Complex above may be unattractively complex. I believe most or all of the elements of the example class are required by the logic of value types. If this is true, a programmer who writes a value type will have to write lots of error-prone boilerplate code. On the other hand, I think nearly all of the code (except for the domain-specific parts like plus and minus) can be implicitly generated. Java has a rule for implicitly defining a class’s constructor, if no it defines no constructors explicitly. Likewise, there are rules for providing default access modifiers for interface members. Because of the highly regular structure of value types, it might be reasonable to perform similar implicit transformations on value types. Here’s an example of a “highly implicit” definition of a complex number type: public class Complex implements ValueType { // implicitly final public double re, im; // implicitly public final //implicit methods are defined elementwise from te fields: // toString, asList, equals(2), hashCode, valueOf, cast //optionally, explicit methods (plus, abs, etc.) would go here } In other words, with the right defaults, a simple value type definition can be a one-liner. The observant reader will have noticed the similarities (and suitable differences) between the explicit methods above and the corresponding methods for List<T>. Another way to abbreviate such a class would be to make an annotation the primary trigger of the functionality, and to add the interface(s) implicitly: public @ValueType class Complex { … // implicitly final, implements ValueType (But to me it seems better to communicate the “magic” via an interface, even if it is rooted in an annotation.) Implicitly Defined Value Types So far we have been working with nominal value types, which is to say that the sequence of typed components is associated with a name and additional methods that convey the intention of the programmer. A simple ordered pair of floating point numbers can be variously interpreted as (to name a few possibilities) a rectangular or polar complex number or Cartesian point. The name and the methods convey the intended meaning. But what if we need a truly simple ordered pair of floating point numbers, without any further conceptual baggage? Perhaps we are writing a method (like “divideAndRemainder”) which naturally returns a pair of numbers instead of a single number. Wrapping the pair of numbers in a nominal type (like “QuotientAndRemainder”) makes as little sense as wrapping a single return value in a nominal type (like “Quotient”). What we need here are structural value types commonly known as tuples. For the present discussion, let us assign a conventional, JVM-friendly name to tuples, roughly as follows: public class java.lang.tuple.$DD extends java.lang.tuple.Tuple { double $1, $2; } Here the component names are fixed and all the required methods are defined implicitly. The supertype is an abstract class which has suitable shared declarations. The name itself mentions a JVM-style method parameter descriptor, which may be “cracked” to determine the number and types of the component fields. The odd thing about such a tuple type (and structural types in general) is it must be instantiated lazily, in response to linkage requests from one or more classes that need it. The JVM and/or its class loaders must be prepared to spin a tuple type on demand, given a simple name reference, $xyz, where the xyz is cracked into a series of component types. (Specifics of naming and name mangling need some tasteful engineering.) Tuples also seem to demand, even more than nominal types, some support from the language. (This is probably because notations for non-nominal types work best as combinations of punctuation and type names, rather than named constructors like Function3 or Tuple2.) At a minimum, languages with tuples usually (I think) have some sort of simple bracket notation for creating tuples, and a corresponding pattern-matching syntax (or “destructuring bind”) for taking tuples apart, at least when they are parameter lists. Designing such a syntax is no simple thing, because it ought to play well with nominal value types, and also with pre-existing Java features, such as method parameter lists, implicit conversions, generic types, and reflection. That is a task for another day. Other Use Cases Besides complex numbers and simple tuples there are many use cases for value types. Many tuple-like types have natural value-type representations. These include rational numbers, point locations and pixel colors, and various kinds of dates and addresses. Other types have a variable-length ‘tail’ of internal values. The most common example of this is String, which is (mathematically) a sequence of UTF-16 character values. Similarly, bit vectors, multiple-precision numbers, and polynomials are composed of sequences of values. Such types include, in their representation, a reference to a variable-sized data structure (often an array) which (somehow) represents the sequence of values. The value type may also include ’header’ information. Variable-sized values often have a length distribution which favors short lengths. In that case, the design of the value type can make the first few values in the sequence be direct ’header’ fields of the value type. In the common case where the header is enough to represent the whole value, the tail can be a shared null value, or even just a null reference. Note that the tail need not be an immutable object, as long as the header type encapsulates it well enough. This is the case with String, where the tail is a mutable (but never mutated) character array. Field types and their order must be a globally visible part of the API. The structure of the value type must be transparent enough to have a globally consistent unboxed representation, so that all callers and callees agree about the type and order of components that appear as parameters, return types, and array elements. This is a trade-off between efficiency and encapsulation, which is forced on us when we remove an indirection enjoyed by boxed representations. A JVM-only transformation would not care about such visibility, but a bytecode transformation would need to take care that (say) the components of complex numbers would not get swapped after a redefinition of Complex and a partial recompile. Perhaps constant pool references to value types need to declare the field order as assumed by each API user. This brings up the delicate status of private fields in a value type. It must always be possible to load, store, and copy value types as coordinated groups, and the JVM performs those movements by moving individual scalar values between locals and stack. If a component field is not public, what is to prevent hostile code from plucking it out of the tuple using a rogue aload or astore instruction? Nothing but the verifier, so we may need to give it more smarts, so that it treats value types as inseparable groups of stack slots or locals (something like long or double). My initial thought was to make the fields always public, which would make the security problem moot. But public is not always the right answer; consider the case of String, where the underlying mutable character array must be encapsulated to prevent security holes. I believe we can win back both sides of the tradeoff, by training the verifier never to split up the components in an unboxed value. Just as the verifier encapsulates the two halves of a 64-bit primitive, it can encapsulate the the header and body of an unboxed String, so that no code other than that of class String itself can take apart the values. Similar to String, we could build an efficient multi-precision decimal type along these lines: public final class DecimalValue extends ValueType { protected final long header; protected private final BigInteger digits; public DecimalValue valueOf(int value, int scale) { assert(scale >= 0); return new DecimalValue(((long)value << 32) + scale, null); } public DecimalValue valueOf(long value, int scale) { if (value == (int) value) return valueOf((int)value, scale); return new DecimalValue(-scale, new BigInteger(value)); } } Values of this type would be passed between methods as two machine words. Small values (those with a significand which fits into 32 bits) would be represented without any heap data at all, unless the DecimalValue itself were boxed. (Note the tension between encapsulation and unboxing in this case. It would be better if the header and digits fields were private, but depending on where the unboxing information must “leak”, it is probably safer to make a public revelation of the internal structure.) Note that, although an array of Complex can be faked with a double-length array of double, there is no easy way to fake an array of unboxed DecimalValues. (Either an array of boxed values or a transposed pair of homogeneous arrays would be reasonable fallbacks, in a current JVM.) Getting the full benefit of unboxing and arrays will require some new JVM magic. Although the JVM emphasizes portability, system dependent code will benefit from using machine-level types larger than 64 bits. For example, the back end of a linear algebra package might benefit from value types like Float4 which map to stock vector types. This is probably only worthwhile if the unboxing arrays can be packed with such values. More Daydreams A more finely-divided design for dynamic enforcement of value safety could feature separate marker interfaces for each invariant. An empty marker interface Unsynchronizable could cause suitable exceptions for monitor instructions on objects in marked classes. More radically, a Interchangeable marker interface could cause JVM primitives that are sensitive to object identity to raise exceptions; the strangest result would be that the acmp instruction would have to be specified as raising an exception. @ValueSafe public interface ValueType extends java.io.Serializable, Unsynchronizable, Interchangeable { … public class Complex implements ValueType { // inherits Serializable, Unsynchronizable, Interchangeable, @ValueSafe … It seems possible that Integer and the other wrapper types could be retro-fitted as value-safe types. This is a major change, since wrapper objects would be unsynchronizable and their references interchangeable. It is likely that code which violates value-safety for wrapper types exists but is uncommon. It is less plausible to retro-fit String, since the prominent operation String.intern is often used with value-unsafe code. We should also reconsider the distinction between boxed and unboxed values in code. The design presented above obscures that distinction. As another thought experiment, we could imagine making a first class distinction in the type system between boxed and unboxed representations. Since only primitive types are named with a lower-case initial letter, we could define that the capitalized version of a value type name always refers to the boxed representation, while the initial lower-case variant always refers to boxed. For example: complex pi = complex.valueOf(Math.PI, 0); Complex boxPi = pi; // convert to boxed myList.add(boxPi); complex z = myList.get(0); // unbox Such a convention could perhaps absorb the current difference between int and Integer, double and Double. It might also allow the programmer to express a helpful distinction among array types. As said above, array types are crucial to bulk data interfaces, but are limited in the JVM. Extending arrays beyond the present limitations is worth thinking about; for example, the Maxine JVM implementation has a hybrid object/array type. Something like this which can also accommodate value type components seems worthwhile. On the other hand, does it make sense for value types to contain short arrays? And why should random-access arrays be the end of our design process, when bulk data is often sequentially accessed, and it might make sense to have heterogeneous streams of data as the natural “jumbo” data structure. These considerations must wait for another day and another note. More Work It seems to me that a good sequence for introducing such value types would be as follows: Add the value-safety restrictions to an experimental version of javac. Code some sample applications with value types, including Complex and DecimalValue. Create an experimental JVM which internally unboxes value types but does not require new bytecodes to do so. Ensure the feasibility of the performance model for the sample applications. Add tuple-like bytecodes (with or without generic type reification) to a major revision of the JVM, and teach the Java compiler to switch in the new bytecodes without code changes. A staggered roll-out like this would decouple language changes from bytecode changes, which is always a convenient thing. A similar investigation should be applied (concurrently) to array types. In this case, it seems to me that the starting point is in the JVM: Add an experimental unboxing array data structure to a production JVM, perhaps along the lines of Maxine hybrids. No bytecode or language support is required at first; everything can be done with encapsulated unsafe operations and/or method handles. Create an experimental JVM which internally unboxes value types but does not require new bytecodes to do so. Ensure the feasibility of the performance model for the sample applications. Add tuple-like bytecodes (with or without generic type reification) to a major revision of the JVM, and teach the Java compiler to switch in the new bytecodes without code changes. That’s enough musing me for now. Back to work!

Read the article

Solving embarassingly parallel problems using Python multiprocessing

- by gotgenes

How does one use multiprocessing to tackle embarrassingly parallel problems? Embarassingly parallel problems typically consist of three basic parts: Read input data (from a file, database, tcp connection, etc.). Run calculations on the input data, where each calculation is independent of any other calculation. Write results of calculations (to a file, database, tcp connection, etc.). We can parallelize the program in two dimensions: Part 2 can run on multiple cores, since each calculation is independent; order of processing doesn't matter. Each part can run independently. Part 1 can place data on an input queue, part 2 can pull data off the input queue and put results onto an output queue, and part 3 can pull results off the output queue and write them out. This seems a most basic pattern in concurrent programming, but I am still lost in trying to solve it, so let's write a canonical example to illustrate how this is done using multiprocessing. Here is the example problem: Given a CSV file with rows of integers as input, compute their sums. Separate the problem into three parts, which can all run in parallel: Process the input file into raw data (lists/iterables of integers) Calculate the sums of the data, in parallel Output the sums Below is traditional, single-process bound Python program which solves these three tasks: #!/usr/bin/env python # -*- coding: UTF-8 -*- # basicsums.py """A program that reads integer values from a CSV file and writes out their sums to another CSV file. """ import csv import optparse import sys def make_cli_parser(): """Make the command line interface parser.""" usage = "\n\n".join(["python %prog INPUT_CSV OUTPUT_CSV", __doc__, """ ARGUMENTS: INPUT_CSV: an input CSV file with rows of numbers OUTPUT_CSV: an output file that will contain the sums\ """]) cli_parser = optparse.OptionParser(usage) return cli_parser def parse_input_csv(csvfile): """Parses the input CSV and yields tuples with the index of the row as the first element, and the integers of the row as the second element. The index is zero-index based. :Parameters: - `csvfile`: a `csv.reader` instance """ for i, row in enumerate(csvfile): row = [int(entry) for entry in row] yield i, row def sum_rows(rows): """Yields a tuple with the index of each input list of integers as the first element, and the sum of the list of integers as the second element. The index is zero-index based. :Parameters: - `rows`: an iterable of tuples, with the index of the original row as the first element, and a list of integers as the second element """ for i, row in rows: yield i, sum(row) def write_results(csvfile, results): """Writes a series of results to an outfile, where the first column is the index of the original row of data, and the second column is the result of the calculation. The index is zero-index based. :Parameters: - `csvfile`: a `csv.writer` instance to which to write results - `results`: an iterable of tuples, with the index (zero-based) of the original row as the first element, and the calculated result from that row as the second element """ for result_row in results: csvfile.writerow(result_row) def main(argv): cli_parser = make_cli_parser() opts, args = cli_parser.parse_args(argv) if len(args) != 2: cli_parser.error("Please provide an input file and output file.") infile = open(args[0]) in_csvfile = csv.reader(infile) outfile = open(args[1], 'w') out_csvfile = csv.writer(outfile) # gets an iterable of rows that's not yet evaluated input_rows = parse_input_csv(in_csvfile) # sends the rows iterable to sum_rows() for results iterable, but # still not evaluated result_rows = sum_rows(input_rows) # finally evaluation takes place as a chain in write_results() write_results(out_csvfile, result_rows) infile.close() outfile.close() if __name__ == '__main__': main(sys.argv[1:]) Let's take this program and rewrite it to use multiprocessing to parallelize the three parts outlined above. Below is a skeleton of this new, parallelized program, that needs to be fleshed out to address the parts in the comments: #!/usr/bin/env python # -*- coding: UTF-8 -*- # multiproc_sums.py """A program that reads integer values from a CSV file and writes out their sums to another CSV file, using multiple processes if desired. """ import csv import multiprocessing import optparse import sys NUM_PROCS = multiprocessing.cpu_count() def make_cli_parser(): """Make the command line interface parser.""" usage = "\n\n".join(["python %prog INPUT_CSV OUTPUT_CSV", __doc__, """ ARGUMENTS: INPUT_CSV: an input CSV file with rows of numbers OUTPUT_CSV: an output file that will contain the sums\ """]) cli_parser = optparse.OptionParser(usage) cli_parser.add_option('-n', '--numprocs', type='int', default=NUM_PROCS, help="Number of processes to launch [DEFAULT: %default]") return cli_parser def main(argv): cli_parser = make_cli_parser() opts, args = cli_parser.parse_args(argv) if len(args) != 2: cli_parser.error("Please provide an input file and output file.") infile = open(args[0]) in_csvfile = csv.reader(infile) outfile = open(args[1], 'w') out_csvfile = csv.writer(outfile) # Parse the input file and add the parsed data to a queue for # processing, possibly chunking to decrease communication between # processes. # Process the parsed data as soon as any (chunks) appear on the # queue, using as many processes as allotted by the user # (opts.numprocs); place results on a queue for output. # # Terminate processes when the parser stops putting data in the # input queue. # Write the results to disk as soon as they appear on the output # queue. # Ensure all child processes have terminated. # Clean up files. infile.close() outfile.close() if __name__ == '__main__': main(sys.argv[1:]) These pieces of code, as well as another piece of code that can generate example CSV files for testing purposes, can be found on github. I would appreciate any insight here as to how you concurrency gurus would approach this problem. Here are some questions I had when thinking about this problem. Bonus points for addressing any/all: Should I have child processes for reading in the data and placing it into the queue, or can the main process do this without blocking until all input is read? Likewise, should I have a child process for writing the results out from the processed queue, or can the main process do this without having to wait for all the results? Should I use a processes pool for the sum operations? If yes, what method do I call on the pool to get it to start processing the results coming into the input queue, without blocking the input and output processes, too? apply_async()? map_async()? imap()? imap_unordered()? Suppose we didn't need to siphon off the input and output queues as data entered them, but could wait until all input was parsed and all results were calculated (e.g., because we know all the input and output will fit in system memory). Should we change the algorithm in any way (e.g., not run any processes concurrently with I/O)?

Read the article

Python regex to parse text file, get the items in list and count the list

- by Nemo

I have a text file which contains some data. I m particularly interested in finding the count of the number of items in v_dims v_dims pattern in my text file looks like this : v_dims={ "Sales", "Product Family", "Sales Organization", "Region", "Sales Area", "Sales office", "Sales Division", "Sales Person", "Sales Channel", "Sales Order Type", "Sales Number", "Sales Person", "Sales Quantity", "Sales Amount" } So I m thinking of getting all the elements in v_dims and dumping them out in a Python list. Then compute the len(mylist) to get the count of the items. The challenge is in getting all the elements of v_dims from my text file and putting them in an empty list. I m particularly interested in items in v_dims in my text file. The text file has data in the form of v_dims pattern i showed in my original post. Some data has nested patterns of v_dims. Thanks. Here's what I have tried and failed. Any help is appreciated. TIA. import re fname = "C:\Users\XXXX\Test.mrk" with open(fname, "r") as fo: content_as_string = fo.read() match = re.findall(r'v_dims={\"(.+?)\"}',content_as_string) Though I have a big text file, Here's a snippet of what's the structure of my text file version "1"; // Computer generated object language file object 'MRKR' "Main" { Data_Type=2, HeaderBlock={ Version_String="6.3 (25)" }, Printer_Info={ Orientation=0, Page_Width=8.50000000, Page_Height=11.00000000, Page_Header="", Page_Footer="", Margin_type=0, Top_Margin=0.50000000, Left_Margin=0.50000000, Bottom_Margin=0.50000000, Right_Margin=0.50000000 }, Marker_Options={ Close_All="TRUE", Hide_Console="FALSE", Console_Left="FALSE", Console_Width=217, Main_Style="Maximized", MDI_Rect={ 0, 0, 892, 1063 } }, Dives={ { Dive="A", Windows={ { View_Index=0, Window_Info={ Window_Rect={ 0, -288, 400, 1008 }, Window_Style="Maximized Front", Window_Name="Theater [Previous Qtr Diveplan-Dive A]" }, Dependent_bool="FALSE", Colset={ Dive_Type="Normal", Dimension_Name="Theater", Action_List={ Actions={ { Action_Type="Select", select_type=5 }, { Action_Type="Select", select_type=0, Key_Names={ "Theater" }, Key_Indexes={ { "AMERICAS" } } }, { Action_Type="Focus", Focus_Rows="True" }, { Action_Type="Dimensions", v_dims={ "Theater", "Product Family", "Division", "Region", "Install at Country Name", "Connect Home Type", "Connect In Type", "SymmConnect Enabled", "Connect Home Refusal Reason", "Sales Order Channel Type", "Maintained By Group", "PS Flag", "Avalanche Flag", "Product Item Family" }, Xtab_Bool="False", Xtab_Flip="False" }, { Action_Type="Select", select_type=5 }, { Action_Type="Select", select_type=0, Key_Names={ "Theater", "Product Family", "Division", "Region", "Install at Country Name", "Connect Home Type", "Connect In Type", "SymmConnect Enabled", "Connect Home Refusal Reason", "Sales Order Channel Type", "Maintained By Group", "PS Flag", "Avalanche Flag" }, Key_Indexes={ { "AMERICAS", "ATMOS", "Latin America CS Division", "37000 CS Region", "Mexico", "", "", "", "", "DIRECT", "EMC", "N", "0" } } } } }, Num_Palette_cols=0, Num_Palette_rows=0 }, Format={ Window_Type="Tabular", Tabular={ Num_row_labels=8 } } } } } }, Widget_Set={ Widget_Layout="Vertical", Go_Button=1, Picklist_Width=0, Sort_Subset_Dimensions="TRUE", Order={ } }, Views={ { Data_Type=1, dbname="Previous Qtr Diveplan", diveline_dbname="Current Qtr Diveplan", logical_name="Current Qtr Diveplan", cols={ { name="Total TSS installs", column_type="Calc[Total TSS installs]", output_type="Number", format_string="." }, { name="TSS Valid Connectivity Records", column_type="Calc[TSS Valid Connectivity Records]", output_type="Number", format_string="." }, { name="% TSS Connectivity Record", column_type="Calc[% TSS Connectivity Record]", output_type="Number" }, { name="TSS Not Applicable", column_type="Calc[TSS Not Applicable]", output_type="Number", format_string="." }, { name="TSS Customer Refusals", column_type="Calc[TSS Customer Refusals]", output_type="Number", format_string="." }, { name="% TSS Refusals", column_type="Calc[% TSS Refusals]", output_type="Number" }, { name="TSS Eligible for Physical Connectivity", column_type="Calc[TSS Eligible for Physical Connectivity]", output_type="Number", format_string="." }, { name="TSS Boxes with Physical Connectivty", column_type="Calc[TSS Boxes with Physical Connectivty]", output_type="Number", format_string="." }, { name="% TSS Physical Connectivity", column_type="Calc[% TSS Physical Connectivity]", output_type="Number" } }, dim_cols={ { name="Model", column_type="Dimension[Model]", output_type="None" }, { name="Model", column_type="Dimension[Model]", output_type="None" }, { name="Connect In Type", column_type="Dimension[Connect In Type]", output_type="None" }, { name="Connect Home Type", column_type="Dimension[Connect Home Type]", output_type="None" }, { name="SymmConnect Enabled", column_type="Dimension[SymmConnect Enabled]", output_type="None" }, { name="Theater", column_type="Dimension[Theater]", output_type="None" }, { name="Division", column_type="Dimension[Division]", output_type="None" }, { name="Region", column_type="Dimension[Region]", output_type="None" }, { name="Sales Order Number", column_type="Dimension[Sales Order Number]", output_type="None" }, { name="Product Item Family", column_type="Dimension[Product Item Family]", output_type="None" }, { name="Item Serial Number", column_type="Dimension[Item Serial Number]", output_type="None" }, { name="Sales Order Deal Number", column_type="Dimension[Sales Order Deal Number]", output_type="None" }, { name="Item Install Date", column_type="Dimension[Item Install Date]", output_type="None" }, { name="SYR Last Dial Home Date", column_type="Dimension[SYR Last Dial Home Date]", output_type="None" }, { name="Maintained By Group", column_type="Dimension[Maintained By Group]", output_type="None" }, { name="PS Flag", column_type="Dimension[PS Flag]", output_type="None" }, { name="Connect Home Refusal Reason", column_type="Dimension[Connect Home Refusal Reason]", output_type="None", col_width=177 }, { name="Cust Name", column_type="Dimension[Cust Name]", output_type="None" }, { name="Sales Order Channel Type", column_type="Dimension[Sales Order Channel Type]", output_type="None" }, { name="Sales Order Type", column_type="Dimension[Sales Order Type]", output_type="None" }, { name="Part Model Key", column_type="Dimension[Part Model Key]", output_type="None" }, { name="Ship Date", column_type="Dimension[Ship Date]", output_type="None" }, { name="Model Number", column_type="Dimension[Model Number]", output_type="None" }, { name="Item Description", column_type="Dimension[Item Description]", output_type="None" }, { name="Customer Classification", column_type="Dimension[Customer Classification]", output_type="None" }, { name="CS Customer Name", column_type="Dimension[CS Customer Name]", output_type="None" }, { name="Install At Customer Number", column_type="Dimension[Install At Customer Number]", output_type="None" }, { name="Install at Country Name", column_type="Dimension[Install at Country Name]", output_type="None" }, { name="TLA Serial Number", column_type="Dimension[TLA Serial Number]", output_type="None" }, { name="Product Version", column_type="Dimension[Product Version]", output_type="None" }, { name="Avalanche Flag", column_type="Dimension[Avalanche Flag]", output_type="None" }, { name="Product Family", column_type="Dimension[Product Family]", output_type="None" }, { name="Project Number", column_type="Dimension[Project Number]", output_type="None" }, { name="PROJECT_STATUS", column_type="Dimension[PROJECT_STATUS]", output_type="None" } }, Available_Columns={ "Total TSS installs", "TSS Valid Connectivity Records", "% TSS Connectivity Record", "TSS Not Applicable", "TSS Customer Refusals", "% TSS Refusals", "TSS Eligible for Physical Connectivity", "TSS Boxes with Physical Connectivty", "% TSS Physical Connectivity", "Total Installs", "All Boxes with Valid Connectivty Record", "% All Connectivity Record", "Overall Refusals", "Overall Refusals %", "All Eligible for Physical Connectivty", "Boxes with Physical Connectivity", "% All with Physical Conectivity" }, Remaining_columns={ { name="Total Installs", column_type="Calc[Total Installs]", output_type="Number", format_string="." }, { name="All Boxes with Valid Connectivty Record", column_type="Calc[All Boxes with Valid Connectivty Record]", output_type="Number", format_string="." }, { name="% All Connectivity Record", column_type="Calc[% All Connectivity Record]", output_type="Number" }, { name="Overall Refusals", column_type="Calc[Overall Refusals]", output_type="Number", format_string="." }, { name="Overall Refusals %", column_type="Calc[Overall Refusals %]", output_type="Number" }, { name="All Eligible for Physical Connectivty", column_type="Calc[All Eligible for Physical Connectivty]", output_type="Number" }, { name="Boxes with Physical Connectivity", column_type="Calc[Boxes with Physical Connectivity]", output_type="Number" }, { name="% All with Physical Conectivity", column_type="Calc[% All with Physical Conectivity]", output_type="Number" } }, calcs={ { name="Total TSS installs", definition="Total[Total TSS installs]", ts_flag="Not TS Calc" }, { name="TSS Valid Connectivity Records", definition="Total[PS Boxes w/ valid connectivity record (1=yes)]", ts_flag="Not TS Calc" }, { name="% TSS Connectivity Record", definition="Total[PS Boxes w/ valid connectivity record (1=yes)] /Total[Total TSS installs]", ts_flag="Not TS Calc" }, { name="TSS Not Applicable", definition="Total[Bozes w/ valid connectivity record (1=yes)]-Total[Boxes Eligible (1=yes)]-Total[TSS Refusals]", ts_flag="Not TS Calc" }, { name="TSS Customer Refusals", definition="Total[TSS Refusals]", ts_flag="Not TS Calc" }, { name="% TSS Refusals", definition="Total[TSS Refusals]/Total[PS Boxes w/ valid connectivity record (1=yes)]", ts_flag="Not TS Calc" }, { name="TSS Eligible for Physical Connectivity", definition="Total[TSS Eligible]-Total[Exception]", ts_flag="Not TS Calc" }, { name="TSS Boxes with Physical Connectivty", definition="Total[PS Physical Connectivity] - Total[PS Physical Connectivity, SymmConnect Enabled=\"Capable not enabled\"]", ts_flag="Not TS Calc" }, { name="% TSS Physical Connectivity", definition="Total[Boxes w/ phys conn]/Total[Boxes Eligible (1=yes)]", ts_flag="Not TS Calc" }, { name="Total Installs", definition="Total[Total Installs]", ts_flag="Not TS Calc" }, { name="All Boxes with Valid Connectivty Record", definition="Total[Bozes w/ valid connectivity record (1=yes)]", ts_flag="Not TS Calc" }, { name="% All Connectivity Record", definition="Total[Bozes w/ valid connectivity record (1=yes)]/Total[Total Installs]", ts_flag="Not TS Calc" }, { name="Overall Refusals", definition="Total[Overall Refusals]", ts_flag="Not TS Calc" }, { name="Overall Refusals %", definition="Total[Overall Refusals]/Total[Bozes w/ valid connectivity record (1=yes)]", ts_flag="Not TS Calc" }, { name="All Eligible for Physical Connectivty", definition="Total[Boxes Eligible (1=yes)]-Total[Exception]", ts_flag="Not TS Calc" }, { name="Boxes with Physical Connectivity", definition="Total[Boxes w/ phys conn]-Total[Boxes w/ phys conn,SymmConnect Enabled=\"Capable not enabled\"]", ts_flag="Not TS Calc" }, { name="% All with Physical Conectivity", definition="Total[Boxes w/ phys conn]/Total[Boxes Eligible (1=yes)]", ts_flag="Not TS Calc" } }, merge_type="consolidate", merge_dbs={ { dbname="connectivityallproducts.mdl", diveline_dbname="/DI_PSREPORTING/connectivityallproducts.mdl" } }, skip_constant_columns="FALSE", categories={ { name="Geography", dimensions={ "Theater", "Division", "Region", "Install at Country Name" } }, { name="Mappings and Flags", dimensions={ "Connect Home Type", "Connect In Type", "SymmConnect Enabled", "Connect Home Refusal Reason", "Sales Order Channel Type", "Maintained By Group", "Customer Installable", "PS Flag", "Top Level Flag", "Avalanche Flag" } }, { name="Product Information", dimensions={ "Product Family", "Product Item Family", "Product Version", "Item Description" } }, { name="Sales Order Info", dimensions={ "Sales Order Deal Number", "Sales Order Number", "Sales Order Type" } }, { name="Dates", dimensions={ "Item Install Date", "Ship Date", "SYR Last Dial Home Date" } }, { name="Details", dimensions={ "Item Serial Number", "TLA Serial Number", "Part Model Key", "Model Number" } }, { name="Customer Infor", dimensions={ "CS Customer Name", "Install At Customer Number", "Customer Classification", "Cust Name" } }, { name="Other Dimensions", dimensions={ "Model" } } }, Maintain_Category_Order="FALSE", popup_info="false" } } };

Read the article

Conceal packet loss in PCM stream

- by ZeroDefect

I am looking to use 'Packet Loss Concealment' to conceal lost PCM frames in an audio stream. Unfortunately, I cannot find a library that is accessible without all the licensing restrictions and code bloat (...up for some suggestions though). I have located some GPL code written by Steve Underwood for the Asterisk project which implements PLC. There are several limitations; although, as Steve suggests in his code, his algorithm can be applied to different streams with a bit of work. Currently, the code works with 8kHz 16-bit signed mono streams. Variations of the code can be found through a simple search of Google Code Search. My hope is that I can adapt the code to work with other streams. Initially, the goal is to adjust the algorithm for 8+ kHz, 16-bit signed, multichannel audio (all in a C++ environment). Eventually, I'm looking to make the code available under the GPL license in hopes that it could be of benefit to others... Attached is the code below with my efforts. The code includes a main function that will "drop" a number of frames with a given probability. Unfortunately, the code does not quite work as expected. I'm receiving EXC_BAD_ACCESS when running in gdb, but I don't get a trace from gdb when using 'bt' command. Clearly, I'm trampimg on memory some where but not sure exactly where. When I comment out the *amdf_pitch* function, the code runs without crashing... int main (int argc, char *argv[]) { std::ifstream fin("C:\\cc32kHz.pcm"); if(!fin.is_open()) { std::cout << "Failed to open input file" << std::endl; return 1; } std::ofstream fout_repaired("C:\\cc32kHz_repaired.pcm"); if(!fout_repaired.is_open()) { std::cout << "Failed to open output repaired file" << std::endl; return 1; } std::ofstream fout_lossy("C:\\cc32kHz_lossy.pcm"); if(!fout_lossy.is_open()) { std::cout << "Failed to open output repaired file" << std::endl; return 1; } audio::PcmConcealer Concealer; Concealer.Init(1, 16, 32000); //Generate random numbers; srand( time(NULL) ); int value = 0; int probability = 5; while(!fin.eof()) { char arr[2]; fin.read(arr, 2); //Generate's random number; value = rand() % 100 + 1; if(value <= probability) { char blank[2] = {0x00, 0x00}; fout_lossy.write(blank, 2); //Fill in data; Concealer.Fill((int16_t *)blank, 1); fout_repaired.write(blank, 2); } else { //Write data to file; fout_repaired.write(arr, 2); fout_lossy.write(arr, 2); Concealer.Receive((int16_t *)arr, 1); } } fin.close(); fout_repaired.close(); fout_lossy.close(); return 0; } PcmConcealer.hpp /* * Code adapted from Steve Underwood of the Asterisk Project. This code inherits * the same licensing restrictions as the Asterisk Project. */ #ifndef __PCMCONCEALER_HPP__ #define __PCMCONCEALER_HPP__ /** 1. What does it do? The packet loss concealment module provides a suitable synthetic fill-in signal, to minimise the audible effect of lost packets in VoIP applications. It is not tied to any particular codec, and could be used with almost any codec which does not specify its own procedure for packet loss concealment. Where a codec specific concealment procedure exists, the algorithm is usually built around knowledge of the characteristics of the particular codec. It will, therefore, generally give better results for that particular codec than this generic concealer will. 2. How does it work? While good packets are being received, the plc_rx() routine keeps a record of the trailing section of the known speech signal. If a packet is missed, plc_fillin() is called to produce a synthetic replacement for the real speech signal. The average mean difference function (AMDF) is applied to the last known good signal, to determine its effective pitch. Based on this, the last pitch period of signal is saved. Essentially, this cycle of speech will be repeated over and over until the real speech resumes. However, several refinements are needed to obtain smooth pleasant sounding results. - The two ends of the stored cycle of speech will not always fit together smoothly. This can cause roughness, or even clicks, at the joins between cycles. To soften this, the 1/4 pitch period of real speech preceeding the cycle to be repeated is blended with the last 1/4 pitch period of the cycle to be repeated, using an overlap-add (OLA) technique (i.e. in total, the last 5/4 pitch periods of real speech are used). - The start of the synthetic speech will not always fit together smoothly with the tail of real speech passed on before the erasure was identified. Ideally, we would like to modify the last 1/4 pitch period of the real speech, to blend it into the synthetic speech. However, it is too late for that. We could have delayed the real speech a little, but that would require more buffer manipulation, and hurt the efficiency of the no-lost-packets case (which we hope is the dominant case). Instead we use a degenerate form of OLA to modify the start of the synthetic data. The last 1/4 pitch period of real speech is time reversed, and OLA is used to blend it with the first 1/4 pitch period of synthetic speech. The result seems quite acceptable. - As we progress into the erasure, the chances of the synthetic signal being anything like correct steadily fall. Therefore, the volume of the synthesized signal is made to decay linearly, such that after 50ms of missing audio it is reduced to silence. - When real speech resumes, an extra 1/4 pitch period of sythetic speech is blended with the start of the real speech. If the erasure is small, this smoothes the transition. If the erasure is long, and the synthetic signal has faded to zero, the blending softens the start up of the real signal, avoiding a kind of "click" or "pop" effect that might occur with a sudden onset. 3. How do I use it? Before audio is processed, call plc_init() to create an instance of the packet loss concealer. For each received audio packet that is acceptable (i.e. not including those being dropped for being too late) call plc_rx() to record the content of the packet. Note this may modify the packet a little after a period of packet loss, to blend real synthetic data smoothly. When a real packet is not available in time, call plc_fillin() to create a sythetic substitute. That's it! */ /*! Minimum allowed pitch (66 Hz) */ #define PLC_PITCH_MIN(SAMPLE_RATE) ((double)(SAMPLE_RATE) / 66.6) /*! Maximum allowed pitch (200 Hz) */ #define PLC_PITCH_MAX(SAMPLE_RATE) ((SAMPLE_RATE) / 200) /*! Maximum pitch OLA window */ //#define PLC_PITCH_OVERLAP_MAX(SAMPLE_RATE) ((PLC_PITCH_MIN(SAMPLE_RATE)) >> 2) /*! The length over which the AMDF function looks for similarity (20 ms) */ #define CORRELATION_SPAN(SAMPLE_RATE) ((20 * (SAMPLE_RATE)) / 1000) /*! History buffer length. The buffer must also be at leat 1.25 times PLC_PITCH_MIN, but that is much smaller than the buffer needs to be for the pitch assessment. */ //#define PLC_HISTORY_LEN(SAMPLE_RATE) ((CORRELATION_SPAN(SAMPLE_RATE)) + (PLC_PITCH_MIN(SAMPLE_RATE))) namespace audio { typedef struct { /*! Consecutive erased samples */ int missing_samples; /*! Current offset into pitch period */ int pitch_offset; /*! Pitch estimate */ int pitch; /*! Buffer for a cycle of speech */ float *pitchbuf;//[PLC_PITCH_MIN]; /*! History buffer */ short *history;//[PLC_HISTORY_LEN]; /*! Current pointer into the history buffer */ int buf_ptr; } plc_state_t; class PcmConcealer { public: PcmConcealer(); ~PcmConcealer(); void Init(int channels, int bit_depth, int sample_rate); //Process a block of received audio samples. int Receive(short amp[], int frames); //Fill-in a block of missing audio samples. int Fill(short amp[], int frames); void Destroy(); private: int amdf_pitch(int min_pitch, int max_pitch, short amp[], int channel_index, int frames); void save_history(plc_state_t *s, short *buf, int channel_index, int frames); void normalise_history(plc_state_t *s); /** Holds the states of each of the channels **/ std::vector< plc_state_t * > ChannelStates; int plc_pitch_min; int plc_pitch_max; int plc_pitch_overlap_max; int correlation_span; int plc_history_len; int channel_count; int sample_rate; bool Initialized; }; } #endif PcmConcealer.cpp /* * Code adapted from Steve Underwood of the Asterisk Project. This code inherits * the same licensing restrictions as the Asterisk Project. */ #include "audio/PcmConcealer.hpp" /* We do a straight line fade to zero volume in 50ms when we are filling in for missing data. */ #define ATTENUATION_INCREMENT 0.0025 /* Attenuation per sample */ #if !defined(INT16_MAX) #define INT16_MAX (32767) #define INT16_MIN (-32767-1) #endif #ifdef WIN32 inline double rint(double x) { return floor(x + 0.5); } #endif inline short fsaturate(double damp) { if (damp > 32767.0) return INT16_MAX; if (damp < -32768.0) return INT16_MIN; return (short)rint(damp); } namespace audio { PcmConcealer::PcmConcealer() : Initialized(false) { } PcmConcealer::~PcmConcealer() { Destroy(); } void PcmConcealer::Init(int channels, int bit_depth, int sample_rate) { if(Initialized) return; if(channels <= 0 || bit_depth != 16) return; Initialized = true; channel_count = channels; this->sample_rate = sample_rate; ////////////// double min = PLC_PITCH_MIN(sample_rate); int imin = (int)min; double max = PLC_PITCH_MAX(sample_rate); int imax = (int)max; plc_pitch_min = imin; plc_pitch_max = imax; plc_pitch_overlap_max = (plc_pitch_min >> 2); correlation_span = CORRELATION_SPAN(sample_rate); plc_history_len = correlation_span + plc_pitch_min; ////////////// for(int i = 0; i < channel_count; i ++) { plc_state_t *t = new plc_state_t; memset(t, 0, sizeof(plc_state_t)); t->pitchbuf = new float[plc_pitch_min]; t->history = new short[plc_history_len]; ChannelStates.push_back(t); } } void PcmConcealer::Destroy() { if(!Initialized) return; while(ChannelStates.size()) { plc_state_t *s = ChannelStates.at(0); if(s) { if(s->history) delete s->history; if(s->pitchbuf) delete s->pitchbuf; memset(s, 0, sizeof(plc_state_t)); delete s; } ChannelStates.erase(ChannelStates.begin()); } ChannelStates.clear(); Initialized = false; } //Process a block of received audio samples. int PcmConcealer::Receive(short amp[], int frames) { if(!Initialized) return 0; int j = 0; for(int k = 0; k < ChannelStates.size(); k++) { int i; int overlap_len; int pitch_overlap; float old_step; float new_step; float old_weight; float new_weight; float gain; plc_state_t *s = ChannelStates.at(k); if (s->missing_samples) { /* Although we have a real signal, we need to smooth it to fit well with the synthetic signal we used for the previous block */ /* The start of the real data is overlapped with the next 1/4 cycle of the synthetic data. */ pitch_overlap = s->pitch >> 2; if (pitch_overlap > frames) pitch_overlap = frames; gain = 1.0 - s->missing_samples * ATTENUATION_INCREMENT; if (gain < 0.0) gain = 0.0; new_step = 1.0/pitch_overlap; old_step = new_step*gain; new_weight = new_step; old_weight = (1.0 - new_step)*gain; for (i = 0; i < pitch_overlap; i++) { int index = (i * channel_count) + j; amp[index] = fsaturate(old_weight * s->pitchbuf[s->pitch_offset] + new_weight * amp[index]); if (++s->pitch_offset >= s->pitch) s->pitch_offset = 0; new_weight += new_step; old_weight -= old_step; if (old_weight < 0.0) old_weight = 0.0; } s->missing_samples = 0; } save_history(s, amp, j, frames); j++; } return frames; } //Fill-in a block of missing audio samples. int PcmConcealer::Fill(short amp[], int frames) { if(!Initialized) return 0; int j =0; for(int k = 0; k < ChannelStates.size(); k++) { short *tmp = new short[plc_pitch_overlap_max]; int i; int pitch_overlap; float old_step; float new_step; float old_weight; float new_weight; float gain; short *orig_amp; int orig_len; orig_amp = amp; orig_len = frames; plc_state_t *s = ChannelStates.at(k); if (s->missing_samples == 0) { // As the gap in real speech starts we need to assess the last known pitch, //and prepare the synthetic data we will use for fill-in normalise_history(s); s->pitch = amdf_pitch(plc_pitch_min, plc_pitch_max, s->history + plc_history_len - correlation_span - plc_pitch_min, j, correlation_span); // We overlap a 1/4 wavelength pitch_overlap = s->pitch >> 2; // Cook up a single cycle of pitch, using a single of the real signal with 1/4 //cycle OLA'ed to make the ends join up nicely // The first 3/4 of the cycle is a simple copy for (i = 0; i < s->pitch - pitch_overlap; i++) s->pitchbuf[i] = s->history[plc_history_len - s->pitch + i]; // The last 1/4 of the cycle is overlapped with the end of the previous cycle new_step = 1.0/pitch_overlap; new_weight = new_step; for ( ; i < s->pitch; i++) { s->pitchbuf[i] = s->history[plc_history_len - s->pitch + i]*(1.0 - new_weight) + s->history[plc_history_len - 2*s->pitch + i]*new_weight; new_weight += new_step; } // We should now be ready to fill in the gap with repeated, decaying cycles // of what is in pitchbuf // We need to OLA the first 1/4 wavelength of the synthetic data, to smooth // it into the previous real data. To avoid the need to introduce a delay // in the stream, reverse the last 1/4 wavelength, and OLA with that. gain = 1.0; new_step = 1.0/pitch_overlap; old_step = new_step; new_weight = new_step; old_weight = 1.0 - new_step; for (i = 0; i < pitch_overlap; i++) { int index = (i * channel_count) + j; amp[index] = fsaturate(old_weight * s->history[plc_history_len - 1 - i] + new_weight * s->pitchbuf[i]); new_weight += new_step; old_weight -= old_step; if (old_weight < 0.0) old_weight = 0.0; } s->pitch_offset = i; } else { gain = 1.0 - s->missing_samples*ATTENUATION_INCREMENT; i = 0; } for ( ; gain > 0.0 && i < frames; i++) { int index = (i * channel_count) + j; amp[index] = s->pitchbuf[s->pitch_offset]*gain; gain -= ATTENUATION_INCREMENT; if (++s->pitch_offset >= s->pitch) s->pitch_offset = 0; } for ( ; i < frames; i++) { int index = (i * channel_count) + j; amp[i] = 0; } s->missing_samples += orig_len; save_history(s, amp, j, frames); delete [] tmp; j++; } return frames; } void PcmConcealer::save_history(plc_state_t *s, short *buf, int channel_index, int frames) { if (frames >= plc_history_len) { /* Just keep the last part of the new data, starting at the beginning of the buffer */ //memcpy(s->history, buf + len - plc_history_len, sizeof(short)*plc_history_len); int frames_to_copy = plc_history_len; for(int i = 0; i < frames_to_copy; i ++) { int index = (channel_count * (i + frames - plc_history_len)) + channel_index; s->history[i] = buf[index]; } s->buf_ptr = 0; return; } if (s->buf_ptr + frames > plc_history_len) { /* Wraps around - must break into two sections */ //memcpy(s->history + s->buf_ptr, buf, sizeof(short)*(plc_history_len - s->buf_ptr)); short *hist_ptr = s->history + s->buf_ptr; int frames_to_copy = plc_history_len - s->buf_ptr; for(int i = 0; i < frames_to_copy; i ++) { int index = (channel_count * i) + channel_index; hist_ptr[i] = buf[index]; } frames -= (plc_history_len - s->buf_ptr); //memcpy(s->history, buf + (plc_history_len - s->buf_ptr), sizeof(short)*len); frames_to_copy = frames; for(int i = 0; i < frames_to_copy; i ++) { int index = (channel_count * (i + (plc_history_len - s->buf_ptr))) + channel_index; s->history[i] = buf[index]; } s->buf_ptr = frames; return; } /* Can use just one section */ //memcpy(s->history + s->buf_ptr, buf, sizeof(short)*len); short *hist_ptr = s->history + s->buf_ptr; int frames_to_copy = frames; for(int i = 0; i < frames_to_copy; i ++) { int index = (channel_count * i) + channel_index; hist_ptr[i] = buf[index]; } s->buf_ptr += frames; } void PcmConcealer::normalise_history(plc_state_t *s) { short *tmp = new short[plc_history_len]; if (s->buf_ptr == 0) return; memcpy(tmp, s->history, sizeof(short)*s->buf_ptr); memcpy(s->history, s->history + s->buf_ptr, sizeof(short)*(plc_history_len - s->buf_ptr)); memcpy(s->history + plc_history_len - s->buf_ptr, tmp, sizeof(short)*s->buf_ptr); s->buf_ptr = 0; delete [] tmp; } int PcmConcealer::amdf_pitch(int min_pitch, int max_pitch, short amp[], int channel_index, int frames) { int i; int j; int acc; int min_acc; int pitch; pitch = min_pitch; min_acc = INT_MAX; for (i = max_pitch; i <= min_pitch; i++) { acc = 0; for (j = 0; j < frames; j++) { int index1 = (channel_count * (i+j)) + channel_index; int index2 = (channel_count * j) + channel_index; //std::cout << "Index 1: " << index1 << ", Index 2: " << index2 << std::endl; acc += abs(amp[index1] - amp[index2]); } if (acc < min_acc) { min_acc = acc; pitch = i; } } std::cout << "Pitch: " << pitch << std::endl; return pitch; } } P.S. - I must confess that digital audio is not my forte...

Read the article

SocketChannel in Java sends data, but it doesn't get to destination application

- by Peterson

Hi Everybody, I'm suffering a lot to create a simple ChatServer in Java, using the NIO libraries. Wonder if someone could help me. I am doing that by using SocketChannel and Selector to handle multiple clients in a single thread. The problem is: I am able to accept new connections and get it's data, but when I try to send data back, the SocketChannel simply doesn't work. In the method write(), it returns a integer that is the same size of the data i'm passing to it, but the client never receives that data. Strangely, when I close the server application, the client receives the data. It's like the socketchannel maintains a buffer, and it only get flushed when I close the application. Here are some more details, to give you more information to help. I'm handling the events in this piece of code: private void run() throws IOException { ServerSocketChannel ssc = ServerSocketChannel.open(); // Set it to non-blocking, so we can use select ssc.configureBlocking( false ); // Get the Socket connected to this channel, and bind it // to the listening port this.serverSocket = ssc.socket(); InetSocketAddress isa = new InetSocketAddress( this.port ); serverSocket.bind( isa ); // Create a new Selector for selecting this.masterSelector = Selector.open(); // Register the ServerSocketChannel, so we can // listen for incoming connections ssc.register( masterSelector, SelectionKey.OP_ACCEPT ); while (true) { // See if we've had any activity -- either // an incoming connection, or incoming data on an // existing connection int num = masterSelector.select(); // If we don't have any activity, loop around and wait // again if (num == 0) { continue; } // Get the keys corresponding to the activity // that has been detected, and process them // one by one Set keys = masterSelector.selectedKeys(); Iterator it = keys.iterator(); while (it.hasNext()) { // Get a key representing one of bits of I/O // activity SelectionKey key = (SelectionKey)it.next(); // What kind of activity is it? if ((key.readyOps() & SelectionKey.OP_ACCEPT) == SelectionKey.OP_ACCEPT) { // Aceita a conexão Socket s = serverSocket.accept(); System.out.println( "LOG: Conexao TCP aceita de " + s.getInetAddress() + ":" + s.getPort() ); // Make sure to make it non-blocking, so we can // use a selector on it. SocketChannel sc = s.getChannel(); sc.configureBlocking( false ); // Registra a conexao no seletor, apenas para leitura sc.register( masterSelector, SelectionKey.OP_READ ); } else if ( key.isReadable() ) { SocketChannel sc = null; // It's incoming data on a connection, so // process it sc = (SocketChannel)key.channel(); // Verifica se a conexão corresponde a um cliente já existente if((clientsMap.getClient(key)) != null){ boolean closedConnection = !processIncomingClientData(key); if(closedConnection){ int id = clientsMap.getClient(key); closeClient(id); } } else { boolean clientAccepted = processIncomingDataFromNewClient(key); if(!clientAccepted){ // Se o cliente não foi aceito, sua conexão é simplesmente fechada sc.socket().close(); sc.close(); key.cancel(); } } } } // We remove the selected keys, because we've dealt // with them. keys.clear(); } } This piece of code is simply handles new clients that wants to connect to the chat. So, a client makes a TCP connection to the server, and once it gets accepted, it sends data to the server following a simply text protocol, informing his id and asking to get registrated to the server. I handle this in the method processIncomingDataFromNewClient(key). I'm also keeping a map of clients and its connections in a data structure similar to a hashtable. I? doing that because I need to recover a client Id from a connection and a connection from a client Id. This is can be shown in: clientsMap.getClient(key). But the problem itself resides in the method processIncomingDataFromNewClient(key). There, I simply read the data that the client sent to me, validate it, and if it's ok, I send a message back to the client to tell that it is connected to the chat server. Here is a similar piece of code: private boolean processIncomingDataFromNewClient(SelectionKey key){ SocketChannel sc = (SocketChannel) key.channel(); String connectionOrigin = sc.socket().getInetAddress() + ":" + sc.socket().getPort(); int id = 0; //id of the client buf.clear(); int bytesRead = 0; try { bytesRead = sc.read(buf); if(bytesRead<=0){ System.out.println("Conexão fechada pelo: " + connectionOrigin); return false; } System.out.println("LOG: " + bytesRead + " bytes lidos de " + connectionOrigin); String msg = new String(buf.array(),0,bytesRead); // Do validations with the client sent me here // gets the client id }catch (Exception e) { e.printStackTrace(); System.out.println("LOG: Oops. Cliente não conhece o protocolo. Fechando a conexão: " + connectionOrigin); System.out.println("LOG: Primeiros 10 caracteres enviados pelo cliente: " + msg); return false; } } } catch (IOException e) { System.out.println("LOG: Erro ao ler dados da conexao: " + connectionOrigin); System.out.println("LOG: "+ e.getLocalizedMessage()); System.out.println("LOG: Fechando a conexão..."); return false; } // If it gets to here, the protocol is ok and we can add the client boolean inserted = clientsMap.addClient(key, id); if(!inserted){ System.out.println("LOG: Não foi possível adicionar o cliente. Ou ele já está conectado ou já têm clientes demais. Id: " + id); System.out.println("LOG: Fechando a conexão: " + connectionOrigin); return false; } System.out.println("LOG: Novo cliente conectado! Enviando mesnsagem de confirmação. Id: " + id + " Conexao: " + connectionOrigin); /* Here is the error */ sendMessage(id, "Servidor pet: connection accepted"); System.out.println("LOG: Novo cliente conectado! Id: " + id + " Conexao: " + connectionOrigin); return true; } And finally, the method sendMessage(SelectionKey key) looks like this: private void sendMessage(int destId, String msg) { Charset charset = Charset.forName("ISO-8859-1"); CharBuffer charBuffer = CharBuffer.wrap(msg, 0, msg.length()); ByteBuffer bf = charset.encode(charBuffer); //bf.flip(); int bytesSent = 0; SelectionKey key = clientsMap.getClient(destId); SocketChannel sc = (SocketChannel) key.channel(); try { / int total_bytes_sent = 0; while(total_bytes_sent < msg.length()){ bytesSent = sc.write(bf); total_bytes_sent += bytesSent; } System.out.println("LOG: Bytes enviados para o cliente " + destId + ": "+ total_bytes_sent + " Tamanho da mensagem: " + msg.length()); } catch (IOException e) { System.out.println("LOG: Erro ao mandar mensagem para: " + destId); System.out.println("LOG: " + e.getLocalizedMessage()); } } So, what is happening is that the server, when send a message, prints something like this: LOG: Bytes sent to the client: 28 Size of the message: 28 So, it tells that it sent the data, but the chat client keeps blocking, waiting in the recv() method. So, the data never gets to it. When I close the server application, though, all the data appears in the client. I wonder why. It is important to say that the client is in C and the server JAVA, and I'm running both in the same machine, an Ubuntu Guest in virtualbox under windows. I also run both under windows host and under linuxes hosts, and keep getting the same strange problem. I'm sorry for the great lenght of this question, but I already searched a lot of places for an answer, found a lot of tutorials and questions, including here at StackOverflow, but coundn't find a reasonable explanation. I am really not liking this Java NIO, and i saw a lot of people complaining about it too. I am thinking that if I had done that in C it would have been a lot easier :-D So, if someone could help me and even discuss this behavor, it would be great! :-) Thanks everybody in advance, Péterson

Read the article

how would you debug this javascript problem?

- by pedalpete

I've been travelling and developing for the past few weeks. The site I'm developing was running well. Then, the other day, i connected to a network and the page 'looked' fine, but it turns out the javascript wasn't running. I checked firebug, and there were no errors, as I was suspecting that maybe a script didn't load (I'm using the google api for jQuery and jQuery UI, as well as loading google maps api and fbconnect). I would suspect that if the issue was with one of these pages not loading I would get an error, and yet there was nothing. Thinking maybe i didn't connect properly or something, i reconnected to the network and even restarted my computer, as well as trying to run the local version. I got nothing. The local version not running also hinted to me that it was the loading of an external javascript which caused the problem. I let it pass as something strange with that one network. Unfortunately now I'm 100s of miles away. Today my brother sent me an e-mail that the network he was on at the airport wouldn't load my page. Same issue. Everything is laid out properly, and part of the layout is set in Javascript, so clearly javascript is running. he too got no errors. Of course, he got on his plane, and now he is no longer at the airport. Now the site works on his computer (and i haven't changed anything). How on earth would you go about figuring out what happened in this situation? That is two of maybe 12 or so networks. But I have no idea how i would find a network that doesn't work (and living in a small town, it could be difficult for me to find a network that doesn't work). Any ideas? The site is still in Dev, so I'd rather not post a link just yet (but could in a few days). What I can see not working is the javascript functions which are called on load, and on click. So i do think it is a javascript issue, but no errors. This wouldn't be as HUGE an issue if I could find and sit on one of these networks, but I can't. So what would you do? EDIT ---------------------------------------------------------- the first function(s - their linked) that doesn't get called is below. I've cut the code of at the .ajax call as the call wasn't being made. function getResultsFromForm(){ jQuery('form#filterList input.button').hide(); var searchAddress=jQuery('form#filterList input#searchTxt').val(); if(searchAddress=='' || searchAddress=='<?php echo $searchLocation; ?>'){ mapShow(20, -40, 0, 'areaMap', 2); jQuery('form#filterList input.button').show(); return; } if (GBrowserIsCompatible()) { var geo = new GClientGeocoder(); geo.setBaseCountryCode(cl.address.country); geo.getLocations(searchAddress, function (result) { if(!result.Placemark && searchAddress!='<?php echo $searchLocation; ?>'){ jQuery('span#addressNotFound').text('<?php echo $addressNotFound; ?>').slideDown('slow'); jQuery('form#filterList input.button').show(); } else { jQuery('span#addressNotFound').slideUp('slow').empty(); jQuery('span#headerLocal').text(searchAddress); var date = new Date(); date.setTime(date.getTime() + (8 * 24 * 60 * 60 * 1000)); jQuery.cookie('address', searchAddress, { expires: date}); var accuracy= result.Placemark[0].AddressDetails.Accuracy; var lat = result.Placemark[0].Point.coordinates[1]; var long = result.Placemark[0].Point.coordinates[0]; lat=parseFloat(lat); long=parseFloat(long); var getTab=jQuery('div#tabs div#active').attr('class'); jQuery('div#tabs').show(); loadForecast(lat, long, getTab, 'true', 0); var zoom=zoomLevel(); mapShow(lat, long, accuracy, 'areaMap', zoom ); } }); } } function zoomLevel(){ var zoomarray= new Array(); zoomarray=jQuery('span.viewDist').attr('id'); zoomarray=zoomarray.split("-"); var zoom=zoomarray[1]; if(zoom==''){ zoom=5; } zoom=parseFloat(zoom); return(zoom); } function loadForecast(lat, long, type, loadForecast, page){ jQuery('div#holdForecast').empty(); var date = new Date(); var d = date.getDate(); var day = (d < 10) ? '0' + d : d; var m = date.getMonth() + 1; var month = (m < 10) ? '0' + m : m; var year='2009'; toDate=year+'-'+month+'-'+day; var genre=jQuery('span.genreblock span#updateGenre').html(); var numDays=''; var numResults=''; var range=jQuery('span.viewDist').attr('id'); var dateRange = jQuery('.updateDate').attr('id'); jQuery('div#holdShows ul.showList').html('<li class="show"><div class="showData"><center><img src="../hwImages/loading.gif"/></center></div></li>'); jQuery('div#holdShows ul.'+type+'List').livequery(function(){ jQuery.ajax({ type: "GET", url: "processes/formatShows.php", data: "output=&genre="+genre+"&numResults="+numResults+"&date="+toDate+"&dateRange="+dateRange+"&range="+range+"&lat="+lat+"&long="+long+'&page='+page, success: function(response){ EDIT 2 ----------------------------------------------------------------------------- Please keep in mind that the problem is not that I can't load the site, the site works fine on most connections, but there are times when the site doesn't work, and no errors are thrown, and nothing changes. My brother couldn't run it earlier today while I had no problems, so it was something to do with his location/network. HOWEVER, the page loads, he had a connection, it was his first time visiting the site, so nothing could have been cashed. Same with when I had the issue a few days before. I didn't change anything, and I got to a different network and everything worked fine.

Read the article

NullPointerException when linking to Service that uses ContentProvider

- by Danny Chia

H.i everyone, this is my first post here! Anyways, I'm trying to write a "todo list" application. It stores the data in a ContentProvider, which is accessed via a Service. However, my app crashes at launch. My code is below: Manifest file: <?xml version="1.0" encoding="utf-8"?> <manifest xmlns:android="http://schemas.android.com/apk/res/android" package="com.examples.todolist" android:versionCode="1" android:versionName="1.0"> <application android:icon="@drawable/icon" android:label="@string/app_name" android:debuggable="True"> <activity android:name=".ToDoList" android:label="@string/app_name" android:theme="@style/ToDoTheme"> <intent-filter> <action android:name="android.intent.action.MAIN" /> <category android:name="android.intent.category.LAUNCHER" /> </intent-filter> </activity> <service android:name="TodoService"/> <provider android:name="TodoProvider" android:authorities="com.examples.provider.todolist" /> </application> <uses-sdk android:minSdkVersion="7" /> </manifest> ToDoList.java: package com.examples.todolist; import com.examples.todolist.TodoService.LocalBinder; import java.util.ArrayList; import java.util.Date; import android.app.Activity; import android.content.SharedPreferences; import android.database.Cursor; import android.os.AsyncTask; import android.os.Bundle; import android.view.ContextMenu; import android.content.ComponentName; import android.content.Context; import android.content.Intent; import android.content.ServiceConnection; import android.os.IBinder; import android.view.KeyEvent; import android.view.Menu; import android.view.MenuItem; import android.view.View; import android.view.View.OnKeyListener; import android.widget.AdapterView; import android.widget.EditText; import android.widget.ListView; import android.widget.Toast; public class ToDoList extends Activity { static final private int ADD_NEW_TODO = Menu.FIRST; static final private int REMOVE_TODO = Menu.FIRST + 1; private static final String TEXT_ENTRY_KEY = "TEXT_ENTRY_KEY"; private static final String ADDING_ITEM_KEY = "ADDING_ITEM_KEY"; private static final String SELECTED_INDEX_KEY = "SELECTED_INDEX_KEY"; private boolean addingNew = false; private ArrayList<ToDoItem> todoItems; private ListView myListView; private EditText myEditText; private ToDoItemAdapter aa; int entries = 0; int notifs = 0; //ToDoDBAdapter toDoDBAdapter; Cursor toDoListCursor; TodoService mService; boolean mBound = false; /** Called when the activity is first created. */ public void onCreate(Bundle icicle) { super.onCreate(icicle); setContentView(R.layout.main); myListView = (ListView)findViewById(R.id.myListView); myEditText = (EditText)findViewById(R.id.myEditText); todoItems = new ArrayList<ToDoItem>(); int resID = R.layout.todolist_item; aa = new ToDoItemAdapter(this, resID, todoItems); myListView.setAdapter(aa); myEditText.setOnKeyListener(new OnKeyListener() { public boolean onKey(View v, int keyCode, KeyEvent event) { if (event.getAction() == KeyEvent.ACTION_DOWN) if (keyCode == KeyEvent.KEYCODE_DPAD_CENTER) { ToDoItem newItem = new ToDoItem(myEditText.getText().toString(), 0); mService.insertTask(newItem); updateArray(); myEditText.setText(""); entries++; Toast.makeText(ToDoList.this, "Entry added", Toast.LENGTH_SHORT).show(); aa.notifyDataSetChanged(); cancelAdd(); return true; } return false; } }); registerForContextMenu(myListView); restoreUIState(); populateTodoList(); } private void populateTodoList() { // Get all the todo list items from the database. toDoListCursor = mService. getAllToDoItemsCursor(); startManagingCursor(toDoListCursor); // Update the array. updateArray(); Toast.makeText(this, "Todo list retrieved", Toast.LENGTH_SHORT).show(); } private void updateArray() { toDoListCursor.requery(); todoItems.clear(); if (toDoListCursor.moveToFirst()) do { String task = toDoListCursor.getString(toDoListCursor.getColumnIndex(ToDoDBAdapter.KEY_TASK)); long created = toDoListCursor.getLong(toDoListCursor.getColumnIndex(ToDoDBAdapter.KEY_CREATION_DATE)); int taskid = toDoListCursor.getInt(toDoListCursor.getColumnIndex(ToDoDBAdapter.KEY_ID)); ToDoItem newItem = new ToDoItem(task, new Date(created), taskid); todoItems.add(0, newItem); } while(toDoListCursor.moveToNext()); aa.notifyDataSetChanged(); } private void restoreUIState() { // Get the activity preferences object. SharedPreferences settings = getPreferences(0); // Read the UI state values, specifying default values. String text = settings.getString(TEXT_ENTRY_KEY, ""); Boolean adding = settings.getBoolean(ADDING_ITEM_KEY, false); // Restore the UI to the previous state. if (adding) { addNewItem(); myEditText.setText(text); } } @Override public void onSaveInstanceState(Bundle outState) { outState.putInt(SELECTED_INDEX_KEY, myListView.getSelectedItemPosition()); super.onSaveInstanceState(outState); } @Override public void onRestoreInstanceState(Bundle savedInstanceState) { int pos = -1; if (savedInstanceState != null) if (savedInstanceState.containsKey(SELECTED_INDEX_KEY)) pos = savedInstanceState.getInt(SELECTED_INDEX_KEY, -1); myListView.setSelection(pos); } @Override protected void onPause() { super.onPause(); // Get the activity preferences object. SharedPreferences uiState = getPreferences(0); // Get the preferences editor. SharedPreferences.Editor editor = uiState.edit(); // Add the UI state preference values. editor.putString(TEXT_ENTRY_KEY, myEditText.getText().toString()); editor.putBoolean(ADDING_ITEM_KEY, addingNew); // Commit the preferences. editor.commit(); } @Override public boolean onCreateOptionsMenu(Menu menu) { super.onCreateOptionsMenu(menu); // Create and add new menu items. MenuItem itemAdd = menu.add(0, ADD_NEW_TODO, Menu.NONE, R.string.add_new); MenuItem itemRem = menu.add(0, REMOVE_TODO, Menu.NONE, R.string.remove); // Assign icons itemAdd.setIcon(R.drawable.add_new_item); itemRem.setIcon(R.drawable.remove_item); // Allocate shortcuts to each of them. itemAdd.setShortcut('0', 'a'); itemRem.setShortcut('1', 'r'); return true; } @Override public boolean onPrepareOptionsMenu(Menu menu) { super.onPrepareOptionsMenu(menu); int idx = myListView.getSelectedItemPosition(); String removeTitle = getString(addingNew ? R.string.cancel : R.string.remove); MenuItem removeItem = menu.findItem(REMOVE_TODO); removeItem.setTitle(removeTitle); removeItem.setVisible(addingNew || idx > -1); return true; } @Override public void onCreateContextMenu(ContextMenu menu, View v, ContextMenu.ContextMenuInfo menuInfo) { super.onCreateContextMenu(menu, v, menuInfo); menu.setHeaderTitle("Selected To Do Item"); menu.add(0, REMOVE_TODO, Menu.NONE, R.string.remove); } @Override public boolean onOptionsItemSelected(MenuItem item) { super.onOptionsItemSelected(item); int index = myListView.getSelectedItemPosition(); switch (item.getItemId()) { case (REMOVE_TODO): { if (addingNew) { cancelAdd(); } else { removeItem(index); } return true; } case (ADD_NEW_TODO): { addNewItem(); return true; } } return false; } @Override public boolean onContextItemSelected(MenuItem item) { super.onContextItemSelected(item); switch (item.getItemId()) { case (REMOVE_TODO): { AdapterView.AdapterContextMenuInfo menuInfo; menuInfo =(AdapterView.AdapterContextMenuInfo)item.getMenuInfo(); int index = menuInfo.position; removeItem(index); return true; } } return false; } @Override public void onDestroy() { super.onDestroy(); } private void cancelAdd() { addingNew = false; myEditText.setVisibility(View.GONE); } private void addNewItem() { addingNew = true; myEditText.setVisibility(View.VISIBLE); myEditText.requestFocus(); } private void removeItem(int _index) { // Items are added to the listview in reverse order, so invert the index. //toDoDBAdapter.removeTask(todoItems.size()-_index); ToDoItem item = todoItems.get(_index); final long selectedId = item.getTaskId(); mService.removeTask(selectedId); entries--; Toast.makeText(this, "Entry deleted", Toast.LENGTH_SHORT).show(); updateArray(); } @Override protected void onStart() { super.onStart(); Intent intent = new Intent(this, TodoService.class); bindService(intent, mConnection, Context.BIND_AUTO_CREATE); } @Override protected void onStop() { super.onStop(); // Unbind from the service if (mBound) { unbindService(mConnection); mBound = false; } } private ServiceConnection mConnection = new ServiceConnection() { public void onServiceConnected(ComponentName className, IBinder service) { LocalBinder binder = (LocalBinder) service; mService = binder.getService(); mBound = true; } public void onServiceDisconnected(ComponentName arg0) { mBound = false; } }; public class TimedToast extends AsyncTask<Long, Integer, Integer> { @Override protected Integer doInBackground(Long... arg0) { if (notifs < 15) { try { Toast.makeText(ToDoList.this, entries + " entries left", Toast.LENGTH_SHORT).show(); notifs++; Thread.sleep(20000); } catch (InterruptedException e) { } } return 0; } } } TodoService.java: package com.examples.todolist; import android.app.Service; import android.content.ContentResolver; import android.content.ContentValues; import android.content.Intent; import android.database.Cursor; import android.os.Binder; import android.os.IBinder; public class TodoService extends Service { private final IBinder mBinder = new LocalBinder(); @Override public IBinder onBind(Intent arg0) { return mBinder; } public class LocalBinder extends Binder { TodoService getService() { return TodoService.this; } } public void insertTask(ToDoItem _task) { ContentResolver cr = getContentResolver(); ContentValues values = new ContentValues(); values.put(TodoProvider.KEY_CREATION_DATE, _task.getCreated().getTime()); values.put(TodoProvider.KEY_TASK, _task.getTask()); cr.insert(TodoProvider.CONTENT_URI, values); } public void updateTask(ToDoItem _task) { long tid = _task.getTaskId(); ContentResolver cr = getContentResolver(); ContentValues values = new ContentValues(); values.put(TodoProvider.KEY_TASK, _task.getTask()); cr.update(TodoProvider.CONTENT_URI, values, TodoProvider.KEY_ID + "=" + tid, null); } public void removeTask(long tid) { ContentResolver cr = getContentResolver(); cr.delete(TodoProvider.CONTENT_URI, TodoProvider.KEY_ID + "=" + tid, null); } public Cursor getAllToDoItemsCursor() { ContentResolver cr = getContentResolver(); return cr.query(TodoProvider.CONTENT_URI, null, null, null, null); } } TodoProvider.java: package com.examples.todolist; import android.content.*; import android.database.Cursor; import android.database.SQLException; import android.database.sqlite.SQLiteOpenHelper; import android.database.sqlite.SQLiteDatabase; import android.database.sqlite.SQLiteQueryBuilder; import android.database.sqlite.SQLiteDatabase.CursorFactory; import android.net.Uri; import android.text.TextUtils; import android.util.Log; public class TodoProvider extends ContentProvider { public static final Uri CONTENT_URI = Uri.parse("content://com.examples.provider.todolist/todo"); @Override public boolean onCreate() { Context context = getContext(); todoHelper dbHelper = new todoHelper(context, DATABASE_NAME, null, DATABASE_VERSION); todoDB = dbHelper.getWritableDatabase(); return (todoDB == null) ? false : true; } @Override public Cursor query(Uri uri, String[] projection, String selection, String[] selectionArgs, String sort) { SQLiteQueryBuilder tb = new SQLiteQueryBuilder(); tb.setTables(TODO_TABLE); // If this is a row query, limit the result set to the passed in row. switch (uriMatcher.match(uri)) { case TASK_ID: tb.appendWhere(KEY_ID + "=" + uri.getPathSegments().get(1)); break; default: break; } // If no sort order is specified sort by date / time String orderBy; if (TextUtils.isEmpty(sort)) { orderBy = KEY_ID; } else { orderBy = sort; } // Apply the query to the underlying database. Cursor c = tb.query(todoDB, projection, selection, selectionArgs, null, null, orderBy); // Register the contexts ContentResolver to be notified if // the cursor result set changes. c.setNotificationUri(getContext().getContentResolver(), uri); // Return a cursor to the query result. return c; } @Override public Uri insert(Uri _uri, ContentValues _initialValues) { // Insert the new row, will return the row number if // successful. long rowID = todoDB.insert(TODO_TABLE, "task", _initialValues); // Return a URI to the newly inserted row on success. if (rowID > 0) { Uri uri = ContentUris.withAppendedId(CONTENT_URI, rowID); getContext().getContentResolver().notifyChange(uri, null); return uri; } throw new SQLException("Failed to insert row into " + _uri); } @Override public int delete(Uri uri, String where, String[] whereArgs) { int count; switch (uriMatcher.match(uri)) { case TASKS: count = todoDB.delete(TODO_TABLE, where, whereArgs); break; case TASK_ID: String segment = uri.getPathSegments().get(1); count = todoDB.delete(TODO_TABLE, KEY_ID + "=" + segment + (!TextUtils.isEmpty(where) ? " AND (" + where + ')' : ""), whereArgs); break; default: throw new IllegalArgumentException("Unsupported URI: " + uri); } getContext().getContentResolver().notifyChange(uri, null); return count; } @Override public int update(Uri uri, ContentValues values, String where, String[] whereArgs) { int count; switch (uriMatcher.match(uri)) { case TASKS: count = todoDB.update(TODO_TABLE, values, where, whereArgs); break; case TASK_ID: String segment = uri.getPathSegments().get(1); count = todoDB.update(TODO_TABLE, values, KEY_ID + "=" + segment + (!TextUtils.isEmpty(where) ? " AND (" + where + ')' : ""), whereArgs); break; default: throw new IllegalArgumentException("Unknown URI " + uri); } getContext().getContentResolver().notifyChange(uri, null); return count; } @Override public String getType(Uri uri) { switch (uriMatcher.match(uri)) { case TASKS: return "vnd.android.cursor.dir/vnd.examples.task"; case TASK_ID: return "vnd.android.cursor.item/vnd.examples.task"; default: throw new IllegalArgumentException("Unsupported URI: " + uri); } } // Create the constants used to differentiate between the different URI // requests. private static final int TASKS = 1; private static final int TASK_ID = 2; private static final UriMatcher uriMatcher; // Allocate the UriMatcher object, where a URI ending in 'tasks' will // correspond to a request for all tasks, and 'tasks' with a // trailing '/[rowID]' will represent a single task row. static { uriMatcher = new UriMatcher(UriMatcher.NO_MATCH); uriMatcher.addURI("com.examples.provider.Todolist", "tasks", TASKS); uriMatcher.addURI("com.examples.provider.Todolist", "tasks/#", TASK_ID); } //The underlying database private SQLiteDatabase todoDB; private static final String TAG = "TodoProvider"; private static final String DATABASE_NAME = "todolist.db"; private static final int DATABASE_VERSION = 1; private static final String TODO_TABLE = "todolist"; // Column Names public static final String KEY_ID = "_id"; public static final String KEY_TASK = "task"; public static final String KEY_CREATION_DATE = "date"; public long insertTask(ToDoItem _task) { // Create a new row of values to insert. ContentValues newTaskValues = new ContentValues(); // Assign values for each row. newTaskValues.put(KEY_TASK, _task.getTask()); newTaskValues.put(KEY_CREATION_DATE, _task.getCreated().getTime()); // Insert the row. return todoDB.insert(TODO_TABLE, null, newTaskValues); } public boolean updateTask(long _rowIndex, String _task) { ContentValues newValue = new ContentValues(); newValue.put(KEY_TASK, _task); return todoDB.update(TODO_TABLE, newValue, KEY_ID + "=" + _rowIndex, null) > 0; } public boolean removeTask(long _rowIndex) { return todoDB.delete(TODO_TABLE, KEY_ID + "=" + _rowIndex, null) > 0; } // Helper class for opening, creating, and managing database version control private static class todoHelper extends SQLiteOpenHelper { private static final String DATABASE_CREATE = "create table " + TODO_TABLE + " (" + KEY_ID + " integer primary key autoincrement, " + KEY_TASK + " TEXT, " + KEY_CREATION_DATE + " INTEGER);"; public todoHelper(Context cn, String name, CursorFactory cf, int ver) { super(cn, name, cf, ver); } @Override public void onCreate(SQLiteDatabase db) { db.execSQL(DATABASE_CREATE); } @Override public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) { Log.w(TAG, "Upgrading database from version " + oldVersion + " to " + newVersion + ", which will destroy all old data"); db.execSQL("DROP TABLE IF EXISTS " + TODO_TABLE); onCreate(db); } } } I've omitted the other files as I'm sure they are correct. When I run the program, LogCat shows that the NullPointerException occurs in populateTodoList(), at toDoListCursor = mService.getAllToDoItemsCursor(). mService is the Cursor object returned by TodoService. I've added the service to the Manifest file, but I still cannot find out why it's causing an exception. Thanks in advance.

Read the article

Dealing with external processes

- by Jesse Aldridge

I've been working on a gui app that needs to manage external processes. Working with external processes leads to a lot of issues that can make a programmer's life difficult. I feel like maintenence on this app is taking an unacceptably long time. I've been trying to list the things that make working with external processes difficult so that I can come up with ways of mitigating the pain. This kind of turned into a rant which I thought I'd post here in order to get some feedback and to provide some guidance to anybody thinking about sailing into these very murky waters. Here's what I've got so far: Output from the child can get mixed up with output from the parent. This can make both outputs misleading and hard to read. It can be hard to tell what came from where. It becomes harder to figure out what's going on when things are asynchronous. Here's a contrived example: import textwrap, os, time from subprocess import Popen test_path = 'test_file.py' with open(test_path, 'w') as file: file.write(textwrap.dedent(''' import time for i in range(3): print 'Hello %i' % i time.sleep(1)''')) proc = Popen('python -B "%s"' % test_path) for i in range(3): print 'Hello %i' % i time.sleep(1) os.remove(test_path) I guess I could have the child process write its output to a file. But it can be annoying to have to open up a file every time I want to see the result of a print statement. If I have code for the child process I could add a label, something like print 'child: Hello %i', but it can be annoying to do that for every print. And it adds some noise to the output. And of course I can't do it if I don't have access to the code. I could manually manage the process output. But then you open up a huge can of worms with threads and polling and stuff like that. A simple solution is to treat processes like synchronous functions, that is, no further code executes until the process completes. In other words, make the process block. But that doesn't work if you're building a gui app. Which brings me to the next problem... Blocking processes cause the gui to become unresponsive. import textwrap, sys, os from subprocess import Popen from PyQt4.QtGui import * from PyQt4.QtCore import * test_path = 'test_file.py' with open(test_path, 'w') as file: file.write(textwrap.dedent(''' import time for i in range(3): print 'Hello %i' % i time.sleep(1)''')) app = QApplication(sys.argv) button = QPushButton('Launch process') def launch_proc(): # Can't move the window until process completes proc = Popen('python -B "%s"' % test_path) proc.communicate() button.connect(button, SIGNAL('clicked()'), launch_proc) button.show() app.exec_() os.remove(test_path) Qt provides a process wrapper of its own called QProcess which can help with this. You can connect functions to signals to capture output relatively easily. This is what I'm currently using. But I'm finding that all these signals behave suspiciously like goto statements and can lead to spaghetti code. I think I want to get sort-of blocking behavior by having the 'finished' signal from QProcess call a function containing all the code that comes after the process call. I think that should work but I'm still a bit fuzzy on the details... Stack traces get interrupted when you go from the child process back to the parent process. If a normal function screws up, you get a nice complete stack trace with filenames and line numbers. If a subprocess screws up, you'll be lucky if you get any output at all. You end up having to do a lot more detective work everytime something goes wrong. Speaking of which, output has a way of disappearing when dealing external processes. Like if you run something via the windows 'cmd' command, the console will pop up, execute the code, and then disappear before you have a chance to see the output. You have to pass the /k flag to make it stick around. Similar issues seem to crop up all the time. I suppose both problems 3 and 4 have the same root cause: no exception handling. Exception handling is meant to be used with functions, it doesn't work with processes. Maybe there's some way to get something like exception handling for processes? I guess that's what stderr is for? But dealing with two different streams can be annoying in itself. Maybe I should look into this more... Processes can hang and stick around in the background without you realizing it. So you end up yelling at your computer cuz it's going so slow until you finally bring up your task manager and see 30 instances of the same process hanging out in the background. Also, hanging background processes can interefere with other instances of the process in various fun ways, such as causing permissions errors by holding a handle to a file or someting like that. It seems like an easy solution to this would be to have the parent process kill the child process on exit if the child process didn't close itself. But if the parent process crashes, cleanup code might not get called and the child can be left hanging. Also, if the parent waits for the child to complete, and the child is in an infinite loop or something, you can end up with two hanging processes. This problem can tie in to problem 2 for extra fun, causing your gui to stop responding entirely and force you to kill everything with the task manager. F***ing quotes Parameters often need to be passed to processes. This is a headache in itself. Especially if you're dealing with file paths. Say... 'C:/My Documents/whatever/'. If you don't have quotes, the string will often be split at the space and interpreted as two arguments. If you need nested quotes you can use ' and ". But if you need to use more than two layers of quotes, you have to do some nasty escaping, for example: "cmd /k 'python \'path 1\' \'path 2\''". A good solution to this problem is passing parameters as a list rather than as a single string. Subprocess allows you to do this. Can't easily return data from a subprocess. You can use stdout of course. But what if you want to throw a print in there for debugging purposes? That's gonna screw up the parent if it's expecting output formatted a certain way. In functions you can print one string and return another and everything works just fine. Obscure command-line flags and a crappy terminal based help system. These are problems I often run into when using os level apps. Like the /k flag I mentioned, for holding a cmd window open, who's idea was that? Unix apps don't tend to be much friendlier in this regard. Hopefully you can use google or StackOverflow to find the answer you need. But if not, you've got a lot of boring reading and frusterating trial and error to do. External factors. This one's kind of fuzzy. But when you leave the relatively sheltered harbor of your own scripts to deal with external processes you find yourself having to deal with the "outside world" to a much greater extent. And that's a scary place. All sorts of things can go wrong. Just to give a random example: the cwd in which a process is run can modify it's behavior. There are probably other issues, but those are the ones I've written down so far. Any other snags you'd like to add? Any suggestions for dealing with these problems?

Read the article

How to find an entry-level job after you already have a graduate degree?

- by Uri

Note: I asked this question in early 2009. A couple of months later, I found a great job. I've previously updated this question with some tips for whoever ends up in a similar situation, and now cleaned it up a little for the benefit of the fresh batch of graduates. Original post: In my early 20s I abandoned a great C++ development career path in a major company to go to graduate school and get a research masters (3 years). I did another year in industrial research, and then moved to the US to attend graduate school again, getting another masters and a Ph.D in software engineering from a top school (another 6 years down the drain). I was coding the whole way throughout my degrees (core Java and Eclipse plug-ins) and working on research related to software engineering (usability of APIs). I ended up graduating the year of the recession, with a son on the way and the prospects of no healthcare. Academic jobs and industrial research jobs are quite scarce. Initially, I was naive, thinking that with my background, I could easily find a coding job. Big mistake. It turns out that I'm in a complicated position. Entry level positions are usually offered to college undergraduates. I attended my school's career fairs, but you could immediately see signs of Ph.D. aversion and overqualification issues. Some of the recruiters I spoke with explicitly told me that they wanted 20 year olds with clean slates, and some were looking for interns since they are in various forms of hiring freezes. I managed to get a couple of interviews from these career fairs and through recruiters. However, since I've been out of school for a long time and programming primarily in Java, I am also no longer proficient in C/C++ and the usual range of college-level interview questions that everyone uses. I had no problems with this when I was 19 and interviewing for my first job since a lot of what you do in C is manipulate pointers and I was coding C++ for fun and for school. Later I was routinely doing pointer manipulation on the job, and during my first masters taught college courses with data structures and C++. But even though I remember many properties of C++ well, it's been close to ten years since I regularly used C++ and pointers. As a Java developer I rarely had to work at this level, but experience in OOD and in writing good maintainable code is meaningless for C++ interviews. Reading books as a refresh and looking at sample code did not do the trick. I also looked at mid-to-senior level Java positions, but most of them focused on J2EE APIs rather than on core Java and required a certain number of years in industrial positions. Coding research tools and prior C++ experience doesn't count. So that sends me back to entry-level jobs that are posted through job-boards, and these are not common (mostly they are Monster junk), and small companies are even less likely to answer a Ph.D. compared to the giants who participate in top-10 career fairs. Even worse, in many companies initial screening is done by HR folks who really don't want to deal with anything anomalous like a Ph.D. Any tips on how I should approach this intractable position? For example, what should I write in cover letters? Note that while immigration is not an issue for me, I cannot go freelance as I need the benefits (and in particular group health insurance). During my studies I had no time to contribute to open-source projects or maintain a popular blog, so even if I invested in that now there would be no immediate benefit. Updates: In the two months after posting this I received several offers to work as a core Java developer in the financial industry and accepted one from a firm where I am working to this day. For those who find themselves in similar situations, here are my tips: Give up on trying to find an entry level positions. You can't undo time. Accept the fact that there is Ph.D. discrimination in the job market (some might say rightfully so). It is legal to discriminate based on education. No point fighting it. The most important tip is to focus on the language you are comfortable with. The sad truth about programming in a particular language is that it is not like riding a bike. If you haven't used a language in the last few years, and can't actually apply it routinely (not just as a refresher) before you start your search, it is going to be very difficult to do well in an interview. Now that I'm interviewing others, I routinely see it in folks with a mixed C++/Java background. We maintain "a shadow" of the old language but end up with a weird mix that makes it hard to interview on either. Entry-level folks are at an advantage here since they usually have one language. Memory can help you do great in a screening interview, but without recent day-to-day experience, code tests will be difficult. Despite the supposed relation, core Java programming and J2EE programming are two different things with different skillsets. If you come from academia, you likely have very little J2EE experience and may find it hard to get accepted for a J2EE job. J2EE jobs seem to have a larger list of acronyms in their requirements. In addition, from interviewing J2EE developers it seems that for many there is a focus on mastering specific APIs and architectures, whereas core Java development tends to be secondary. In the same way that I can no longer manipulate pointers well, a J2EE developer may have difficulties doing low level Java manipulation. This puts you at a relative advantage in competing for core Java jobs! If you are able to work for startups (in terms of family life and stability) or migrate to startup-rich areas such as the west coast, you can find many exciting opportunities where advanced degrees are a benefit. I've since been approached by several startups, although I had to decline. Work through a recruiter if possible. They have direct contacts with the hiring parties, allowing you to "stand out". It is better to get a clear yes/no confirmation from a recruiter on whether a company might be interested in interviewing you, than it is to send your resume and hope that someone will ever see it. Recruiters are also a great way of bypassing HR. However, also beware of recruiters. They have a vested interest and will go to various shady practices and pressure tactics. To find a good recruiter, talk to a friend who declined a job offer he got through a recruiter. A good recruiter, to me, is measured in how they handle that. Interview for the jobs that require your core strength. If you're rusty or entirely unfamiliar with a technology around which the job revolves, you're probably not a good match. Yes, you probably have the talent to master them, but most companies would want "instant gratification". I got my offers from companies that wanted core Java developer. I didn't do well on places that wanted advance C++ because I am too rusty and not up to date on recent libraries. I also didn't hear from companies that wanted lots of J2EE experience, and that's ok. Finding companies that want core Java without web is harder, but exists in specific industries (e.g., finance, defense). This requires a lot more legwork in terms of search, but these jobs do exist. There are different interview styles. Some companies focus on puzzles, some companies focus on algorithms, and some companies focus on design and coding skills. I had the most success in places where the questions were the most related to the function I would have been performing. Pick companies accordingly as well.

Read the article

C++ simple logging class with UTF-8 output [code example]

- by Andrew

Hello everyone, I was working on one of my academic projects and for the first time I needed pure C++ without GUI. After googling for a while, I did not find any simple and easy to use implementation for logging and created my own. This is a simple implementation with iostreams that logs messages to screen and to the file simultaneously. I was thinking of using templates but then I realized that I do not expect any changes and removed that. It is modified std::wostream with two added modifiers: 1. TimeStamp - prints time-stamp 2. LogMode(LogModes) - switches output: file only, screen only, file+screen. *Boost::utf8_codecvt_facet* is used for UTF-8 output. // ############################################################################ // # Name: MyLog.h # // # Purpose: Logging Class Header # // # Author: Andrew Drach # // # Modified by: <somebody> # // # Created: 03/21/10 # // # SVN-ID: $Id$ # // # Copyright: (c) 2010 Andrew Drach # // # Licence: <license> # // ############################################################################ #ifndef INCLUDED_MYLOG_H #define INCLUDED_MYLOG_H // headers -------------------------------------------------------------------- #include <string> #include <iostream> #include <fstream> #include <exception> #include <boost/program_options/detail/utf8_codecvt_facet.hpp> using namespace std; // definitions ---------------------------------------------------------------- // ---------------------------------------------------------------------------- // DblBuf class // Splits up output stream into two // Inspired by http://wordaligned.org/articles/cpp-streambufs // ---------------------------------------------------------------------------- class DblBuf : public wstreambuf { private: // private member declarations DblBuf(); wstreambuf *bf1; wstreambuf *bf2; virtual int_type overflow(int_type ch) { int_type eof = traits_type::eof(); int_type not_eof = !eof; if ( traits_type::eq_int_type(ch,eof) ) return not_eof; else { char_type ch1 = traits_type::to_char_type(ch); int_type r1( bf1on ? bf1->sputc(ch1) : not_eof ); int_type r2( bf2on ? bf2->sputc(ch1) : not_eof ); return (traits_type::eq_int_type(r1,eof) || traits_type::eq_int_type(r2,eof) ) ? eof : ch; } } virtual int sync() { int r1( bf1on ? bf1->pubsync() : NULL ); int r2( bf2on ? bf2->pubsync() : NULL ); return (r1 == 0 && r2 == 0) ? 0 : -1; } public: // public member declarations explicit DblBuf(wstreambuf *bf1, wstreambuf *bf2) : bf1(bf1), bf2(bf2) { if (bf1) bf1on = true; else bf1on = false; if (bf2) bf2on = true; else bf2on = false; } bool bf1on; bool bf2on; }; // ---------------------------------------------------------------------------- // logstream class // Wrapper for a standard wostream with access to modified buffer // ---------------------------------------------------------------------------- class logstream : public wostream { private: // private member declarations logstream(); public: // public member declarations DblBuf *buf; explicit logstream(wstreambuf *StrBuf, bool isStd = false) : wostream(StrBuf, isStd), buf((DblBuf*)StrBuf) {} }; // ---------------------------------------------------------------------------- // Logging mode Class // ---------------------------------------------------------------------------- enum LogModes{LogToFile=1, LogToScreen, LogToBoth}; class LogMode { private: // private member declarations LogMode(); short mode; public: // public member declarations LogMode(short mode1) : mode(mode1) {} logstream& operator()(logstream &stream1) { switch(mode) { case LogToFile: stream1.buf->bf1on = true; stream1.buf->bf2on = false; break; case LogToScreen: stream1.buf->bf1on = false; stream1.buf->bf2on = true; break; case LogToBoth: stream1.buf->bf1on = true; stream1.buf->bf2on = true; } return stream1; } }; logstream& operator<<(logstream &out, LogMode mode) { return mode(out); } wostream& TimeStamp1(wostream &out1) { time_t time1; struct tm timeinfo; wchar_t timestr[512]; // Get current time and convert it to a string time(&time1); localtime_s (&timeinfo, &time1); wcsftime(timestr, 512,L"[%Y-%b-%d %H:%M:%S %p] ",&timeinfo); return out1 << timestr; } // ---------------------------------------------------------------------------- // MyLog class // Logs events to both file and screen // ---------------------------------------------------------------------------- class MyLog { private: // private member declarations MyLog(); auto_ptr<DblBuf> buf; string mErrorMsg1; string mErrorMsg2; string mErrorMsg3; string mErrorMsg4; public: // public member declarations explicit MyLog(string FileName1, wostream *ScrLog1, locale utf8locale1); ~MyLog(); void NewEvent(wstring str1, bool TimeStamp = true); string FileName; wostream *ScrLog; wofstream File; auto_ptr<logstream> Log; locale utf8locale; }; // ---------------------------------------------------------------------------- // MyLog constructor // ---------------------------------------------------------------------------- MyLog::MyLog(string FileName1, wostream *ScrLog1, locale utf8locale1) : // ctors mErrorMsg1("Failed to open file for application logging! []"), mErrorMsg2("Failed to write BOM! []"), mErrorMsg3("Failed to write to file! []"), mErrorMsg4("Failed to close file! []"), FileName(FileName1), ScrLog(ScrLog1), utf8locale(utf8locale1), File(FileName1.c_str()) { // Adjust error strings mErrorMsg1.insert(mErrorMsg1.length()-1,FileName1); mErrorMsg2.insert(mErrorMsg2.length()-1,FileName1); mErrorMsg3.insert(mErrorMsg3.length()-1,FileName1); mErrorMsg4.insert(mErrorMsg4.length()-1,FileName1); // check for file open errors if ( !File ) throw ofstream::failure(mErrorMsg1); // write UTF-8 BOM File << wchar_t(0xEF) << wchar_t(0xBB) << wchar_t(0xBF); // switch locale to UTF-8 File.imbue(utf8locale); // check for write errors if ( File.bad() ) throw ofstream::failure(mErrorMsg2); buf.reset( new DblBuf(File.rdbuf(),ScrLog->rdbuf()) ); Log.reset( new logstream(&*buf) ); } // ---------------------------------------------------------------------------- // MyLog destructor // ---------------------------------------------------------------------------- MyLog::~MyLog() { *Log << TimeStamp1 << "Log finished." << endl; // clean up objects Log.reset(); buf.reset(); File.close(); // check for file close errors if ( File.bad() ) throw ofstream::failure(mErrorMsg4); } //--------------------------------------------------------------------------- #endif // INCLUDED_MYLOG_H Tested on MSVC 2008, boost 1.42. I do not know if this is the right place to share it. Hope it helps anybody. Feel free to make it better.

Read the article

how to pass an id number string to this class

- by Phil

I'm very much a vb person, but have had to use this id number class in c#. I got it from http://www.codingsanity.com/idnumber.htm : using System; using System.Text.RegularExpressions; namespace Utilities.SouthAfrica { /// <summary> /// Represents a South African Identity Number. /// valid number = 7707215230080 /// invalid test number = 1234567891234 /// /// </summary> [Serializable()] public class IdentityNumber { #region Enumerations /// <summary> /// Indicates a gender. /// </summary> public enum PersonGender { Female = 0, Male = 5 } public enum PersonCitizenship { SouthAfrican = 0, Foreign = 1 } #endregion #region Declarations static Regex _expression; Match _match; const string _IDExpression = @"(?<Year>[0-9][0-9])(?<Month>([0][1-9])|([1][0-2]))(?<Day>([0-2][0-9])|([3][0-1]))(?<Gender>[0-9])(?<Series>[0-9]{3})(?<Citizenship>[0-9])(?<Uniform>[0-9])(?<Control>[0-9])"; #endregion #region Constuctors /// <summary> /// Sets up the shared objects for ID validation. /// </summary> static IdentityNumber() { _expression = new Regex(_IDExpression, RegexOptions.Compiled | RegexOptions.Singleline); } /// <summary> /// Creates the ID number from a string. /// </summary> /// <param name="IDNumber">The string ID number.</param> public IdentityNumber(string IDNumber) { _match = _expression.Match(IDNumber.Trim()); } #endregion #region Properties /// <summary> /// Indicates the date of birth encoded in the ID Number. /// </summary> /// <exception cref="System.ArgumentException">Thrown if the ID Number is not usable.</exception> public DateTime DateOfBirth { get { if(IsUsable == false) { throw new ArgumentException("ID Number is unusable!", "IDNumber"); } int year = int.Parse(_match.Groups["Year"].Value); // NOTE: Do not optimize by moving these to static, otherwise the calculation may be incorrect // over year changes, especially century changes. int currentCentury = int.Parse(DateTime.Now.Year.ToString().Substring(0, 2) + "00"); int lastCentury = currentCentury - 100; int currentYear = int.Parse(DateTime.Now.Year.ToString().Substring(2, 2)); // If the year is after or at the current YY, then add last century to it, otherwise add // this century. // TODO: YY -> YYYY logic needs thinking about if(year > currentYear) { year += lastCentury; } else { year += currentCentury; } return new DateTime(year, int.Parse(_match.Groups["Month"].Value), int.Parse(_match.Groups["Day"].Value)); } } /// <summary> /// Indicates the gender for the ID number. /// </summary> /// <exception cref="System.ArgumentException">Thrown if the ID Number is not usable.</exception> public PersonGender Gender { get { if(IsUsable == false) { throw new ArgumentException("ID Number is unusable!", "IDNumber"); } int gender = int.Parse(_match.Groups["Gender"].Value); if(gender < (int) PersonGender.Male) { return PersonGender.Female; } else { return PersonGender.Male; } } } /// <summary> /// Indicates the citizenship for the ID number. /// </summary> /// <exception cref="System.ArgumentException">Thrown if the ID Number is not usable.</exception> public PersonCitizenship Citizenship { get { if(IsUsable == false) { throw new ArgumentException("ID Number is unusable!", "IDNumber"); } return (PersonCitizenship) Enum.Parse(typeof(PersonCitizenship), _match.Groups["Citizenship"].Value); } } /// <summary> /// Indicates if the IDNumber is usable or not. /// </summary> public bool IsUsable { get { return _match.Success; } } /// <summary> /// Indicates if the IDNumber is valid or not. /// </summary> public bool IsValid { get { if(IsUsable == true) { // Calculate total A by adding the figures in the odd positions i.e. the first, third, fifth, // seventh, ninth and eleventh digits. int a = int.Parse(_match.Value.Substring(0, 1)) + int.Parse(_match.Value.Substring(2, 1)) + int.Parse(_match.Value.Substring(4, 1)) + int.Parse(_match.Value.Substring(6, 1)) + int.Parse(_match.Value.Substring(8, 1)) + int.Parse(_match.Value.Substring(10, 1)); // Calculate total B by taking the even figures of the number as a whole number, and then // multiplying that number by 2, and then add the individual figures together. int b = int.Parse(_match.Value.Substring(1, 1) + _match.Value.Substring(3, 1) + _match.Value.Substring(5, 1) + _match.Value.Substring(7, 1) + _match.Value.Substring(9, 1) + _match.Value.Substring(11, 1)); b *= 2; string bString = b.ToString(); b = 0; for(int index = 0; index < bString.Length; index++) { b += int.Parse(bString.Substring(index, 1)); } // Calculate total C by adding total A to total B. int c = a + b; // The control-figure can now be determined by subtracting the ones in figure C from 10. string cString = c.ToString() ; cString = cString.Substring(cString.Length - 1, 1) ; int control = 0; // Where the total C is a multiple of 10, the control figure will be 0. if(cString != "0") { control = 10 - int.Parse(cString.Substring(cString.Length - 1, 1)); } if(_match.Groups["Control"].Value == control.ToString()) { return true; } } return false; } } #endregion } } Here is the code from my default.aspx.cs page: using System; using System.Collections.Generic; using System.Linq; using System.Web; using System.Web.UI; using System.Web.UI.WebControls; using Utilities.Southafrica; <- this is the one i added to public partial class _Default : System.Web.UI.Page { protected void Page_Load(object sender, EventArgs e) { var someNumber = new IdentityNumber("123456"); <- gives error } } Can someone please tell the syntax for how I pass an id number to the class? Thanks

Read the article

Form File Upload with other TextBox Inputs + Creating Custom Form Action attribute

- by Jonathan Stowell

Hi All, I am attempting to create a form where a user is able to enter your typical form values textboxes etc, but also upload a file as part of the form submission. This is my View code it can be seen that the File upload is identified by the MCF id: <% using (Html.BeginForm("Create", "Problem", FormMethod.Post, new { id = "ProblemForm", enctype = "multipart/form-data" })) {%> <p> <label for="StudentEmail">Student Email (*)</label> <br /> <%= Html.TextBox("StudentEmail", Model.Problem.StudentEmail, new { size = "30", maxlength=26 })%> <%= Html.ValidationMessage("StudentEmail", "*") %> </p> <p> <label for="Type">Communication Type (*)</label> <br /> <%= Html.DropDownList("Type") %> <%= Html.ValidationMessage("Type", "*") %> </p> <p> <label for="ProblemDateTime">Problem Date (*)</label> <br /> <%= Html.TextBox("ProblemDateTime", String.Format("{0:d}", Model.Problem.ProblemDateTime), new { maxlength = 10 })%> <%= Html.ValidationMessage("ProblemDateTime", "*") %> </p> <p> <label for="ProblemCategory">Problem Category (* OR Problem Outline)</label> <br /> <%= Html.DropDownList("ProblemCategory", null, "Please Select...")%> <%= Html.ValidationMessage("ProblemCategory", "*")%> </p> <p> <label for="ProblemOutline">Problem Outline (* OR Problem Category)</label> <br /> <%= Html.TextArea("ProblemOutline", Model.Problem.ProblemOutline, 6, 75, new { maxlength = 255 })%> <%= Html.ValidationMessage("ProblemOutline", "*") %> </p> <p> <label for="MCF">Mitigating Circumstance Form</label> <br /> <input id="MCF" type="file" /> <%= Html.ValidationMessage("MCF", "*") %> </p> <p> <label for="MCL">Mitigating Circumstance Level</label> <br /> <%= Html.DropDownList("MCL") %> <%= Html.ValidationMessage("MCL", "*") %> </p> <p> <label for="AbsentFrom">Date Absent From</label> <br /> <%= Html.TextBox("AbsentFrom", String.Format("{0:d}", Model.Problem.AbsentFrom), new { maxlength = 10 })%> <%= Html.ValidationMessage("AbsentFrom", "*") %> </p> <p> <label for="AbsentUntil">Date Absent Until</label> <br /> <%= Html.TextBox("AbsentUntil", String.Format("{0:d}", Model.Problem.AbsentUntil), new { maxlength = 10 })%> <%= Html.ValidationMessage("AbsentUntil", "*") %> </p> <p> <label for="AssessmentID">Assessment Extension</label> <br /> <%= Html.DropDownList("AssessmentID") %> <%= Html.ValidationMessage("AssessmentID", "*") %> <%= Html.TextBox("DateUntil", String.Format("{0:d}", Model.AssessmentExtension.DateUntil), new { maxlength = 16 })%> <%= Html.ValidationMessage("DateUntil", "*") %> </p> <p> <label for="Details">Assessment Extension Details</label> <br /> <%= Html.TextArea("Details", Model.AssessmentExtension.Details, 6, 75, new { maxlength = 255 })%> <%= Html.ValidationMessage("Details", "*") %> </p> <p> <label for="RequestedFollowUp">Requested Follow Up</label> <br /> <%= Html.TextBox("RequestedFollowUp", String.Format("{0:d}", Model.Problem.RequestedFollowUp), new { maxlength = 16 })%> <%= Html.ValidationMessage("RequestedFollowUp", "*") %> </p> <p> <label for="StaffEmail">Staff</label> <br /> <%= Html.ListBox("StaffEmail", Model.StaffEmail, new { @class = "multiselect" })%> <%= Html.ValidationMessage("StaffEmail", "*")%> </p> <p> <input class="button" type="submit" value="Create Problem" /> </p> This is my controller code: [AcceptVerbs(HttpVerbs.Post)] public ActionResult Create(Problem problem, AssessmentExtension assessmentExtension, Staff staffMember, HttpPostedFileBase file, string[] StaffEmail) { if (ModelState.IsValid) { try { Student student = studentRepository.GetStudent(problem.StudentEmail); Staff currentUserStaffMember = staffRepository.GetStaffWindowsLogon(User.Identity.Name); var fileName = Path.Combine(Request.MapPath("~/App_Data"), Path.GetFileName(file.FileName)); file.SaveAs(@"C:\Temp\" + fileName); if (problem.RequestedFollowUp.HasValue) { String meetingName = student.FirstName + " " + student.LastName + " " + "Mitigating Circumstance Meeting"; OutlookAppointment outlookAppointment = new OutlookAppointment(currentUserStaffMember.Email, meetingName, (DateTime)problem.RequestedFollowUp, (DateTime)problem.RequestedFollowUp.Value.AddMinutes(30)); } problemRepository.Add(problem); problemRepository.Save(); if (assessmentExtension.DateUntil != null) { assessmentExtension.ProblemID = problem.ProblemID; assessmentExtensionRepository.Add(assessmentExtension); assessmentExtensionRepository.Save(); } ProblemPrivacy problemPrivacy = new ProblemPrivacy(); problemPrivacy.ProblemID = problem.ProblemID; problemPrivacy.StaffEmail = currentUserStaffMember.Email; problemPrivacyRepository.Add(problemPrivacy); if (StaffEmail != null) { for (int i = 0; i < StaffEmail.Length; i++) { ProblemPrivacy probPrivacy = new ProblemPrivacy(); probPrivacy.ProblemID = problem.ProblemID; probPrivacy.StaffEmail = StaffEmail[i]; problemPrivacyRepository.Add(probPrivacy); } } problemPrivacyRepository.Save(); return RedirectToAction("Details", "Student", new { id = student.Email }); } catch { ModelState.AddRuleViolations(problem.GetRuleViolations()); } } return View(new ProblemFormViewModel(problem, assessmentExtension, staffMember)); } This form was working correctly before I had to switch to using a non-AJAX file upload, this was due to an issue with Flash when enabling Windows Authentication which I need to use. It appears that when I submit the form the file is not sent and I am unsure as to why? I have also been unsuccessful in finding an example online where a file upload is used in conjunction with other input types. Another query I have is that for Create, and Edit operations I have used a PartialView for my forms to make my application have higher code reuse. The form action is normally generated by just using: Html.BeginForm() And this populates the action depending on which Url is being used Edit or Create. However when populating HTML attributes you have to provide a action and controller value to pass HTML attributes. using (Html.BeginForm("Create", "Problem", FormMethod.Post, new { id = "ProblemForm", enctype = "multipart/form-data" })) Is it possible to somehow populate the action and controller value depending on the URL to maintain code reuse? Thinking about it whilst typing this I could set two values in the original controller action request view data and then just populate the value using the viewdata values? Any help on these two issues would be appreciated, I'm new to asp.net mvc :-) Thanks, Jon

Read the article

how to pass an id number string to this class (asp.net, c#)

- by Phil

I'm very much a vb person, but have had to use this id number class in c#. I got it from http://www.codingsanity.com/idnumber.htm : using System; using System.Text.RegularExpressions; namespace Utilities.SouthAfrica { /// <summary> /// Represents a South African Identity Number. /// valid number = 7707215230080 /// invalid test number = 1234567891234 /// /// </summary> [Serializable()] public class IdentityNumber { #region Enumerations /// <summary> /// Indicates a gender. /// </summary> public enum PersonGender { Female = 0, Male = 5 } public enum PersonCitizenship { SouthAfrican = 0, Foreign = 1 } #endregion #region Declarations static Regex _expression; Match _match; const string _IDExpression = @"(?<Year>[0-9][0-9])(?<Month>([0][1-9])|([1][0-2]))(?<Day>([0-2][0-9])|([3][0-1]))(?<Gender>[0-9])(?<Series>[0-9]{3})(?<Citizenship>[0-9])(?<Uniform>[0-9])(?<Control>[0-9])"; #endregion #region Constuctors /// <summary> /// Sets up the shared objects for ID validation. /// </summary> static IdentityNumber() { _expression = new Regex(_IDExpression, RegexOptions.Compiled | RegexOptions.Singleline); } /// <summary> /// Creates the ID number from a string. /// </summary> /// <param name="IDNumber">The string ID number.</param> public IdentityNumber(string IDNumber) { _match = _expression.Match(IDNumber.Trim()); } #endregion #region Properties /// <summary> /// Indicates the date of birth encoded in the ID Number. /// </summary> /// <exception cref="System.ArgumentException">Thrown if the ID Number is not usable.</exception> public DateTime DateOfBirth { get { if(IsUsable == false) { throw new ArgumentException("ID Number is unusable!", "IDNumber"); } int year = int.Parse(_match.Groups["Year"].Value); // NOTE: Do not optimize by moving these to static, otherwise the calculation may be incorrect // over year changes, especially century changes. int currentCentury = int.Parse(DateTime.Now.Year.ToString().Substring(0, 2) + "00"); int lastCentury = currentCentury - 100; int currentYear = int.Parse(DateTime.Now.Year.ToString().Substring(2, 2)); // If the year is after or at the current YY, then add last century to it, otherwise add // this century. // TODO: YY -> YYYY logic needs thinking about if(year > currentYear) { year += lastCentury; } else { year += currentCentury; } return new DateTime(year, int.Parse(_match.Groups["Month"].Value), int.Parse(_match.Groups["Day"].Value)); } } /// <summary> /// Indicates the gender for the ID number. /// </summary> /// <exception cref="System.ArgumentException">Thrown if the ID Number is not usable.</exception> public PersonGender Gender { get { if(IsUsable == false) { throw new ArgumentException("ID Number is unusable!", "IDNumber"); } int gender = int.Parse(_match.Groups["Gender"].Value); if(gender < (int) PersonGender.Male) { return PersonGender.Female; } else { return PersonGender.Male; } } } /// <summary> /// Indicates the citizenship for the ID number. /// </summary> /// <exception cref="System.ArgumentException">Thrown if the ID Number is not usable.</exception> public PersonCitizenship Citizenship { get { if(IsUsable == false) { throw new ArgumentException("ID Number is unusable!", "IDNumber"); } return (PersonCitizenship) Enum.Parse(typeof(PersonCitizenship), _match.Groups["Citizenship"].Value); } } /// <summary> /// Indicates if the IDNumber is usable or not. /// </summary> public bool IsUsable { get { return _match.Success; } } /// <summary> /// Indicates if the IDNumber is valid or not. /// </summary> public bool IsValid { get { if(IsUsable == true) { // Calculate total A by adding the figures in the odd positions i.e. the first, third, fifth, // seventh, ninth and eleventh digits. int a = int.Parse(_match.Value.Substring(0, 1)) + int.Parse(_match.Value.Substring(2, 1)) + int.Parse(_match.Value.Substring(4, 1)) + int.Parse(_match.Value.Substring(6, 1)) + int.Parse(_match.Value.Substring(8, 1)) + int.Parse(_match.Value.Substring(10, 1)); // Calculate total B by taking the even figures of the number as a whole number, and then // multiplying that number by 2, and then add the individual figures together. int b = int.Parse(_match.Value.Substring(1, 1) + _match.Value.Substring(3, 1) + _match.Value.Substring(5, 1) + _match.Value.Substring(7, 1) + _match.Value.Substring(9, 1) + _match.Value.Substring(11, 1)); b *= 2; string bString = b.ToString(); b = 0; for(int index = 0; index < bString.Length; index++) { b += int.Parse(bString.Substring(index, 1)); } // Calculate total C by adding total A to total B. int c = a + b; // The control-figure can now be determined by subtracting the ones in figure C from 10. string cString = c.ToString() ; cString = cString.Substring(cString.Length - 1, 1) ; int control = 0; // Where the total C is a multiple of 10, the control figure will be 0. if(cString != "0") { control = 10 - int.Parse(cString.Substring(cString.Length - 1, 1)); } if(_match.Groups["Control"].Value == control.ToString()) { return true; } } return false; } } #endregion } } Can someone please tell the syntax for how I pass an id number to the class? Thanks

Read the article

Using Flot's Bar Graph in an Android WebView with Highlighting

- by Nicholi

The issue is unhighlighting bars which are no longer selected in a bar graph plotted by flot in a WebView on Android. Got no other issues drawing the actual graphs (which look beautiful for something so simple btw). I am not extremely knowledgeable in terms of javascript and web design/development but it seems little should have been needed, if it would just work!! :( I believe I'm following the Flot API correctly, if not someone please scream and yell at me. It seems to work just fine in a non-mobile browser at least. Hoping someone has done this before, but if not I've got the minimal necessary code to poke at your droids if inquiring minds would like to test. I've tested on two Nexus Ones (both 2.2.1), and have tried targeting with Andriod 1.5 and 2.2 SDKs (my intention is to target 1.5 if possible). I've been attempting to hack away at this for far too long on my own now. What happens: 1. Graph loads fine with bars. All bars unhighlighted. 2. Select a bar in graph, gets highlighted fine (and a tooltip is placed). 3. Select a different bar in graph, old bar is unhighlighted, old tooltip removed, new bar highlighted and tooltip placed (still no problems). 4. Click in the vast darkness of the graph which should then unhighlight the last bar... but it doesn't. I've tried disabling flot's autohighlight and manually doing it as well to no avail. Looking into flot itself and only getting down to drawOverlay() where the issue seems to begin... An even more disturbing bug(?) appears if the fill bar option is enabled in the graph, but I'd rather just forget about that for now. Also grabbed the latest version of flot from their svn (r290), but made no different from last public release (v0.6). As a complete guess I'm thinking it's an issue with WebKit's javascript implementation (or something specific to Nexus Ones, which wouldn't be so bad), but if there is any ugly hack to just get it to work I'm all ears. I've thrown the graph data directly into the html/js, rather than deal with showing all the code involved in the Java-javascript handler and callbacks. The simple html placed in 'assets/flot/test/' with jquery.js and jquery.flot.js: <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> <script src="jquery.js"></script> <script src="jquery.flot.js"></script> <script id="source" language="javascript" type="text/javascript"> var lastItem = null; var plot = null; $(document).ready(function () { //window.testhandler.loadGraph(); // bind plotclick here $("#graphHolder").bind("plotclick", function (event, pos, item) { if (item) { var lastPoint = null; if (lastItem != null) lastPoint = lastItem.datapoint; if (!pointEquals(lastPoint, item.datapoint)) { //if (lastItem != null) // plot.unhighlight(lastItem.series, lastItem.datapoint); lastItem = item; $("#tooltip").remove(); //plot.highlight(item.series, item.datapoint); showTooltip(item.pageX, item.pageY, item.datapoint[1]); } } else if (lastItem != null) { plot.unhighlight(lastItem.series, lastItem.datapoint); // not unhighlighting anything //plot.unhighlight(); // doesn't work either, supposed to unhighlight everything lastItem = null; $("#tooltip").remove(); } }); GotGraph(); }); /** * Show a tooltip above bar in graph * @param {int} x Left coordinate of div * @param {int} y Top coordinate of div * @param {String} contents text to place in div */ function showTooltip(x, y, contents) { $('<div id="tooltip">' + contents + '</div>').css( { position: 'absolute', display: 'none', top: y, left: x, border: '1px solid #fdd', padding: '2px', 'background-color': '#fee', opacity: 0.80 }).appendTo("body").fadeIn(200); } /** * Draw the graph. This is a callback which will be called by Java * * @param {Object} seriesData * @param {Object} seriesOptions */ function GotGraph() { //seriesData, seriesOptions) { var seriesData = [{ "bars":{"lineWidth":2,"show":true,"barWidth":86400000,"align":"center","fill":false}, "data":[[1288569600000,10],[1288656000000,5],[1288742400000,12],[1288828800000,20],[1288915200000,14],[1289001600000,3],[1289174400000,22],[1289260800000,20],[1289347200000,10],[1289433600000,5],[1289520000000,12],[1289606400000,20],[1289692800000,14],[1289779200000,35]]}]; var seriesOptions = { "xaxis":{"twelveHourClock":false,"minTickSize":[1,"day"],"tickSize":[1,"day"],"timeformat":"%d","mode":"time"}, "yaxis":{"min":0}, "grid":{"clickable":true,"autoHighlight":true,"hoverable":false}}; plot = $.plot($("#graphHolder"), seriesData, seriesOptions); } function pointEquals(point1, point2) { if (point1 != null && point2 != null && typeof(point1) == typeof(point2) && point1.length == point2.length) { var i; for (i=0;i<point1.length;i++) { if (point1[i] != point2[i]) { return false; } } return true; } return false; } </script> </head> <body> <div id="graphHolder" STYLE="height:200px;width:400px"></div> </body> </html> The minimal amount of code necessary in onCreate in startup activity: @Override public void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); WebView mytestView = new WebView(this); mytestView.setLayoutParams(new LayoutParams(LayoutParams.FILL_PARENT, LayoutParams.FILL_PARENT)); setContentView(mytestView); mytestView.setBackgroundColor(0); mytestView.getSettings().setJavaScriptEnabled(true); mytestView.setClickable(true); mytestView.setFocusable(false); mytestView.setFocusableInTouchMode(false); mytestView.loadUrl("file:///android_asset/flot/test/stats_graph.html"); }

Read the article

Databinding, using formulas for unusual binding possible?

- by Rattenmann

Edit: added Info for WPF being used I am trying to bind a list of custom objects to a DataGrid. Straight binding seems easy enough, but i need to specify some complex formulas for some extra fields that do not directly show up in my class. Also i want to be able to EDIT the data in the Grid and get updates on related fields. Let me show you an example, because it is really hard to explain. I will simplify it to rooms with items. Each item can be red and blue. My Class looks like this: public class room { public string strRoomName { set; get; } public string strItemname { set; get; } public int intRedItem { set; get; } public int intBlueItem { set; get; } } Now if i use dataTable.ItemSource = myList; i get something like this: nr. | room | name | red | blue 1. living room, ball, 2, 1 2. sleeping room, bunny, 4, 1 3. living room, chair, 3, 2 4. kitchen, ball, 4, 7 5. garage, chair, 1, 4 Now for the complex part i need help with. I want every item to be the same number, red and blue. And because this does not hold true i want to see the "inbalance" per room AND globally like this: nr. | room | name | red | blue | missing | global red | global blue | global missing 1. living room, ball, 2, 1, 1 blue, 6, 7, 1 red 2. sleeping room, bunny, 4, 1, 3 blue, 4, 1, 3 blue 3. living room, chair, 3, 2, 1 blue, 4, 6, 2 red 4. kitchen, ball, 4, 7, 3 red, 6, 7, 1 red 5. garage, chair, 1, 4, 3 red, 4, 6, 2 red As you can see this smeels like excel formulas, i am unsure how to handle this in c# code however. You can also see i need to use data in the same row, but also get data from other rows that match one propertiy (the items name). Also if i change the blue value=1 in line 1 to value=2, i want line 1 to read like this: 1. living room, ball, 2, 2, even, 6, 8, 2 red and of corse line 4 needs to change to: 4. kitchen, ball, 4, 7, 3 red, 6, 8, 2 red As i said, this smells like excel, that's why i am really upset about myself not finding an easy solution. Surely enough c# offers some way to handle this stuff, right? Disclaimer: It is totally possible that i need a complete differend approach, pointing that out ot me is perfectly fine. Be it other ways to handle this, or a better way to structure my class. I am ok with every way to handle this as it is for learning purposes. I am simply doing programms for fun next to my college and just so happen to hit these kinda things that bug me out because i don't find a clean solution. And then i neglect my studies because i want to solve my (unreleated to studys,...) issue. Just can't stand having unsolved coding stuff around, don't judge me! ;-) And big thanks in advance if you have gotten this far in my post. It sure must be confusing with all those reds and blues. Edit: After reading trough your answers and testing my skills to implement your hints, i now have the following code as my class: public class RoomList : ObservableCollection<room> { public RoomList() : base() { Add(new room() { strRoomName = "living room", strItemname = "ball", intRedItem = 2, intBlueItem = 1 }); Add(new room() { strRoomName = "sleeping room", strItemname = "bunny", intRedItem = 4, intBlueItem = 1 }); Add(new room() { strRoomName = "living room", strItemname = "chair", intRedItem = 3, intBlueItem = 2 }); Add(new room() { strRoomName = "kitchen", strItemname = "ball", intRedItem = 4, intBlueItem = 7 }); Add(new room() { strRoomName = "garage", strItemname = "chair", intRedItem = 1, intBlueItem = 4 }); } } //rooms public class room : INotifyPropertyChanged { public string strRoomName { set; get; } public string strItemname { set; get; } public int intRedItem { get { return intRedItem; } set { intRedItem = value; NotifyPropertyChanged("intRedItem", "strMissing"); } } public int intBlueItem { get { return intBlueItem; } set { intBlueItem = value; NotifyPropertyChanged("intBlueItem", "strMissing"); } } public string strMissing { get { int missingCount = intRedItem - intBlueItem; return missingCount == 0 ? "Even" : missingCount.ToString(); } } public event PropertyChangedEventHandler PropertyChanged; public void NotifyPropertyChanged(params string[] propertyNames) { if (PropertyChanged != null) { foreach (string propertyName in propertyNames) { PropertyChanged(this, new PropertyChangedEventArgs(propertyName)); } } } } I got the "missing" field working right away, thanks alot for that tip. It really was as easy as i imagined and will be of great use for future projects. Still two (three maybe....) things i am missing tho. The above code terminates with a "System.StackOverflowException" in the setter of intRedItem and intBlueItem. I fail to see the error, that could be due to being 4:30am here, or my lack of understanding. Second issue: I followed the link to ObservableCollections as you can see from my code above. Yet i am unsure how to actually use that collection. Putting it as DataContent like suggested on that page shows a missing ressource. Adding it as a ressource like listed there crashes my VSExpress designer and leads to the programm not starting. So for now i am still using my old approach of a list like this: listRooms.Add(new room() { strRoomName = "living room", strItemname = "ball", intRedItem = 2, intBlueItem = 1 }); listRooms.Add(new room() { strRoomName = "sleeping room", strItemname = "bunny", intRedItem = 4, intBlueItem = 1 }); listRooms.Add(new room() { strRoomName = "living room", strItemname = "chair", intRedItem = 3, intBlueItem = 2 }); listRooms.Add(new room() { strRoomName = "kitchen", strItemname = "ball", intRedItem = 4, intBlueItem = 7 }); listRooms.Add(new room() { strRoomName = "garage", strItemname = "chair", intRedItem = 1, intBlueItem = 4 }); datagridRooms.ItemsSource = listRooms; And lastly: When testing before adding the notifyevents i tried to implement a proterty that looped trough the other objects, without any luck. The "missingItem" property worked so easy, yet it only tries to access "it's own" properties kind of. I need to access other objects, like "all objects that have the same room value". The idea behind this is that i am trying to calculate a value from other objects without even having those objects yet, at least in my logic. Where is the flaw in my thinking? Those 5 objects are added and created (?) one after another. So if the first tries to set it's "all red balls in my room AND all other rooms" value,.. how could it know about the balls in the kitchen, that get added as 4th object? So far so good tho, got on the right track i think. Just need some sleep first.

Read the article

Using java to create a dynamic rich:panelMenu

- by Mark Lewis

Hail This attempt at using component libs to access a Map<String, ArrayList<String>> <rich:panelMenu style="width:35%" mode="ajax" iconExpandedTopGroup="/img/logListIcon.png" iconCollapsedTopGroup="/img/logListIcon.png" iconCollapsedGroup="/img/logListFolderIconClosed.png" iconExpandedGroup="/img/logListFolderIconOpen.png" iconGroupTopPosition="left" iconItem="/img/logFileIcon.png">  <ui:repeat items="#{PanelMenu.panelNodes}" var="map"> <rich:panelMenuGroup label="#{map.key}" style="padding-left: 20px;"> <ui:repeat items="${map.value}" var="entry"> <rich:panelMenuItem label="#{entry}" action="#{PanelMenu.updateCurrent}"> <f:param name="current" value="#{entry}" /> </rich:panelMenuItem> </ui:repeat> </rich:panelMenuGroup> </ui:repeat> </rich:panelMenu> <rich:panel style="width:65%" bodyClass="rich-laguna-panel-no-header"> <a4j:outputPanel ajaxRendered="true"> <h:outputText value="#{PanelMenu.currentLog}" id="current" /> </a4j:outputPanel> </rich:panel> resulted in |STDOUT| 2010-03-06 22:46:22,212 | ERROR | [http-8181-2]: Exception in the filter chain javax.servlet.ServletException: com.sun.facelets.component.UIRepeat cannot be cast to org.richfaces.component.UIPanelMenuItem so I think I need to get onto a resource to use FacesContext context = FacesContext.getCurrentInstance(); but as I'm fairly new to both java and jsf I need a bit of help. Can anyone suggest either the (probably small) java method to represent this intention or a really good resource specifically about programming dynamic stuff in RichFaces using java? Naturally I've read the javadocs, but they don't give me a concrete example of some JSF, the associated backing bean java code, and a screen shot of the resulting component. It may be asking a bit much, but I'm sure I'm not alone in thinking that with these three, I'd be able to understand the idea and apply it to many other component types in this family. The RF demo doesn't include dynamic panelMenus or drop downs so I'm a bit stumped. Thanks IA EDIT @BalusC: Thanks for the response, here is complete stack trace: 07-Mar-2010 19:05:52 com.sun.facelets.FaceletViewHandler handleRenderException SEVERE: Error Rendering View[/index.xhtml] java.lang.ClassCastException: com.sun.facelets.component.UIRepeat cannot be cast to org.richfaces.component.UIPanelMenuItem at org.richfaces.renderkit.html.PanelMenuRenderer.insertScript(PanelMenuRenderer.java:141) at org.richfaces.renderkit.html.HtmlPanelMenuRenderer.doEncodeEnd(HtmlPanelMenuRenderer.java:260) at org.richfaces.renderkit.html.HtmlPanelMenuRenderer.doEncodeEnd(HtmlPanelMenuRenderer.java:271) at org.ajax4jsf.renderkit.RendererBase.encodeEnd(RendererBase.java:134) at javax.faces.component.UIComponentBase.encodeEnd(UIComponentBase.java:861) at org.ajax4jsf.renderkit.RendererBase.renderChild(RendererBase.java:281) at org.ajax4jsf.renderkit.RendererBase.renderChildren(RendererBase.java:258) at org.richfaces.renderkit.TabRendererBase.encodeChildren(TabRendererBase.java:113) at javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.java:837) at org.ajax4jsf.renderkit.RendererBase.renderChild(RendererBase.java:277) at org.ajax4jsf.renderkit.RendererBase.renderChildren(RendererBase.java:258) at org.richfaces.renderkit.html.TabPanelRenderer.doEncodeChildren(TabPanelRenderer.java:266) at org.richfaces.renderkit.html.TabPanelRenderer.doEncodeChildren(TabPanelRenderer.java:261) at org.ajax4jsf.renderkit.RendererBase.encodeChildren(RendererBase.java:120) at javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.java:837) at com.sun.faces.renderkit.html_basic.HtmlBasicRenderer.encodeRecursive(HtmlBasicRenderer.java:234) at com.sun.faces.renderkit.html_basic.GridRenderer.renderRow(GridRenderer.java:180) at com.sun.faces.renderkit.html_basic.GridRenderer.encodeChildren(GridRenderer.java:127) at javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.java:837) at com.sun.faces.renderkit.html_basic.HtmlBasicRenderer.encodeRecursive(HtmlBasicRenderer.java:234) at com.sun.faces.renderkit.html_basic.HtmlBasicRenderer.encodeRecursive(HtmlBasicRenderer.java:239) at com.sun.faces.renderkit.html_basic.HtmlBasicRenderer.encodeRecursive(HtmlBasicRenderer.java:239) at com.sun.faces.renderkit.html_basic.GroupRenderer.encodeChildren(GroupRenderer.java:106) at javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.java:837) at org.ajax4jsf.renderkit.RendererBase.renderChild(RendererBase.java:277) at org.ajax4jsf.renderkit.RendererBase.renderChildren(RendererBase.java:258) at org.richfaces.renderkit.html.PanelRenderer.doEncodeChildren(PanelRenderer.java:200) at org.richfaces.renderkit.html.PanelRenderer.doEncodeChildren(PanelRenderer.java:195) at org.ajax4jsf.renderkit.RendererBase.encodeChildren(RendererBase.java:120) at javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.java:837) at org.ajax4jsf.renderkit.RendererBase.renderChild(RendererBase.java:277) at org.ajax4jsf.renderkit.RendererBase.renderChildren(RendererBase.java:258) at org.richfaces.renderkit.html.PageRenderer.doEncodeChildren(PageRenderer.java:265) at org.richfaces.renderkit.html.PageRenderer.doEncodeChildren(PageRenderer.java:254) at org.ajax4jsf.renderkit.RendererBase.encodeChildren(RendererBase.java:120) at javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.java:837) at javax.faces.component.UIComponent.encodeAll(UIComponent.java:930) at javax.faces.component.UIComponent.encodeAll(UIComponent.java:933) at com.sun.facelets.FaceletViewHandler.renderView(FaceletViewHandler.java:594) at org.ajax4jsf.application.ViewHandlerWrapper.renderView(ViewHandlerWrapper.java:100) at org.ajax4jsf.application.AjaxViewHandler.renderView(AjaxViewHandler.java:176) at com.sun.faces.lifecycle.RenderResponsePhase.execute(RenderResponsePhase.java:110) at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:100) at com.sun.faces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:139) at javax.faces.webapp.FacesServlet.service(FacesServlet.java:266) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.ajax4jsf.webapp.BaseXMLFilter.doXmlFilter(BaseXMLFilter.java:178) at org.ajax4jsf.webapp.BaseFilter.handleRequest(BaseFilter.java:290) at org.ajax4jsf.webapp.BaseFilter.processUploadsAndHandleRequest(BaseFilter.java:388) at org.ajax4jsf.webapp.BaseFilter.doFilter(BaseFilter.java:515) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:619) 07-Mar-2010 19:05:52 com.sun.faces.lifecycle.Phase doPhase SEVERE: JSF1054: (Phase ID: RENDER_RESPONSE 6, View ID: /index.xhtml) Exception thrown during phase execution: javax.faces.event.PhaseEvent[source=com.sun.faces.lifecycle.LifecycleImpl@30f369] 07-Mar-2010 19:05:52 org.apache.catalina.core.StandardWrapperValve invoke SEVERE: Servlet.service() for servlet Faces Servlet threw exception java.lang.ClassCastException: com.sun.facelets.component.UIRepeat cannot be cast to org.richfaces.component.UIPanelMenuItem at org.richfaces.renderkit.html.PanelMenuRenderer.insertScript(PanelMenuRenderer.java:141) at org.richfaces.renderkit.html.HtmlPanelMenuRenderer.doEncodeEnd(HtmlPanelMenuRenderer.java:260) at org.richfaces.renderkit.html.HtmlPanelMenuRenderer.doEncodeEnd(HtmlPanelMenuRenderer.java:271) at org.ajax4jsf.renderkit.RendererBase.encodeEnd(RendererBase.java:134) at javax.faces.component.UIComponentBase.encodeEnd(UIComponentBase.java:861) at org.ajax4jsf.renderkit.RendererBase.renderChild(RendererBase.java:281) at org.ajax4jsf.renderkit.RendererBase.renderChildren(RendererBase.java:258) at org.richfaces.renderkit.TabRendererBase.encodeChildren(TabRendererBase.java:113) at javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.java:837) at org.ajax4jsf.renderkit.RendererBase.renderChild(RendererBase.java:277) at org.ajax4jsf.renderkit.RendererBase.renderChildren(RendererBase.java:258) at org.richfaces.renderkit.html.TabPanelRenderer.doEncodeChildren(TabPanelRenderer.java:266) at org.richfaces.renderkit.html.TabPanelRenderer.doEncodeChildren(TabPanelRenderer.java:261) at org.ajax4jsf.renderkit.RendererBase.encodeChildren(RendererBase.java:120) at javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.java:837) at com.sun.faces.renderkit.html_basic.HtmlBasicRenderer.encodeRecursive(HtmlBasicRenderer.java:234) at com.sun.faces.renderkit.html_basic.GridRenderer.renderRow(GridRenderer.java:180) at com.sun.faces.renderkit.html_basic.GridRenderer.encodeChildren(GridRenderer.java:127) at javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.java:837) at com.sun.faces.renderkit.html_basic.HtmlBasicRenderer.encodeRecursive(HtmlBasicRenderer.java:234) at com.sun.faces.renderkit.html_basic.HtmlBasicRenderer.encodeRecursive(HtmlBasicRenderer.java:239) at com.sun.faces.renderkit.html_basic.HtmlBasicRenderer.encodeRecursive(HtmlBasicRenderer.java:239) at com.sun.faces.renderkit.html_basic.GroupRenderer.encodeChildren(GroupRenderer.java:106) at javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.java:837) at org.ajax4jsf.renderkit.RendererBase.renderChild(RendererBase.java:277) at org.ajax4jsf.renderkit.RendererBase.renderChildren(RendererBase.java:258) at org.richfaces.renderkit.html.PanelRenderer.doEncodeChildren(PanelRenderer.java:200) at org.richfaces.renderkit.html.PanelRenderer.doEncodeChildren(PanelRenderer.java:195) at org.ajax4jsf.renderkit.RendererBase.encodeChildren(RendererBase.java:120) at javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.java:837) at org.ajax4jsf.renderkit.RendererBase.renderChild(RendererBase.java:277) at org.ajax4jsf.renderkit.RendererBase.renderChildren(RendererBase.java:258) at org.richfaces.renderkit.html.PageRenderer.doEncodeChildren(PageRenderer.java:265) at org.richfaces.renderkit.html.PageRenderer.doEncodeChildren(PageRenderer.java:254) at org.ajax4jsf.renderkit.RendererBase.encodeChildren(RendererBase.java:120) at javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.java:837) at javax.faces.component.UIComponent.encodeAll(UIComponent.java:930) at javax.faces.component.UIComponent.encodeAll(UIComponent.java:933) at com.sun.facelets.FaceletViewHandler.renderView(FaceletViewHandler.java:594) at org.ajax4jsf.application.ViewHandlerWrapper.renderView(ViewHandlerWrapper.java:100) at org.ajax4jsf.application.AjaxViewHandler.renderView(AjaxViewHandler.java:176) at com.sun.faces.lifecycle.RenderResponsePhase.execute(RenderResponsePhase.java:110) at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:100) at com.sun.faces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:139) at javax.faces.webapp.FacesServlet.service(FacesServlet.java:266) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.ajax4jsf.webapp.BaseXMLFilter.doXmlFilter(BaseXMLFilter.java:178) at org.ajax4jsf.webapp.BaseFilter.handleRequest(BaseFilter.java:290) at org.ajax4jsf.webapp.BaseFilter.processUploadsAndHandleRequest(BaseFilter.java:388) at org.ajax4jsf.webapp.BaseFilter.doFilter(BaseFilter.java:515) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:619) |STDOUT| 2010-03-07 19:05:52,683 | ERROR | [http-8181-1]: Exception in the filter chain javax.servlet.ServletException: com.sun.facelets.component.UIRepeat cannot be cast to org.richfaces.component.UIPanelMenuItem at javax.faces.webapp.FacesServlet.service(FacesServlet.java:277) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.ajax4jsf.webapp.BaseXMLFilter.doXmlFilter(BaseXMLFilter.java:178) at org.ajax4jsf.webapp.BaseFilter.handleRequest(BaseFilter.java:290) at org.ajax4jsf.webapp.BaseFilter.processUploadsAndHandleRequest(BaseFilter.java:388) at org.ajax4jsf.webapp.BaseFilter.doFilter(BaseFilter.java:515) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:619) Caused by: java.lang.ClassCastException: com.sun.facelets.component.UIRepeat cannot be cast to org.richfaces.component.UIPanelMenuItem at org.richfaces.renderkit.html.PanelMenuRenderer.insertScript(PanelMenuRenderer.java:141) at org.richfaces.renderkit.html.HtmlPanelMenuRenderer.doEncodeEnd(HtmlPanelMenuRenderer.java:260) at org.richfaces.renderkit.html.HtmlPanelMenuRenderer.doEncodeEnd(HtmlPanelMenuRenderer.java:271) at org.ajax4jsf.renderkit.RendererBase.encodeEnd(RendererBase.java:134) at javax.faces.component.UIComponentBase.encodeEnd(UIComponentBase.java:861) at org.ajax4jsf.renderkit.RendererBase.renderChild(RendererBase.java:281) at org.ajax4jsf.renderkit.RendererBase.renderChildren(RendererBase.java:258) at org.richfaces.renderkit.TabRendererBase.encodeChildren(TabRendererBase.java:113) at javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.java:837) at org.ajax4jsf.renderkit.RendererBase.renderChild(RendererBase.java:277) at org.ajax4jsf.renderkit.RendererBase.renderChildren(RendererBase.java:258) at org.richfaces.renderkit.html.TabPanelRenderer.doEncodeChildren(TabPanelRenderer.java:266) at org.richfaces.renderkit.html.TabPanelRenderer.doEncodeChildren(TabPanelRenderer.java:261) at org.ajax4jsf.renderkit.RendererBase.encodeChildren(RendererBase.java:120) at javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.java:837) at com.sun.faces.renderkit.html_basic.HtmlBasicRenderer.encodeRecursive(HtmlBasicRenderer.java:234) at com.sun.faces.renderkit.html_basic.GridRenderer.renderRow(GridRenderer.java:180) at com.sun.faces.renderkit.html_basic.GridRenderer.encodeChildren(GridRenderer.java:127) at javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.java:837) at com.sun.faces.renderkit.html_basic.HtmlBasicRenderer.encodeRecursive(HtmlBasicRenderer.java:234) at com.sun.faces.renderkit.html_basic.HtmlBasicRenderer.encodeRecursive(HtmlBasicRenderer.java:239) at com.sun.faces.renderkit.html_basic.HtmlBasicRenderer.encodeRecursive(HtmlBasicRenderer.java:239) at com.sun.faces.renderkit.html_basic.GroupRenderer.encodeChildren(GroupRenderer.java:106) at javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.java:837) at org.ajax4jsf.renderkit.RendererBase.renderChild(RendererBase.java:277) at org.ajax4jsf.renderkit.RendererBase.renderChildren(RendererBase.java:258) at org.richfaces.renderkit.html.PanelRenderer.doEncodeChildren(PanelRenderer.java:200) at org.richfaces.renderkit.html.PanelRenderer.doEncodeChildren(PanelRenderer.java:195) at org.ajax4jsf.renderkit.RendererBase.encodeChildren(RendererBase.java:120) at javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.java:837) at org.ajax4jsf.renderkit.RendererBase.renderChild(RendererBase.java:277) at org.ajax4jsf.renderkit.RendererBase.renderChildren(RendererBase.java:258) at org.richfaces.renderkit.html.PageRenderer.doEncodeChildren(PageRenderer.java:265) at org.richfaces.renderkit.html.PageRenderer.doEncodeChildren(PageRenderer.java:254) at org.ajax4jsf.renderkit.RendererBase.encodeChildren(RendererBase.java:120) at javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.java:837) at javax.faces.component.UIComponent.encodeAll(UIComponent.java:930) at javax.faces.component.UIComponent.encodeAll(UIComponent.java:933) at com.sun.facelets.FaceletViewHandler.renderView(FaceletViewHandler.java:594) at org.ajax4jsf.application.ViewHandlerWrapper.renderView(ViewHandlerWrapper.java:100) at org.ajax4jsf.application.AjaxViewHandler.renderView(AjaxViewHandler.java:176) at com.sun.faces.lifecycle.RenderResponsePhase.execute(RenderResponsePhase.java:110) at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:100) at com.sun.faces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:139) at javax.faces.webapp.FacesServlet.service(FacesServlet.java:266) ... 18 more

Read the article

Netbeans / persistence API error

- by Danny

I am following this tutorial, I'm using netbeans 6.5.1 http://www.netbeans.org/kb/docs/java/gui-db-custom.html When I get to the part where I create the "new entity class from database", which is in the "Customizing the Master/Detail View" section of the tutoiral. I can't ever compile because I get this error in the task list (and I get a runtime error when I run)... Atleast I think these two are related. Error Named queries can be defined only on an Entity or MappedSuperclass class. Countries.java 27 C:/Users/Danny/dev/NetBeansProjects/Test/src/test Error Named queries can be defined only on an Entity or MappedSuperclass class. Products.java 28 C:/Users/Danny/dev/NetBeansProjects/Test/src/test note that all I do is do newentity classes from database, and do exactly as the tutorial says. I stopped on the tutoral at the "Adding Dialog Boxes" heading, because the tutorial writer says the application should be "partially functional" which makes me think it should atleast RUN. I haven't edited the generated code outside of what I've been instructed to do. This is the output produced by the netbeans output console: run: Jun 23, 2009 3:03:23 PM org.jdesktop.application.Application$1 run SEVERE: Application class test.TestApp failed to launch javax.persistence.PersistenceException: No Persistence provider for EntityManager named MyBusinessRecordsPU: Provider named oracle.toplink.essentials.PersistenceProvider threw unexpected exception at create EntityManagerFactory: oracle.toplink.essentials.exceptions.PersistenceUnitLoadingException Local Exception Stack: Exception [TOPLINK-30005] (Oracle TopLink Essentials - 2.0.1 (Build b09d-fcs (12/06/2007))): oracle.toplink.essentials.exceptions.PersistenceUnitLoadingException Exception Description: An exception was thrown while searching for persistence archives with ClassLoader: sun.misc.Launcher$AppClassLoader@11b86e7 Internal Exception: javax.persistence.PersistenceException: Exception [TOPLINK-28018] (Oracle TopLink Essentials - 2.0.1 (Build b09d-fcs (12/06/2007))): oracle.toplink.essentials.exceptions.EntityManagerSetupException Exception Description: predeploy for PersistenceUnit [MyBusinessRecordsPU] failed. Internal Exception: Exception [TOPLINK-7244] (Oracle TopLink Essentials - 2.0.1 (Build b09d-fcs (12/06/2007))): oracle.toplink.essentials.exceptions.ValidationException Exception Description: An incompatible mapping has been encountered between [class test.Products] and [class test.Orders]. This usually occurs when the cardinality of a mapping does not correspond with the cardinality of its backpointer. at oracle.toplink.essentials.exceptions.PersistenceUnitLoadingException.exceptionSearchingForPersistenceResources(PersistenceUnitLoadingException.java:143) at oracle.toplink.essentials.ejb.cmp3.EntityManagerFactoryProvider.createEntityManagerFactory(EntityManagerFactoryProvider.java:169) at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:110) at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:83) at test.TestView.initComponents(TestView.java:337) at test.TestView.(TestView.java:39) at test.TestApp.startup(TestApp.java:19) at org.jdesktop.application.Application$1.run(Application.java:171) at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:209) at java.awt.EventQueue.dispatchEvent(EventQueue.java:597) at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:269) at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:184) at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:174) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:169) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:161) at java.awt.EventDispatchThread.run(EventDispatchThread.java:122) Caused by: javax.persistence.PersistenceException: Exception [TOPLINK-28018] (Oracle TopLink Essentials - 2.0.1 (Build b09d-fcs (12/06/2007))): oracle.toplink.essentials.exceptions.EntityManagerSetupException Exception Description: predeploy for PersistenceUnit [MyBusinessRecordsPU] failed. Internal Exception: Exception [TOPLINK-7244] (Oracle TopLink Essentials - 2.0.1 (Build b09d-fcs (12/06/2007))): oracle.toplink.essentials.exceptions.ValidationException Exception Description: An incompatible mapping has been encountered between [class test.Products] and [class test.Orders]. This usually occurs when the cardinality of a mapping does not correspond with the cardinality of its backpointer. at oracle.toplink.essentials.internal.ejb.cmp3.EntityManagerSetupImpl.predeploy(EntityManagerSetupImpl.java:643) at oracle.toplink.essentials.internal.ejb.cmp3.JavaSECMPInitializer.callPredeploy(JavaSECMPInitializer.java:171) at oracle.toplink.essentials.internal.ejb.cmp3.JavaSECMPInitializer.initPersistenceUnits(JavaSECMPInitializer.java:239) at oracle.toplink.essentials.internal.ejb.cmp3.JavaSECMPInitializer.initialize(JavaSECMPInitializer.java:255) at oracle.toplink.essentials.ejb.cmp3.EntityManagerFactoryProvider.createEntityManagerFactory(EntityManagerFactoryProvider.java:155) ... 14 more Caused by: Exception [TOPLINK-28018] (Oracle TopLink Essentials - 2.0.1 (Build b09d-fcs (12/06/2007))): oracle.toplink.essentials.exceptions.EntityManagerSetupException Exception Description: predeploy for PersistenceUnit [MyBusinessRecordsPU] failed. Internal Exception: Exception [TOPLINK-7244] (Oracle TopLink Essentials - 2.0.1 (Build b09d-fcs (12/06/2007))): oracle.toplink.essentials.exceptions.ValidationException Exception Description: An incompatible mapping has been encountered between [class test.Products] and [class test.Orders]. This usually occurs when the cardinality of a mapping does not correspond with the cardinality of its backpointer. at oracle.toplink.essentials.exceptions.EntityManagerSetupException.predeployFailed(EntityManagerSetupException.java:228) ... 19 more Caused by: Exception [TOPLINK-7244] (Oracle TopLink Essentials - 2.0.1 (Build b09d-fcs (12/06/2007))): oracle.toplink.essentials.exceptions.ValidationException Exception Description: An incompatible mapping has been encountered between [class test.Products] and [class test.Orders]. This usually occurs when the cardinality of a mapping does not correspond with the cardinality of its backpointer. at oracle.toplink.essentials.exceptions.ValidationException.invalidMapping(ValidationException.java:1069) at oracle.toplink.essentials.internal.ejb.cmp3.metadata.MetadataValidator.throwInvalidMappingEncountered(MetadataValidator.java:275) at oracle.toplink.essentials.internal.ejb.cmp3.metadata.accessors.OneToManyAccessor.process(OneToManyAccessor.java:161) at oracle.toplink.essentials.internal.ejb.cmp3.metadata.accessors.RelationshipAccessor.processRelationship(RelationshipAccessor.java:290) at oracle.toplink.essentials.internal.ejb.cmp3.metadata.MetadataProject.processRelationshipDescriptors(MetadataProject.java:579) at oracle.toplink.essentials.internal.ejb.cmp3.metadata.MetadataProject.process(MetadataProject.java:512) at oracle.toplink.essentials.internal.ejb.cmp3.metadata.MetadataProcessor.processAnnotations(MetadataProcessor.java:246) at oracle.toplink.essentials.ejb.cmp3.persistence.PersistenceUnitProcessor.processORMetadata(PersistenceUnitProcessor.java:370) at oracle.toplink.essentials.internal.ejb.cmp3.EntityManagerSetupImpl.predeploy(EntityManagerSetupImpl.java:607) ... 18 more The following providers: oracle.toplink.essentials.ejb.cmp3.EntityManagerFactoryProvider Returned null to createEntityManagerFactory. at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:154) at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:83) at test.TestView.initComponents(TestView.java:337) at test.TestView.(TestView.java:39) at test.TestApp.startup(TestApp.java:19) at org.jdesktop.application.Application$1.run(Application.java:171) at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:209) at java.awt.EventQueue.dispatchEvent(EventQueue.java:597) at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:269) at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:184) at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:174) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:169) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:161) at java.awt.EventDispatchThread.run(EventDispatchThread.java:122) Exception in thread "AWT-EventQueue-0" java.lang.Error: Application class test.TestApp failed to launch at org.jdesktop.application.Application$1.run(Application.java:177) at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:209) at java.awt.EventQueue.dispatchEvent(EventQueue.java:597) at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:269) at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:184) at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:174) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:169) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:161) at java.awt.EventDispatchThread.run(EventDispatchThread.java:122) Caused by: javax.persistence.PersistenceException: No Persistence provider for EntityManager named MyBusinessRecordsPU: Provider named oracle.toplink.essentials.PersistenceProvider threw unexpected exception at create EntityManagerFactory: oracle.toplink.essentials.exceptions.PersistenceUnitLoadingException Local Exception Stack: Exception [TOPLINK-30005] (Oracle TopLink Essentials - 2.0.1 (Build b09d-fcs (12/06/2007))): oracle.toplink.essentials.exceptions.PersistenceUnitLoadingException Exception Description: An exception was thrown while searching for persistence archives with ClassLoader: sun.misc.Launcher$AppClassLoader@11b86e7 Internal Exception: javax.persistence.PersistenceException: Exception [TOPLINK-28018] (Oracle TopLink Essentials - 2.0.1 (Build b09d-fcs (12/06/2007))): oracle.toplink.essentials.exceptions.EntityManagerSetupException Exception Description: predeploy for PersistenceUnit [MyBusinessRecordsPU] failed. Internal Exception: Exception [TOPLINK-7244] (Oracle TopLink Essentials - 2.0.1 (Build b09d-fcs (12/06/2007))): oracle.toplink.essentials.exceptions.ValidationException Exception Description: An incompatible mapping has been encountered between [class test.Products] and [class test.Orders]. This usually occurs when the cardinality of a mapping does not correspond with the cardinality of its backpointer. at oracle.toplink.essentials.exceptions.PersistenceUnitLoadingException.exceptionSearchingForPersistenceResources(PersistenceUnitLoadingException.java:143) at oracle.toplink.essentials.ejb.cmp3.EntityManagerFactoryProvider.createEntityManagerFactory(EntityManagerFactoryProvider.java:169) at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:110) at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:83) at test.TestView.initComponents(TestView.java:337) at test.TestView.(TestView.java:39) at test.TestApp.startup(TestApp.java:19) at org.jdesktop.application.Application$1.run(Application.java:171) at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:209) at java.awt.EventQueue.dispatchEvent(EventQueue.java:597) at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:269) at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:184) at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:174) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:169) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:161) at java.awt.EventDispatchThread.run(EventDispatchThread.java:122) Caused by: javax.persistence.PersistenceException: Exception [TOPLINK-28018] (Oracle TopLink Essentials - 2.0.1 (Build b09d-fcs (12/06/2007))): oracle.toplink.essentials.exceptions.EntityManagerSetupException Exception Description: predeploy for PersistenceUnit [MyBusinessRecordsPU] failed. Internal Exception: Exception [TOPLINK-7244] (Oracle TopLink Essentials - 2.0.1 (Build b09d-fcs (12/06/2007))): oracle.toplink.essentials.exceptions.ValidationException Exception Description: An incompatible mapping has been encountered between [class test.Products] and [class test.Orders]. This usually occurs when the cardinality of a mapping does not correspond with the cardinality of its backpointer. at oracle.toplink.essentials.internal.ejb.cmp3.EntityManagerSetupImpl.predeploy(EntityManagerSetupImpl.java:643) at oracle.toplink.essentials.internal.ejb.cmp3.JavaSECMPInitializer.callPredeploy(JavaSECMPInitializer.java:171) at oracle.toplink.essentials.internal.ejb.cmp3.JavaSECMPInitializer.initPersistenceUnits(JavaSECMPInitializer.java:239) at oracle.toplink.essentials.internal.ejb.cmp3.JavaSECMPInitializer.initialize(JavaSECMPInitializer.java:255) at oracle.toplink.essentials.ejb.cmp3.EntityManagerFactoryProvider.createEntityManagerFactory(EntityManagerFactoryProvider.java:155) ... 14 more Caused by: Exception [TOPLINK-28018] (Oracle TopLink Essentials - 2.0.1 (Build b09d-fcs (12/06/2007))): oracle.toplink.essentials.exceptions.EntityManagerSetupException Exception Description: predeploy for PersistenceUnit [MyBusinessRecordsPU] failed. Internal Exception: Exception [TOPLINK-7244] (Oracle TopLink Essentials - 2.0.1 (Build b09d-fcs (12/06/2007))): oracle.toplink.essentials.exceptions.ValidationException Exception Description: An incompatible mapping has been encountered between [class test.Products] and [class test.Orders]. This usually occurs when the cardinality of a mapping does not correspond with the cardinality of its backpointer. at oracle.toplink.essentials.exceptions.EntityManagerSetupException.predeployFailed(EntityManagerSetupException.java:228) ... 19 more Caused by: Exception [TOPLINK-7244] (Oracle TopLink Essentials - 2.0.1 (Build b09d-fcs (12/06/2007))): oracle.toplink.essentials.exceptions.ValidationException Exception Description: An incompatible mapping has been encountered between [class test.Products] and [class test.Orders]. This usually occurs when the cardinality of a mapping does not correspond with the cardinality of its backpointer. at oracle.toplink.essentials.exceptions.ValidationException.invalidMapping(ValidationException.java:1069) at oracle.toplink.essentials.internal.ejb.cmp3.metadata.MetadataValidator.throwInvalidMappingEncountered(MetadataValidator.java:275) at oracle.toplink.essentials.internal.ejb.cmp3.metadata.accessors.OneToManyAccessor.process(OneToManyAccessor.java:161) at oracle.toplink.essentials.internal.ejb.cmp3.metadata.accessors.RelationshipAccessor.processRelationship(RelationshipAccessor.java:290) at oracle.toplink.essentials.internal.ejb.cmp3.metadata.MetadataProject.processRelationshipDescriptors(MetadataProject.java:579) at oracle.toplink.essentials.internal.ejb.cmp3.metadata.MetadataProject.process(MetadataProject.java:512) at oracle.toplink.essentials.internal.ejb.cmp3.metadata.MetadataProcessor.processAnnotations(MetadataProcessor.java:246) at oracle.toplink.essentials.ejb.cmp3.persistence.PersistenceUnitProcessor.processORMetadata(PersistenceUnitProcessor.java:370) at oracle.toplink.essentials.internal.ejb.cmp3.EntityManagerSetupImpl.predeploy(EntityManagerSetupImpl.java:607) ... 18 more The following providers: oracle.toplink.essentials.ejb.cmp3.EntityManagerFactoryProvider Returned null to createEntityManagerFactory. at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:154) at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:83) at test.TestView.initComponents(TestView.java:337) at test.TestView.(TestView.java:39) at test.TestApp.startup(TestApp.java:19) at org.jdesktop.application.Application$1.run(Application.java:171) ... 8 more BUILD SUCCESSFUL (total time: 2 seconds) I'm thinking my netbeans application must be misconfigured or something, I can't seem to find anyone with the same problem as myself

Read the article

Optimized OCR black/white pixel algorithm

- by eagle

I am writing a simple OCR solution for a finite set of characters. That is, I know the exact way all 26 letters in the alphabet will look like. I am using C# and am able to easily determine if a given pixel should be treated as black or white. I am generating a matrix of black/white pixels for every single character. So for example, the letter I (capital i), might look like the following: 01110 00100 00100 00100 01110 Note: all points, which I use later in this post, assume that the top left pixel is (0, 0), bottom right pixel is (4, 4). 1's represent black pixels, and 0's represent white pixels. I would create a corresponding matrix in C# like this: CreateLetter("I", new List<List<bool>>() { new List<bool>() { false, true, true, true, false }, new List<bool>() { false, false, true, false, false }, new List<bool>() { false, false, true, false, false }, new List<bool>() { false, false, true, false, false }, new List<bool>() { false, true, true, true, false } }); I know I could probably optimize this part by using a multi-dimensional array instead, but let's ignore that for now, this is for illustrative purposes. Every letter is exactly the same dimensions, 10px by 11px (10px by 11px is the actual dimensions of a character in my real program. I simplified this to 5px by 5px in this posting since it is much easier to "draw" the letters using 0's and 1's on a smaller image). Now when I give it a 10px by 11px part of an image to analyze with OCR, it would need to run on every single letter (26) on every single pixel (10 * 11 = 110) which would mean 2,860 (26 * 110) iterations (in the worst case) for every single character. I was thinking this could be optimized by defining the unique characteristics of every character. So, for example, let's assume that the set of characters only consists of 5 distinct letters: I, A, O, B, and L. These might look like the following: 01110 00100 00100 01100 01000 00100 01010 01010 01010 01000 00100 01110 01010 01100 01000 00100 01010 01010 01010 01000 01110 01010 00100 01100 01110 After analyzing the unique characteristics of every character, I can significantly reduce the number of tests that need to be performed to test for a character. For example, for the "I" character, I could define it's unique characteristics as having a black pixel in the coordinate (3, 0) since no other characters have that pixel as black. So instead of testing 110 pixels for a match on the "I" character, I reduced it to a 1 pixel test. This is what it might look like for all these characters: var LetterI = new OcrLetter() { Name = "I", BlackPixels = new List<Point>() { new Point (3, 0) } } var LetterA = new OcrLetter() { Name = "A", WhitePixels = new List<Point>() { new Point(2, 4) } } var LetterO = new OcrLetter() { Name = "O", BlackPixels = new List<Point>() { new Point(3, 2) }, WhitePixels = new List<Point>() { new Point(2, 2) } } var LetterB = new OcrLetter() { Name = "B", BlackPixels = new List<Point>() { new Point(3, 1) }, WhitePixels = new List<Point>() { new Point(3, 2) } } var LetterL = new OcrLetter() { Name = "L", BlackPixels = new List<Point>() { new Point(1, 1), new Point(3, 4) }, WhitePixels = new List<Point>() { new Point(2, 2) } } This is challenging to do manually for 5 characters and gets much harder the greater the amount of letters that are added. You also want to guarantee that you have the minimum set of unique characteristics of a letter since you want it to be optimized as much as possible. I want to create an algorithm that will identify the unique characteristics of all the letters and would generate similar code to that above. I would then use this optimized black/white matrix to identify characters. How do I take the 26 letters that have all their black/white pixels filled in (e.g. the CreateLetter code block) and convert them to an optimized set of unique characteristics that define a letter (e.g. the new OcrLetter() code block)? And how would I guarantee that it is the most efficient definition set of unique characteristics (e.g. instead of defining 6 points as the unique characteristics, there might be a way to do it with 1 or 2 points, as the letter "I" in my example was able to). An alternative solution I've come up with is using a hash table, which will reduce it from 2,860 iterations to 110 iterations, a 26 time reduction. This is how it might work: I would populate it with data similar to the following: Letters["01110 00100 00100 00100 01110"] = "I"; Letters["00100 01010 01110 01010 01010"] = "A"; Letters["00100 01010 01010 01010 00100"] = "O"; Letters["01100 01010 01100 01010 01100"] = "B"; Now when I reach a location in the image to process, I convert it to a string such as: "01110 00100 00100 00100 01110" and simply find it in the hash table. This solution seems very simple, however, this still requires 110 iterations to generate this string for each letter. In big O notation, the algorithm is the same since O(110N) = O(2860N) = O(N) for N letters to process on the page. However, it is still improved by a constant factor of 26, a significant improvement (e.g. instead of it taking 26 minutes, it would take 1 minute). Update: Most of the solutions provided so far have not addressed the issue of identifying the unique characteristics of a character and rather provide alternative solutions. I am still looking for this solution which, as far as I can tell, is the only way to achieve the fastest OCR processing. I just came up with a partial solution: For each pixel, in the grid, store the letters that have it as a black pixel. Using these letters: I A O B L 01110 00100 00100 01100 01000 00100 01010 01010 01010 01000 00100 01110 01010 01100 01000 00100 01010 01010 01010 01000 01110 01010 00100 01100 01110 You would have something like this: CreatePixel(new Point(0, 0), new List<Char>() { }); CreatePixel(new Point(1, 0), new List<Char>() { 'I', 'B', 'L' }); CreatePixel(new Point(2, 0), new List<Char>() { 'I', 'A', 'O', 'B' }); CreatePixel(new Point(3, 0), new List<Char>() { 'I' }); CreatePixel(new Point(4, 0), new List<Char>() { }); CreatePixel(new Point(0, 1), new List<Char>() { }); CreatePixel(new Point(1, 1), new List<Char>() { 'A', 'B', 'L' }); CreatePixel(new Point(2, 1), new List<Char>() { 'I' }); CreatePixel(new Point(3, 1), new List<Char>() { 'A', 'O', 'B' }); // ... CreatePixel(new Point(2, 2), new List<Char>() { 'I', 'A', 'B' }); CreatePixel(new Point(3, 2), new List<Char>() { 'A', 'O' }); // ... CreatePixel(new Point(2, 4), new List<Char>() { 'I', 'O', 'B', 'L' }); CreatePixel(new Point(3, 4), new List<Char>() { 'I', 'A', 'L' }); CreatePixel(new Point(4, 4), new List<Char>() { }); Now for every letter, in order to find the unique characteristics, you need to look at which buckets it belongs to, as well as the amount of other characters in the bucket. So let's take the example of "I". We go to all the buckets it belongs to (1,0; 2,0; 3,0; ...; 3,4) and see that the one with the least amount of other characters is (3,0). In fact, it only has 1 character, meaning it must be an "I" in this case, and we found our unique characteristic. You can also do the same for pixels that would be white. Notice that bucket (2,0) contains all the letters except for "L", this means that it could be used as a white pixel test. Similarly, (2,4) doesn't contain an 'A'. Buckets that either contain all the letters or none of the letters can be discarded immediately, since these pixels can't help define a unique characteristic (e.g. 1,1; 4,0; 0,1; 4,4). It gets trickier when you don't have a 1 pixel test for a letter, for example in the case of 'O' and 'B'. Let's walk through the test for 'O'... It's contained in the following buckets: // Bucket Count Letters // 2,0 4 I, A, O, B // 3,1 3 A, O, B // 3,2 2 A, O // 2,4 4 I, O, B, L Additionally, we also have a few white pixel tests that can help: (I only listed those that are missing at most 2). The Missing Count was calculated as (5 - Bucket.Count). // Bucket Missing Count Missing Letters // 1,0 2 A, O // 1,1 2 I, O // 2,2 2 O, L // 3,4 2 O, B So now we can take the shortest black pixel bucket (3,2) and see that when we test for (3,2) we know it is either an 'A' or an 'O'. So we need an easy way to tell the difference between an 'A' and an 'O'. We could either look for a black pixel bucket that contains 'O' but not 'A' (e.g. 2,4) or a white pixel bucket that contains an 'O' but not an 'A' (e.g. 1,1). Either of these could be used in combination with the (3,2) pixel to uniquely identify the letter 'O' with only 2 tests. This seems like a simple algorithm when there are 5 characters, but how would I do this when there are 26 letters and a lot more pixels overlapping? For example, let's say that after the (3,2) pixel test, it found 10 different characters that contain the pixel (and this was the least from all the buckets). Now I need to find differences from 9 other characters instead of only 1 other character. How would I achieve my goal of getting the least amount of checks as possible, and ensure that I am not running extraneous tests?

Read the article

Get to know what HTML-button is clicked with PHP, no Name of value available

- by koko

I have a wizard with 4 steps and in one of the steps, you can add items. By default, there are 3 time 5 empty input-fields listed. When you click a button, there have to be added some more. The 3 times 5 fields are added by a loop. How can I determine where I have to add some fields (in the first 5 field of the second or the third). I can't use another name for every button, because sometimes there can be more then 3 groups of 5 fields. In my code (PHP), I use the name of the button to know what happened in the form, so the name can't be different for the button in every group ... Further, the whole step is 1 form, so the post can't go to another script ... The value of the button has to be the same for every group. I've been thinking on possible solutions and these came up: On click a javascript adds a value to the post (in a hidden field?) But is this posted with the form? Could it be possible to get the ID of the clicked button? This might be different :-) I need a refresh after the click, so pure javascript isn't a solution either ... It is a very specific question, but I hope someone can give me some direction. Here is the code I'm talking about: <h3><label class="list_list:table_title">Eten</label></h3> <table id="eten"> <thead> <tr> <th>Naam item</th> <th>Aantal</th> </tr> </thead> <tbody> <tr> <td class="item"><input type="text" name="item_1_1" value="" class="inputText" maxlength="64" size="46" tabindex="11"/></td> <td class="amount"><input type="text" name="amount_1_1" value="" class="inputText" maxlength="64" size="10" tabindex="11"/></td> </tr> <tr> <td class="item"><input type="text" name="item_1_2" value="" class="inputText" maxlength="64" size="46" tabindex="12"/></td> <td class="amount"><input type="text" name="amount_1_2" value="" class="inputText" maxlength="64" size="10" tabindex="12"/></td> </tr> <tr> <td class="item"><input type="text" name="item_1_3" value="" class="inputText" maxlength="64" size="46" tabindex="13"/></td> <td class="amount"><input type="text" name="amount_1_3" value="" class="inputText" maxlength="64" size="10" tabindex="13"/></td> </tr> <tr> <td class="item"><input type="text" name="item_1_4" value="" class="inputText" maxlength="64" size="46" tabindex="14"/></td> <td class="amount"><input type="text" name="amount_1_4" value="" class="inputText" maxlength="64" size="10" tabindex="14"/></td> </tr> <tr> <td class="item"><input type="text" name="item_1_5" value="" class="inputText" maxlength="64" size="46" tabindex="15"/></td> <td class="amount"><input type="text" name="amount_1_5" value="" class="inputText" maxlength="64" size="10" tabindex="15"/></td> </tr> </tbody> </table> <input id="list_list:add" type="submit" name="list_list:add" value="voeg extra items toe" class="add" /> <div class="spacer"></div> <h3><label class="list_list:table_title">Drinken</label></h3> <table id="drinken"> <thead> <tr> <th>Naam item</th> <th>Aantal</th> </tr> </thead> <tbody> <tr> <td class="item"><input type="text" name="item_2_1" value="" class="inputText" maxlength="64" size="46" tabindex="21"/></td> <td class="amount"><input type="text" name="amount_2_1" value="" class="inputText" maxlength="64" size="10" tabindex="21"/></td> </tr> <tr> <td class="item"><input type="text" name="item_2_2" value="" class="inputText" maxlength="64" size="46" tabindex="22"/></td> <td class="amount"><input type="text" name="amount_2_2" value="" class="inputText" maxlength="64" size="10" tabindex="22"/></td> </tr> <tr> <td class="item"><input type="text" name="item_2_3" value="" class="inputText" maxlength="64" size="46" tabindex="23"/></td> <td class="amount"><input type="text" name="amount_2_3" value="" class="inputText" maxlength="64" size="10" tabindex="23"/></td> </tr> <tr> <td class="item"><input type="text" name="item_2_4" value="" class="inputText" maxlength="64" size="46" tabindex="24"/></td> <td class="amount"><input type="text" name="amount_2_4" value="" class="inputText" maxlength="64" size="10" tabindex="24"/></td> </tr> <tr> <td class="item"><input type="text" name="item_2_5" value="" class="inputText" maxlength="64" size="46" tabindex="25"/></td> <td class="amount"><input type="text" name="amount_2_5" value="" class="inputText" maxlength="64" size="10" tabindex="25"/></td> </tr> </tbody> </table> <input id="list_list:add" type="submit" name="list_list:add" value="voeg extra items toe" class="add" /> <div class="spacer"></div> <h3><label class="list_list:table_title">Varia</label></h3> <table id="varia"> <thead> <tr> <th>Naam item</th> <th>Aantal</th> </tr> </thead> <tbody> <tr> <td class="item"><input type="text" name="item_3_1" value="" class="inputText" maxlength="64" size="46" tabindex="31"/></td> <td class="amount"><input type="text" name="amount_3_1" value="" class="inputText" maxlength="64" size="10" tabindex="31"/></td> </tr> <tr> <td class="item"><input type="text" name="item_3_2" value="" class="inputText" maxlength="64" size="46" tabindex="32"/></td> <td class="amount"><input type="text" name="amount_3_2" value="" class="inputText" maxlength="64" size="10" tabindex="32"/></td> </tr> <tr> <td class="item"><input type="text" name="item_3_3" value="" class="inputText" maxlength="64" size="46" tabindex="33"/></td> <td class="amount"><input type="text" name="amount_3_3" value="" class="inputText" maxlength="64" size="10" tabindex="33"/></td> </tr> <tr> <td class="item"><input type="text" name="item_3_4" value="" class="inputText" maxlength="64" size="46" tabindex="34"/></td> <td class="amount"><input type="text" name="amount_3_4" value="" class="inputText" maxlength="64" size="10" tabindex="34"/></td> </tr> <tr> <td class="item"><input type="text" name="item_3_5" value="" class="inputText" maxlength="64" size="46" tabindex="35"/></td> <td class="amount"><input type="text" name="amount_3_5" value="" class="inputText" maxlength="64" size="10" tabindex="35"/></td> </tr> </tbody> </table> <input id="list_list:add" type="submit" name="list_list:add" value="voeg extra items toe" class="add" /> <div class="spacer"></div> </div> <div id="formButtons"> <input id="list_info:back" type="submit" name="list_info:back" value="<< Terug" tabindex="11" class="back" /> <input id="list_info:next" type="submit" name="list_info:next" value="Volgende >>" tabindex="12" class="next" /> <input id="list_info:options" type="submit" name="list_info:options" value="Opties" tabindex="13" class="options" /> <input id="list_info:finish" type="submit" name="list_info:finish" value="Voltooien" tabindex="14" disabled="disabled" class="finish" /> </div>

Read the article

Turning PHP page calling Zend functions procedurally into Zend Framework MVC-help!

- by Joel

Hi guys, I posted much of this question, but if didn't include all the Zend stuff because I thought it'd be overkill, but now I'm thinking it's not easy to figure out an OO way of doing this without that code... So with that said, please forgive the verbose code. I'm learning how to use MVC and OO in general, and I have a website that is all in PHP but most of the pages are basic static pages. I have already converted them all to views in Zend Framework, and have the Controller and layout set. All is good there. The one remaining page I have is the main reason I did this...it in fact uses Zend library (for gData connection and pulling info from a Google Calendar and displaying it on the page. I don't know enough about this to know where to begin to refactor the code to fit in the Zend Framework MVC model. Any help would be greatly appreciated!! .phtml view page: <div id="dhtmltooltip" align="left"></div> <script src="../js/tooltip.js" type="text/javascript"> </script> <div id="container"> <div id="conten"> <a name="C4"></a> <?php function get_desc_second_part(&$value) { list(,$val_b) = explode('==',$value); $value = trim($val_b); } function filterEventDetails($contentText) { $data = array(); foreach($contentText as $row) { if(strstr($row, 'When: ')) { ##cleaning "when" string to get date in the format "May 28, 2009"## $data['duration'] = str_replace('When: ','',$row); list($when, ) = explode(' to ',$data['duration']); $data['when'] = substr($when,4); if(strlen($data['when'])>13) $data['when'] = trim(str_replace(strrchr($data['when'], ' '),'',$data['when'])); $data['duration'] = substr($data['duration'], 0, strlen($data['duration'])-4); //trimming time zone identifier (UTC etc.) } if(strstr($row, 'Where: ')) { $data['where'] = str_replace('Where: ','',$row); //pr($row); //$where = strstr($row, 'Where: '); //pr($where); } if(strstr($row, 'Event Description: ')) { $event_desc = str_replace('Event Description: ','',$row); //$event_desc = strstr($row, 'Event Description: '); ## Filtering event description and extracting venue, ticket urls etc from it. //$event_desc = str_replace('Event Description: ','',$contentText[3]); $event_desc_array = explode('|',$event_desc); array_walk($event_desc_array,'get_desc_second_part'); //pr($event_desc_array); $data['venue_url'] = $event_desc_array[0]; $data['details'] = $event_desc_array[1]; $data['tickets_url'] = $event_desc_array[2]; $data['tickets_button'] = $event_desc_array[3]; $data['facebook_url'] = $event_desc_array[4]; $data['facebook_icon'] = $event_desc_array[5]; } } return $data; } // load library require_once 'Zend/Loader.php'; Zend_Loader::loadClass('Zend_Gdata'); Zend_Loader::loadClass('Zend_Gdata_ClientLogin'); Zend_Loader::loadClass('Zend_Gdata_Calendar'); Zend_Loader::loadClass('Zend_Http_Client'); // create authenticated HTTP client for Calendar service $gcal = Zend_Gdata_Calendar::AUTH_SERVICE_NAME; $user = "[email protected]"; $pass = "xxxxxxxx"; $client = Zend_Gdata_ClientLogin::getHttpClient($user, $pass, $gcal); $gcal = new Zend_Gdata_Calendar($client); $query = $gcal->newEventQuery(); $query->setUser('[email protected]'); $secondary=true; $query->setVisibility('private'); $query->setProjection('basic'); $query->setOrderby('starttime'); $query->setSortOrder('ascending'); //$query->setFutureevents('true'); $startDate=date('Y-m-d h:i:s'); $endDate="2015-12-31"; $query->setStartMin($startDate); $query->setStartMax($endDate); $query->setMaxResults(30); try { $feed = $gcal->getCalendarEventFeed($query); } catch (Zend_Gdata_App_Exception $e) { echo "Error: " . $e->getResponse(); } ?> <h1><?php echo $feed->title; ?></h1> <?php echo $feed->totalResults; ?> event(s) found. <table width="90%" border="3" align="center"> <tr> <td width="20%" align="center" valign="middle"><b>;DATE</b></td> <td width="25%" align="center" valign="middle"><b>VENUE</b></td> <td width="20%" align="center" valign="middle"><b>CITY</b></td> <td width="20%" align="center" valign="middle"><b>DETAILS</b></td> <td width="15%" align="center" valign="middle"><b>LINKS</b></td> </tr> <?php if((int)$feed->totalResults>0) { //checking if at least one event is there in this date range foreach ($feed as $event) { //iterating through all events //pr($event);die; $contentText = stripslashes($event->content->text); //striping any escape character $contentText = preg_replace('/\<br \/\>[\n\t\s]{1,}\<br \/\>/','<br />',stripslashes($event->content->text)); //replacing multiple breaks with a single break //die(); $contentText = explode('<br />',$contentText); //splitting data by break tag $eventData = filterEventDetails($contentText); $when = $eventData['when']; $where = $eventData['where']; $duration = $eventData['duration']; $venue_url = $eventData['venue_url']; $details = $eventData['details']; $tickets_url = $eventData['tickets_url']; $tickets_button = $eventData['tickets_button']; $facebook_url = $eventData['facebook_url']; $facebook_icon = $eventData['facebook_icon']; $title = stripslashes($event->title); echo '<tr>'; echo '<td width="20%" align="center" valign="middle" nowrap="nowrap">'; echo $when; echo '</td>'; echo '<td width="20%" align="center" valign="middle">'; if($venue_url!='') { echo '<a href="'.$venue_url.'" target="_blank">'.$title.'</a>'; } else { echo $title; } echo '</td>'; echo '<td width="20%" align="center" valign="middle">'; echo $where; echo '</td>'; echo '<td width="20%" align="center" valign="middle">'; $details = str_replace("\n","<br>",htmlentities($details)); $duration = str_replace("\n","<br>",$duration); $detailed_description = "<b>When</b>: <br>".$duration."<br><br>"; $detailed_description .= "<b>Description</b>: <br>".$details; echo '<a href="javascript:void(0);" onmouseover="ddrivetip(\''.$detailed_description.'\')" onmouseout="hideddrivetip()" onclick="return false">View Details</a>'; echo '</td>'; echo '<td width="20%" valign="middle">'; if(trim($tickets_url) !='' && trim($tickets_button)!='') { echo '<a href="'.$tickets_url.'" target="_blank"><img src="'.$tickets_button.'" border="0" ></a>'; } if(trim($facebook_url) !='' && trim($facebook_icon)!='') { echo '<a href="'.$facebook_url.'" target="_blank"><img src="'.$facebook_icon.'" border="0" ></a>'; } else { echo '......'; } echo '</td>'; echo '</tr>'; } } else { //else show 'no event found' message echo '<tr>'; echo '<td width="100%" align="center" valign="middle" colspan="5">'; echo "No event found"; echo '</td>'; } ?> </table> <h3><a href="#pastevents">Scroll down for a list of past shows.</a></h3> <br /> <a name="pastevents"></a> <ul class="pastShows"> <?php $startDate='2005-01-01'; $endDate=date('Y-m-d'); /*$gcal = Zend_Gdata_Calendar::AUTH_SERVICE_NAME; $user = "[email protected]"; $pass = "silverroof10"; $client = Zend_Gdata_ClientLogin::getHttpClient($user, $pass, $gcal); $gcal = new Zend_Gdata_Calendar($client); $query = $gcal->newEventQuery(); $query->setUser('[email protected]'); $query->setVisibility('private'); $query->setProjection('basic');*/ $query->setOrderby('starttime'); $query->setSortOrder('descending'); $query->setFutureevents('false'); $query->setStartMin($startDate); $query->setStartMax($endDate); $query->setMaxResults(1000); try { $feed = $gcal->getCalendarEventFeed($query); } catch (Zend_Gdata_App_Exception $e) { echo "Error: " . $e->getResponse(); } if((int)$feed->totalResults>0) { //checking if at least one event is there in this date range foreach ($feed as $event) { //iterating through all events $contentText = stripslashes($event->content->text); //striping any escape character $contentText = preg_replace('/\<br \/\>[\n\t\s]{1,}\<br \/\>/','<br />',stripslashes($event->content->text)); //replacing multiple breaks with a single break $contentText = explode('<br />',$contentText); //splitting data by break tag $eventData = filterEventDetails($contentText); $when = $eventData['when']; $where = $eventData['where']; $duration = $eventData['duration']; $title = stripslashes($event->title); echo '<li class="pastShows">' . $when . " - " . $title . ", " . $where . '</li>'; } } ?> </div> </div>

Read the article

Agile Development

- by James Oloo Onyango

Alot of literature has and is being written about agile developement and its surrounding philosophies. In my quest to find the best way to express the importance of agile methodologies, i have found Robert C. Martin's "A Satire Of Two Companies" to be both the most concise and thorough! Enjoy the read! Rufus Inc Project Kick Off Your name is Bob. The date is January 3, 2001, and your head still aches from the recent millennial revelry. You are sitting in a conference room with several managers and a group of your peers. You are a project team leader. Your boss is there, and he has brought along all of his team leaders. His boss called the meeting. "We have a new project to develop," says your boss's boss. Call him BB. The points in his hair are so long that they scrape the ceiling. Your boss's points are just starting to grow, but he eagerly awaits the day when he can leave Brylcream stains on the acoustic tiles. BB describes the essence of the new market they have identified and the product they want to develop to exploit this market. "We must have this new project up and working by fourth quarter October 1," BB demands. "Nothing is of higher priority, so we are cancelling your current project." The reaction in the room is stunned silence. Months of work are simply going to be thrown away. Slowly, a murmur of objection begins to circulate around the conference table. His points give off an evil green glow as BB meets the eyes of everyone in the room. One by one, that insidious stare reduces each attendee to quivering lumps of protoplasm. It is clear that he will brook no discussion on this matter. Once silence has been restored, BB says, "We need to begin immediately. How long will it take you to do the analysis?" You raise your hand. Your boss tries to stop you, but his spitwad misses you and you are unaware of his efforts. "Sir, we can't tell you how long the analysis will take until we have some requirements." "The requirements document won't be ready for 3 or 4 weeks," BB says, his points vibrating with frustration. "So, pretend that you have the requirements in front of you now. How long will you require for analysis?" No one breathes. Everyone looks around to see whether anyone has some idea. "If analysis goes beyond April 1, we have a problem. Can you finish the analysis by then?" Your boss visibly gathers his courage: "We'll find a way, sir!" His points grow 3 mm, and your headache increases by two Tylenol. "Good." BB smiles. "Now, how long will it take to do the design?" "Sir," you say. Your boss visibly pales. He is clearly worried that his 3 mms are at risk. "Without an analysis, it will not be possible to tell you how long design will take." BB's expression shifts beyond austere. "PRETEND you have the analysis already!" he says, while fixing you with his vacant, beady little eyes. "How long will it take you to do the design?" Two Tylenol are not going to cut it. Your boss, in a desperate attempt to save his new growth, babbles: "Well, sir, with only six months left to complete the project, design had better take no longer than 3 months." "I'm glad you agree, Smithers!" BB says, beaming. Your boss relaxes. He knows his points are secure. After a while, he starts lightly humming the Brylcream jingle. BB continues, "So, analysis will be complete by April 1, design will be complete by July 1, and that gives you 3 months to implement the project. This meeting is an example of how well our new consensus and empowerment policies are working. Now, get out there and start working. I'll expect to see TQM plans and QIT assignments on my desk by next week. Oh, and don't forget that your crossfunctional team meetings and reports will be needed for next month's quality audit." "Forget the Tylenol," you think to yourself as you return to your cubicle. "I need bourbon." Visibly excited, your boss comes over to you and says, "Gosh, what a great meeting. I think we're really going to do some world shaking with this project." You nod in agreement, too disgusted to do anything else. "Oh," your boss continues, "I almost forgot." He hands you a 30-page document. "Remember that the SEI is coming to do an evaluation next week. This is the evaluation guide. You need to read through it, memorize it, and then shred it. It tells you how to answer any questions that the SEI auditors ask you. It also tells you what parts of the building you are allowed to take them to and what parts to avoid. We are determined to be a CMM level 3 organization by June!" You and your peers start working on the analysis of the new project. This is difficult because you have no requirements. But from the 10-minute introduction given by BB on that fateful morning, you have some idea of what the product is supposed to do. Corporate process demands that you begin by creating a use case document. You and your team begin enumerating use cases and drawing oval and stick diagrams. Philosophical debates break out among the team members. There is disagreement as to whether certain use cases should be connected with <<extends>> or <<includes>> relationships. Competing models are created, but nobody knows how to evaluate them. The debate continues, effectively paralyzing progress. After a week, somebody finds the iceberg.com Web site, which recommends disposing entirely of <<extends>> and <<includes>> and replacing them with <<precedes>> and <<uses>>. The documents on this Web site, authored by Don Sengroiux, describes a method known as stalwart-analysis, which claims to be a step-by-step method for translating use cases into design diagrams. More competing use case models are created using this new scheme, but again, people can't agree on how to evaluate them. The thrashing continues. More and more, the use case meetings are driven by emotion rather than by reason. If it weren't for the fact that you don't have requirements, you'd be pretty upset by the lack of progress you are making. The requirements document arrives on February 15. And then again on February 20, 25, and every week thereafter. Each new version contradicts the previous one. Clearly, the marketing folks who are writing the requirements, empowered though they might be, are not finding consensus. At the same time, several new competing use case templates have been proposed by the various team members. Each template presents its own particularly creative way of delaying progress. The debates rage on. On March 1, Prudence Putrigence, the process proctor, succeeds in integrating all the competing use case forms and templates into a single, all-encompassing form. Just the blank form is 15 pages long. She has managed to include every field that appeared on all the competing templates. She also presents a 159- page document describing how to fill out the use case form. All current use cases must be rewritten according to the new standard. You marvel to yourself that it now requires 15 pages of fill-in-the-blank and essay questions to answer the question: What should the system do when the user presses Return? The corporate process (authored by L. E. Ott, famed author of "Holistic Analysis: A Progressive Dialectic for Software Engineers") insists that you discover all primary use cases, 87 percent of all secondary use cases, and 36.274 percent of all tertiary use cases before you can complete analysis and enter the design phase. You have no idea what a tertiary use case is. So in an attempt to meet this requirement, you try to get your use case document reviewed by the marketing department, which you hope will know what a tertiary use case is. Unfortunately, the marketing folks are too busy with sales support to talk to you. Indeed, since the project started, you have not been able to get a single meeting with marketing, which has provided a never-ending stream of changing and contradictory requirements documents. While one team has been spinning endlessly on the use case document, another team has been working out the domain model. Endless variations of UML documents are pouring out of this team. Every week, the model is reworked. The team members can't decide whether to use <<interfaces>> or <<types>> in the model. A huge disagreement has been raging on the proper syntax and application of OCL. Others on the team just got back from a 5-day class on catabolism, and have been producing incredibly detailed and arcane diagrams that nobody else can fathom. On March 27, with one week to go before analysis is to be complete, you have produced a sea of documents and diagrams but are no closer to a cogent analysis of the problem than you were on January 3. **** And then, a miracle happens. **** On Saturday, April 1, you check your e-mail from home. You see a memo from your boss to BB. It states unequivocally that you are done with the analysis! You phone your boss and complain. "How could you have told BB that we were done with the analysis?" "Have you looked at a calendar lately?" he responds. "It's April 1!" The irony of that date does not escape you. "But we have so much more to think about. So much more to analyze! We haven't even decided whether to use <<extends>> or <<precedes>>!" "Where is your evidence that you are not done?" inquires your boss, impatiently. "Whaaa . . . ." But he cuts you off. "Analysis can go on forever; it has to be stopped at some point. And since this is the date it was scheduled to stop, it has been stopped. Now, on Monday, I want you to gather up all existing analysis materials and put them into a public folder. Release that folder to Prudence so that she can log it in the CM system by Monday afternoon. Then get busy and start designing." As you hang up the phone, you begin to consider the benefits of keeping a bottle of bourbon in your bottom desk drawer. They threw a party to celebrate the on-time completion of the analysis phase. BB gave a colon-stirring speech on empowerment. And your boss, another 3 mm taller, congratulated his team on the incredible show of unity and teamwork. Finally, the CIO takes the stage to tell everyone that the SEI audit went very well and to thank everyone for studying and shredding the evaluation guides that were passed out. Level 3 now seems assured and will be awarded by June. (Scuttlebutt has it that managers at the level of BB and above are to receive significant bonuses once the SEI awards level 3.) As the weeks flow by, you and your team work on the design of the system. Of course, you find that the analysis that the design is supposedly based on is flawedno, useless; no, worse than useless. But when you tell your boss that you need to go back and work some more on the analysis to shore up its weaker sections, he simply states, "The analysis phase is over. The only allowable activity is design. Now get back to it." So, you and your team hack the design as best you can, unsure of whether the requirements have been properly analyzed. Of course, it really doesn't matter much, since the requirements document is still thrashing with weekly revisions, and the marketing department still refuses to meet with you. The design is a nightmare. Your boss recently misread a book named The Finish Line in which the author, Mark DeThomaso, blithely suggested that design documents should be taken down to code-level detail. "If we are going to be working at that level of detail," you ask, "why don't we simply write the code instead?" "Because then you wouldn't be designing, of course. And the only allowable activity in the design phase is design!" "Besides," he continues, "we have just purchased a companywide license for Dandelion! This tool enables 'Round the Horn Engineering!' You are to transfer all design diagrams into this tool. It will automatically generate our code for us! It will also keep the design diagrams in sync with the code!" Your boss hands you a brightly colored shrinkwrapped box containing the Dandelion distribution. You accept it numbly and shuffle off to your cubicle. Twelve hours, eight crashes, one disk reformatting, and eight shots of 151 later, you finally have the tool installed on your server. You consider the week your team will lose while attending Dandelion training. Then you smile and think, "Any week I'm not here is a good week." Design diagram after design diagram is created by your team. Dandelion makes it very difficult to draw these diagrams. There are dozens and dozens of deeply nested dialog boxes with funny text fields and check boxes that must all be filled in correctly. And then there's the problem of moving classes between packages. At first, these diagram are driven from the use cases. But the requirements are changing so often that the use cases rapidly become meaningless. Debates rage about whether VISITOR or DECORATOR design patterns should be used. One developer refuses to use VISITOR in any form, claiming that it's not a properly object-oriented construct. Someone refuses to use multiple inheritance, since it is the spawn of the devil. Review meetings rapidly degenerate into debates about the meaning of object orientation, the definition of analysis versus design, or when to use aggregation versus association. Midway through the design cycle, the marketing folks announce that they have rethought the focus of the system. Their new requirements document is completely restructured. They have eliminated several major feature areas and replaced them with feature areas that they anticipate customer surveys will show to be more appropriate. You tell your boss that these changes mean that you need to reanalyze and redesign much of the system. But he says, "The analysis phase is system. But he says, "The analysis phase is over. The only allowable activity is design. Now get back to it." You suggest that it might be better to create a simple prototype to show to the marketing folks and even some potential customers. But your boss says, "The analysis phase is over. The only allowable activity is design. Now get back to it." Hack, hack, hack, hack. You try to create some kind of a design document that might reflect the new requirements documents. However, the revolution of the requirements has not caused them to stop thrashing. Indeed, if anything, the wild oscillations of the requirements document have only increased in frequency and amplitude. You slog your way through them. On June 15, the Dandelion database gets corrupted. Apparently, the corruption has been progressive. Small errors in the DB accumulated over the months into bigger and bigger errors. Eventually, the CASE tool just stopped working. Of course, the slowly encroaching corruption is present on all the backups. Calls to the Dandelion technical support line go unanswered for several days. Finally, you receive a brief e-mail from Dandelion, informing you that this is a known problem and that the solution is to purchase the new version, which they promise will be ready some time next quarter, and then reenter all the diagrams by hand. **** Then, on July 1 another miracle happens! You are done with the design! Rather than go to your boss and complain, you stock your middle desk drawer with some vodka. **** They threw a party to celebrate the on-time completion of the design phase and their graduation to CMM level 3. This time, you find BB's speech so stirring that you have to use the restroom before it begins. New banners and plaques are all over your workplace. They show pictures of eagles and mountain climbers, and they talk about teamwork and empowerment. They read better after a few scotches. That reminds you that you need to clear out your file cabinet to make room for the brandy. You and your team begin to code. But you rapidly discover that the design is lacking in some significant areas. Actually, it's lacking any significance at all. You convene a design session in one of the conference rooms to try to work through some of the nastier problems. But your boss catches you at it and disbands the meeting, saying, "The design phase is over. The only allowable activity is coding. Now get back to it." **** The code generated by Dandelion is really hideous. It turns out that you and your team were using association and aggregation the wrong way, after all. All the generated code has to be edited to correct these flaws. Editing this code is extremely difficult because it has been instrumented with ugly comment blocks that have special syntax that Dandelion needs in order to keep the diagrams in sync with the code. If you accidentally alter one of these comments, the diagrams will be regenerated incorrectly. It turns out that "Round the Horn Engineering" requires an awful lot of effort. The more you try to keep the code compatible with Dandelion, the more errors Dandelion generates. In the end, you give up and decide to keep the diagrams up to date manually. A second later, you decide that there's no point in keeping the diagrams up to date at all. Besides, who has time? Your boss hires a consultant to build tools to count the number of lines of code that are being produced. He puts a big thermometer graph on the wall with the number 1,000,000 on the top. Every day, he extends the red line to show how many lines have been added. Three days after the thermometer appears on the wall, your boss stops you in the hall. "That graph isn't growing quickly enough. We need to have a million lines done by October 1." "We aren't even sh-sh-sure that the proshect will require a m-million linezh," you blather. "We have to have a million lines done by October 1," your boss reiterates. His points have grown again, and the Grecian formula he uses on them creates an aura of authority and competence. "Are you sure your comment blocks are big enough?" Then, in a flash of managerial insight, he says, "I have it! I want you to institute a new policy among the engineers. No line of code is to be longer than 20 characters. Any such line must be split into two or more preferably more. All existing code needs to be reworked to this standard. That'll get our line count up!" You decide not to tell him that this will require two unscheduled work months. You decide not to tell him anything at all. You decide that intravenous injections of pure ethanol are the only solution. You make the appropriate arrangements. Hack, hack, hack, and hack. You and your team madly code away. By August 1, your boss, frowning at the thermometer on the wall, institutes a mandatory 50-hour workweek. Hack, hack, hack, and hack. By September 1st, the thermometer is at 1.2 million lines and your boss asks you to write a report describing why you exceeded the coding budget by 20 percent. He institutes mandatory Saturdays and demands that the project be brought back down to a million lines. You start a campaign of remerging lines. Hack, hack, hack, and hack. Tempers are flaring; people are quitting; QA is raining trouble reports down on you. Customers are demanding installation and user manuals; salespeople are demanding advance demonstrations for special customers; the requirements document is still thrashing, the marketing folks are complaining that the product isn't anything like they specified, and the liquor store won't accept your credit card anymore. Something has to give. On September 15, BB calls a meeting. As he enters the room, his points are emitting clouds of steam. When he speaks, the bass overtones of his carefully manicured voice cause the pit of your stomach to roll over. "The QA manager has told me that this project has less than 50 percent of the required features implemented. He has also informed me that the system crashes all the time, yields wrong results, and is hideously slow. He has also complained that he cannot keep up with the continuous train of daily releases, each more buggy than the last!" He stops for a few seconds, visibly trying to compose himself. "The QA manager estimates that, at this rate of development, we won't be able to ship the product until December!" Actually, you think it's more like March, but you don't say anything. "December!" BB roars with such derision that people duck their heads as though he were pointing an assault rifle at them. "December is absolutely out of the question. Team leaders, I want new estimates on my desk in the morning. I am hereby mandating 65-hour work weeks until this project is complete. And it better be complete by November 1." As he leaves the conference room, he is heard to mutter: "Empowermentbah!" * * * Your boss is bald; his points are mounted on BB's wall. The fluorescent lights reflecting off his pate momentarily dazzle you. "Do you have anything to drink?" he asks. Having just finished your last bottle of Boone's Farm, you pull a bottle of Thunderbird from your bookshelf and pour it into his coffee mug. "What's it going to take to get this project done? " he asks. "We need to freeze the requirements, analyze them, design them, and then implement them," you say callously. "By November 1?" your boss exclaims incredulously. "No way! Just get back to coding the damned thing." He storms out, scratching his vacant head. A few days later, you find that your boss has been transferred to the corporate research division. Turnover has skyrocketed. Customers, informed at the last minute that their orders cannot be fulfilled on time, have begun to cancel their orders. Marketing is re-evaluating whether this product aligns with the overall goals of the company. Memos fly, heads roll, policies change, and things are, overall, pretty grim. Finally, by March, after far too many sixty-five hour weeks, a very shaky version of the software is ready. In the field, bug-discovery rates are high, and the technical support staff are at their wits' end, trying to cope with the complaints and demands of the irate customers. Nobody is happy. In April, BB decides to buy his way out of the problem by licensing a product produced by Rupert Industries and redistributing it. The customers are mollified, the marketing folks are smug, and you are laid off. Rupert Industries: Project Alpha Your name is Robert. The date is January 3, 2001. The quiet hours spent with your family this holiday have left you refreshed and ready for work. You are sitting in a conference room with your team of professionals. The manager of the division called the meeting. "We have some ideas for a new project," says the division manager. Call him Russ. He is a high-strung British chap with more energy than a fusion reactor. He is ambitious and driven but understands the value of a team. Russ describes the essence of the new market opportunity the company has identified and introduces you to Jane, the marketing manager, who is responsible for defining the products that will address it. Addressing you, Jane says, "We'd like to start defining our first product offering as soon as possible. When can you and your team meet with me?" You reply, "We'll be done with the current iteration of our project this Friday. We can spare a few hours for you between now and then. After that, we'll take a few people from the team and dedicate them to you. We'll begin hiring their replacements and the new people for your team immediately." "Great," says Russ, "but I want you to understand that it is critical that we have something to exhibit at the trade show coming up this July. If we can't be there with something significant, we'll lose the opportunity." "I understand," you reply. "I don't yet know what it is that you have in mind, but I'm sure we can have something by July. I just can't tell you what that something will be right now. In any case, you and Jane are going to have complete control over what we developers do, so you can rest assured that by July, you'll have the most important things that can be accomplished in that time ready to exhibit." Russ nods in satisfaction. He knows how this works. Your team has always kept him advised and allowed him to steer their development. He has the utmost confidence that your team will work on the most important things first and will produce a high-quality product. * * * "So, Robert," says Jane at their first meeting, "How does your team feel about being split up?" "We'll miss working with each other," you answer, "but some of us were getting pretty tired of that last project and are looking forward to a change. So, what are you people cooking up?" Jane beams. "You know how much trouble our customers currently have . . ." And she spends a half hour or so describing the problem and possible solution. "OK, wait a second" you respond. "I need to be clear about this." And so you and Jane talk about how this system might work. Some of her ideas aren't fully formed. You suggest possible solutions. She likes some of them. You continue discussing. During the discussion, as each new topic is addressed, Jane writes user story cards. Each card represents something that the new system has to do. The cards accumulate on the table and are spread out in front of you. Both you and Jane point at them, pick them up, and make notes on them as you discuss the stories. The cards are powerful mnemonic devices that you can use to represent complex ideas that are barely formed. At the end of the meeting, you say, "OK, I've got a general idea of what you want. I'm going to talk to the team about it. I imagine they'll want to run some experiments with various database structures and presentation formats. Next time we meet, it'll be as a group, and we'll start identifying the most important features of the system." A week later, your nascent team meets with Jane. They spread the existing user story cards out on the table and begin to get into some of the details of the system. The meeting is very dynamic. Jane presents the stories in the order of their importance. There is much discussion about each one. The developers are concerned about keeping the stories small enough to estimate and test. So they continually ask Jane to split one story into several smaller stories. Jane is concerned that each story have a clear business value and priority, so as she splits them, she makes sure that this stays true. The stories accumulate on the table. Jane writes them, but the developers make notes on them as needed. Nobody tries to capture everything that is said; the cards are not meant to capture everything but are simply reminders of the conversation. As the developers become more comfortable with the stories, they begin writing estimates on them. These estimates are crude and budgetary, but they give Jane an idea of what the story will cost. At the end of the meeting, it is clear that many more stories could be discussed. It is also clear that the most important stories have been addressed and that they represent several months worth of work. Jane closes the meeting by taking the cards with her and promising to have a proposal for the first release in the morning. * * * The next morning, you reconvene the meeting. Jane chooses five cards and places them on the table. "According to your estimates, these cards represent about one perfect team-week's worth of work. The last iteration of the previous project managed to get one perfect team-week done in 3 real weeks. If we can get these five stories done in 3 weeks, we'll be able to demonstrate them to Russ. That will make him feel very comfortable about our progress." Jane is pushing it. The sheepish look on her face lets you know that she knows it too. You reply, "Jane, this is a new team, working on a new project. It's a bit presumptuous to expect that our velocity will be the same as the previous team's. However, I met with the team yesterday afternoon, and we all agreed that our initial velocity should, in fact, be set to one perfectweek for every 3 real-weeks. So you've lucked out on this one." "Just remember," you continue, "that the story estimates and the story velocity are very tentative at this point. We'll learn more when we plan the iteration and even more when we implement it." Jane looks over her glasses at you as if to say "Who's the boss around here, anyway?" and then smiles and says, "Yeah, don't worry. I know the drill by now."Jane then puts 15 more cards on the table. She says, "If we can get all these cards done by the end of March, we can turn the system over to our beta test customers. And we'll get good feedback from them." You reply, "OK, so we've got our first iteration defined, and we have the stories for the next three iterations after that. These four iterations will make our first release." "So," says Jane, can you really do these five stories in the next 3 weeks?" "I don't know for sure, Jane," you reply. "Let's break them down into tasks and see what we get." So Jane, you, and your team spend the next several hours taking each of the five stories that Jane chose for the first iteration and breaking them down into small tasks. The developers quickly realize that some of the tasks can be shared between stories and that other tasks have commonalities that can probably be taken advantage of. It is clear that potential designs are popping into the developers' heads. From time to time, they form little discussion knots and scribble UML diagrams on some cards. Soon, the whiteboard is filled with the tasks that, once completed, will implement the five stories for this iteration. You start the sign-up process by saying, "OK, let's sign up for these tasks." "I'll take the initial database generation." Says Pete. "That's what I did on the last project, and this doesn't look very different. I estimate it at two of my perfect workdays." "OK, well, then, I'll take the login screen," says Joe. "Aw, darn," says Elaine, the junior member of the team, "I've never done a GUI, and kinda wanted to try that one." "Ah, the impatience of youth," Joe says sagely, with a wink in your direction. "You can assist me with it, young Jedi." To Jane: "I think it'll take me about three of my perfect workdays." One by one, the developers sign up for tasks and estimate them in terms of their own perfect workdays. Both you and Jane know that it is best to let the developers volunteer for tasks than to assign the tasks to them. You also know full well that you daren't challenge any of the developers' estimates. You know these people, and you trust them. You know that they are going to do the very best they can. The developers know that they can't sign up for more perfect workdays than they finished in the last iteration they worked on. Once each developer has filled his or her schedule for the iteration, they stop signing up for tasks. Eventually, all the developers have stopped signing up for tasks. But, of course, tasks are still left on the board. "I was worried that that might happen," you say, "OK, there's only one thing to do, Jane. We've got too much to do in this iteration. What stories or tasks can we remove?" Jane sighs. She knows that this is the only option. Working overtime at the beginning of a project is insane, and projects where she's tried it have not fared well. So Jane starts to remove the least-important functionality. "Well, we really don't need the login screen just yet. We can simply start the system in the logged-in state." "Rats!" cries Elaine. "I really wanted to do that." "Patience, grasshopper." says Joe. "Those who wait for the bees to leave the hive will not have lips too swollen to relish the honey." Elaine looks confused. Everyone looks confused. "So . . .," Jane continues, "I think we can also do away with . . ." And so, bit by bit, the list of tasks shrinks. Developers who lose a task sign up for one of the remaining ones. The negotiation is not painless. Several times, Jane exhibits obvious frustration and impatience. Once, when tensions are especially high, Elaine volunteers, "I'll work extra hard to make up some of the missing time." You are about to correct her when, fortunately, Joe looks her in the eye and says, "When once you proceed down the dark path, forever will it dominate your destiny." In the end, an iteration acceptable to Jane is reached. It's not what Jane wanted. Indeed, it is significantly less. But it's something the team feels that can be achieved in the next 3 weeks. And, after all, it still addresses the most important things that Jane wanted in the iteration. "So, Jane," you say when things had quieted down a bit, "when can we expect acceptance tests from you?" Jane sighs. This is the other side of the coin. For every story the development team implements, Jane must supply a suite of acceptance tests that prove that it works. And the team needs these long before the end of the iteration, since they will certainly point out differences in the way Jane and the developers imagine the system's behaviour. "I'll get you some example test scripts today," Jane promises. "I'll add to them every day after that. You'll have the entire suite by the middle of the iteration." * * * The iteration begins on Monday morning with a flurry of Class, Responsibilities, Collaborators sessions. By midmorning, all the developers have assembled into pairs and are rapidly coding away. "And now, my young apprentice," Joe says to Elaine, "you shall learn the mysteries of test-first design!" "Wow, that sounds pretty rad," Elaine replies. "How do you do it?" Joe beams. It's clear that he has been anticipating this moment. "OK, what does the code do right now?" "Huh?" replied Elaine, "It doesn't do anything at all; there is no code." "So, consider our task; can you think of something the code should do?" "Sure," Elaine said with youthful assurance, "First, it should connect to the database." "And thereupon, what must needs be required to connecteth the database?" "You sure talk weird," laughed Elaine. "I think we'd have to get the database object from some registry and call the Connect() method. "Ah, astute young wizard. Thou perceives correctly that we requireth an object within which we can cacheth the database object." "Is 'cacheth' really a word?" "It is when I say it! So, what test can we write that we know the database registry should pass?" Elaine sighs. She knows she'll just have to play along. "We should be able to create a database object and pass it to the registry in a Store() method. And then we should be able to pull it out of the registry with a Get() method and make sure it's the same object." "Oh, well said, my prepubescent sprite!" "Hay!" "So, now, let's write a test function that proves your case." "But shouldn't we write the database object and registry object first?" "Ah, you've much to learn, my young impatient one. Just write the test first." "But it won't even compile!" "Are you sure? What if it did?" "Uh . . ." "Just write the test, Elaine. Trust me." And so Joe, Elaine, and all the other developers began to code their tasks, one test case at a time. The room in which they worked was abuzz with the conversations between the pairs. The murmur was punctuated by an occasional high five when a pair managed to finish a task or a difficult test case. As development proceeded, the developers changed partners once or twice a day. Each developer got to see what all the others were doing, and so knowledge of the code spread generally throughout the team. Whenever a pair finished something significant whether a whole task or simply an important part of a task they integrated what they had with the rest of the system. Thus, the code base grew daily, and integration difficulties were minimized. The developers communicated with Jane on a daily basis. They'd go to her whenever they had a question about the functionality of the system or the interpretation of an acceptance test case. Jane, good as her word, supplied the team with a steady stream of acceptance test scripts. The team read these carefully and thereby gained a much better understanding of what Jane expected the system to do. By the beginning of the second week, there was enough functionality to demonstrate to Jane. She watched eagerly as the demonstration passed test case after test case. "This is really cool," Jane said as the demonstration finally ended. "But this doesn't seem like one-third of the tasks. Is your velocity slower than anticipated?" You grimace. You'd been waiting for a good time to mention this to Jane but now she was forcing the issue. "Yes, unfortunately, we are going more slowly than we had expected. The new application server we are using is turning out to be a pain to configure. Also, it takes forever to reboot, and we have to reboot it whenever we make even the slightest change to its configuration." Jane eyes you with suspicion. The stress of last Monday's negotiations had still not entirely dissipated. She says, "And what does this mean to our schedule? We can't slip it again, we just can't. Russ will have a fit! He'll haul us all into the woodshed and ream us some new ones." You look Jane right in the eyes. There's no pleasant way to give someone news like this. So you just blurt out, "Look, if things keep going like they're going, we're not going to be done with everything by next Friday. Now it's possible that we'll figure out a way to go faster. But, frankly, I wouldn't depend on that. You should start thinking about one or two tasks that could be removed from the iteration without ruining the demonstration for Russ. Come hell or high water, we are going to give that demonstration on Friday, and I don't think you want us to choose which tasks to omit." "Aw forchrisakes!" Jane barely manages to stifle yelling that last word as she stalks away, shaking her head. Not for the first time, you say to yourself, "Nobody ever promised me project management would be easy." You are pretty sure it won't be the last time, either. Actually, things went a bit better than you had hoped. The team did, in fact, have to drop one task from the iteration, but Jane had chosen wisely, and the demonstration for Russ went without a hitch. Russ was not impressed with the progress, but neither was he dismayed. He simply said, "This is pretty good. But remember, we have to be able to demonstrate this system at the trade show in July, and at this rate, it doesn't look like you'll have all that much to show." Jane, whose attitude had improved dramatically with the completion of the iteration, responded to Russ by saying, "Russ, this team is working hard, and well. When July comes around, I am confident that we'll have something significant to demonstrate. It won't be everything, and some of it may be smoke and mirrors, but we'll have something." Painful though the last iteration was, it had calibrated your velocity numbers. The next iteration went much better. Not because your team got more done than in the last iteration but simply because the team didn't have to remove any tasks or stories in the middle of the iteration. By the start of the fourth iteration, a natural rhythm has been established. Jane, you, and the team know exactly what to expect from one another. The team is running hard, but the pace is sustainable. You are confident that the team can keep up this pace for a year or more. The number of surprises in the schedule diminishes to near zero; however, the number of surprises in the requirements does not. Jane and Russ frequently look over the growing system and make recommendations or changes to the existing functionality. But all parties realize that these changes take time and must be scheduled. So the changes do not cause anyone's expectations to be violated. In March, there is a major demonstration of the system to the board of directors. The system is very limited and is not yet in a form good enough to take to the trade show, but progress is steady, and the board is reasonably impressed. The second release goes even more smoothly than the first. By now, the team has figured out a way to automate Jane's acceptance test scripts. The team has also refactored the design of the system to the point that it is really easy to add new features and change old ones. The second release was done by the end of June and was taken to the trade show. It had less in it than Jane and Russ would have liked, but it did demonstrate the most important features of the system. Although customers at the trade show noticed that certain features were missing, they were very impressed overall. You, Russ, and Jane all returned from the trade show with smiles on your faces. You all felt as though this project was a winner. Indeed, many months later, you are contacted by Rufus Inc. That company had been working on a system like this for its internal operations. Rufus has canceled the development of that system after a death-march project and is negotiating to license your technology for its environment. Indeed, things are looking up!

Read the article

Sensible Way to Pass Web Data in XML to a SQL Server Database

- by Emtucifor

After exploring several different ways to pass web data to a database for update purposes, I'm wondering if XML might be a good strategy. The database is currently SQL 2000. In a few months it will move to SQL 2005 and I will be able to change things if needed, but I need a SQL 2000 solution now. First of all, the database in question uses the EAV model. I know that this kind of database is generally highly frowned on, so for the purposes of this question, please just accept that this is not going to change. The current update method has the web server inserting values (that have all been converted first to their correct underlying types, then to sql_variant) to a temp table. A stored procedure is then run which expects the temp table to exist and it takes care of updating, inserting, or deleting things as needed. So far, only a single element has needed to be updated at a time. But now, there is a requirement to be able to edit multiple elements at once, and also to support hierarchical elements, each of which can have its own list of attributes. Here's some example XML I hand-typed to demonstrate what I'm thinking of. Note that in this database the Entity is Element and an ID of 0 signifies "create" aka an insert of a new item. <Elements> <Element ID="1234"> <Attr ID="221">Value</Attr> <Attr ID="225">287</Attr> <Attr ID="234"> <Element ID="99825"> <Attr ID="7">Value1</Attr> <Attr ID="8">Value2</Attr> <Attr ID="9" Action="delete" /> </Element> <Element ID="99826" Action="delete" /> <Element ID="0" Type="24"> <Attr ID="7">Value4</Attr> <Attr ID="8">Value5</Attr> <Attr ID="9">Value6</Attr> </Element> <Element ID="0" Type="24"> <Attr ID="7">Value7</Attr> <Attr ID="8">Value8</Attr> <Attr ID="9">Value9</Attr> </Element> </Attr> <Rel ID="3827" Action="delete" /> <Rel ID="2284" Role="parent"> <Element ID="3827" /> <Element ID="3829" /> <Attr ID="665">1</Attr> </Rel> <Rel ID="0" Type="23" Role="child"> <Element ID="3830" /> <Attr ID="67" </Rel> </Element> <Element ID="0" Type="87"> <Attr ID="221">Value</Attr> <Attr ID="225">569</Attr> <Attr ID="234"> <Element ID="0" Type="24"> <Attr ID="7">Value10</Attr> <Attr ID="8">Value11</Attr> <Attr ID="9">Value12</Attr> </Element> </Attr> </Element> <Element ID="1235" Action="delete" /> </Elements> Some Attributes are straight value types, such as AttrID 221. But AttrID 234 is a special "multi-value" type that can have a list of elements underneath it, and each one can have one or more values. Types only need to be presented when a new item is created, since the ElementID fully implies the type if it already exists. I'll probably support only passing in changed items (as detected by javascript). And there may be an Action="Delete" on Attr elements as well, since NULLs are treated as "unselected"--sometimes it's very important to know if a Yes/No question has intentionally been answered No or if no one's bothered to say Yes yet. There is also a different kind of data, a Relationship. At this time, those are updated through individual AJAX calls as things are edited in the UI, but I'd like to include those so that changes to relationships can be canceled (right now, once you change it, it's done). So those are really elements, too, but they are called Rel instead of Element. Relationships are implemented as ElementID1 and ElementID2, so the RelID 2284 in the XML above is in the database as: ElementID 2284 ElementID1 1234 ElementID2 3827 Having multiple children in one relationship isn't currently supported, but it would be nice later. Does this strategy and the example XML make sense? Is there a more sensible way? I'm just looking for some broad critique to help save me from going down a bad path. Any aspect that you'd like to comment on would be helpful. The web language happens to be Classic ASP, but that could change to ASP.Net at some point. A persistence engine like Linq or nHibernate is probably not acceptable right now--I just want to get this already working application enhanced without a huge amount of development time. I'll choose the answer that shows experience and has a balance of good warnings about what not to do, confirmations of what I'm planning to do, and recommendations about something else to do. I'll make it as objective as possible. P.S. I'd like to handle unicode characters as well as very long strings (10k +). UPDATE I have had this working for some time and I used the ADO Recordset Save-To-Stream trick to make creating the XML really easy. The result seems to be fairly fast, though if speed ever becomes a problem I may revisit this. In the meantime, my code works to handle any number of elements and attributes on the page at once, including updating, deleting, and creating new items all in one go. I settled on a scheme like so for all my elements: Existing data elements Example: input name e12345_a678 (element 12345, attribute 678), the input value is the value of the attribute. New elements Javascript copies a hidden template of the set of HTML elements needed for the type into the correct location on the page, increments a counter to get a new ID for this item, and prepends the number to the names of the form items. var newid = 0; function metadataAdd(reference, nameid, value) { var t = document.createElement('input'); t.setAttribute('name', nameid); t.setAttribute('id', nameid); t.setAttribute('type', 'hidden'); t.setAttribute('value', value); reference.appendChild(t); } function multiAdd(target, parentelementid, attrid, elementtypeid) { var proto = document.getElementById('a' + attrid + '_proto'); var instance = document.createElement('p'); target.parentNode.parentNode.insertBefore(instance, target.parentNode); var thisid = ++newid; instance.innerHTML = proto.innerHTML.replace(/{prefix}/g, 'n' + thisid + '_'); instance.id = 'n' + thisid; instance.className += ' new'; metadataAdd(instance, 'n' + thisid + '_p', parentelementid); metadataAdd(instance, 'n' + thisid + '_c', attrid); metadataAdd(instance, 'n' + thisid + '_t', elementtypeid); return false; } Example: Template input name _a678 becomes n1_a678 (a new element, the first one on the page, attribute 678). all attributes of this new element are tagged with the same prefix of n1. The next new item will be n2, and so on. Some hidden form inputs are created: n1_t, value is the elementtype of the element to be created n1_p, value is the parent id of the element (if it is a relationship) n1_c, value is the child id of the element (if it is a relationship) Deleting elements A hidden input is created in the form e12345_t with value set to 0. The existing controls displaying that attribute's values are disabled so they are not included in the form post. So "set type to 0" is treated as delete. With this scheme, every item on the page has a unique name and can be distinguished properly, and every action can be represented properly. When the form is posted, here's a sample of building one of the two recordsets used (classic ASP code): Set Data = Server.CreateObject("ADODB.Recordset") Data.Fields.Append "ElementID", adInteger, 4, adFldKeyColumn Data.Fields.Append "AttrID", adInteger, 4, adFldKeyColumn Data.Fields.Append "Value", adLongVarWChar, 2147483647, adFldIsNullable Or adFldMayBeNull Data.CursorLocation = adUseClient Data.CursorType = adOpenDynamic Data.Open This is the recordset for values, the other is for the elements themselves. I step through the posted form and for the element recordset use a Scripting.Dictionary populated with instances of a custom Class that has the properties I need, so that I can add the values piecemeal, since they don't always come in order. New elements are added as negative to distinguish them from regular elements (rather than requiring a separate column to indicate if it is new or addresses an existing element). I use regular expression to tear apart the form keys: "^(e|n)([0-9]{1,10})_(a|p|t|c)([0-9]{0,10})$" Then, adding an attribute looks like this. Data.AddNew ElementID.Value = DataID AttrID.Value = Integerize(Matches(0).SubMatches(3)) AttrValue.Value = Request.Form(Key) Data.Update ElementID, AttrID, and AttrValue are references to the fields of the recordset. This method is hugely faster than using Data.Fields("ElementID").Value each time. I loop through the Dictionary of element updates and ignore any that don't have all the proper information, adding the good ones to the recordset. Then I call my data-updating stored procedure like so: Set Cmd = Server.CreateObject("ADODB.Command") With Cmd Set .ActiveConnection = MyDBConn .CommandType = adCmdStoredProc .CommandText = "DataPost" .Prepared = False .Parameters.Append .CreateParameter("@ElementMetadata", adLongVarWChar, adParamInput, 2147483647, XMLFromRecordset(Element)) .Parameters.Append .CreateParameter("@ElementData", adLongVarWChar, adParamInput, 2147483647, XMLFromRecordset(Data)) End With Result.Open Cmd ' previously created recordset object with options set Here's the function that does the xml conversion: Private Function XMLFromRecordset(Recordset) Dim Stream Set Stream = Server.CreateObject("ADODB.Stream") Stream.Open Recordset.Save Stream, adPersistXML Stream.Position = 0 XMLFromRecordset = Stream.ReadText End Function Just in case the web page needs to know, the SP returns a recordset of any new elements, showing their page value and their created value (so I can see that n1 is now e12346 for example). Here are some key snippets from the stored procedure. Note this is SQL 2000 for now, though I'll be able to switch to 2005 soon: CREATE PROCEDURE [dbo].[DataPost] @ElementMetaData ntext, @ElementData ntext AS DECLARE @hdoc int --- snip --- EXEC sp_xml_preparedocument @hdoc OUTPUT, @ElementMetaData, '<xml xmlns:s="uuid:BDC6E3F0-6DA3-11d1-A2A3-00AA00C14882" xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:rs="urn:schemas-microsoft-com:rowset" xmlns:z="#RowsetSchema" />' INSERT #ElementMetadata (ElementID, ElementTypeID, ElementID1, ElementID2) SELECT * FROM OPENXML(@hdoc, '/xml/rs:data/rs:insert/z:row', 0) WITH ( ElementID int, ElementTypeID int, ElementID1 int, ElementID2 int ) ORDER BY ElementID -- orders negative items (new elements) first so they begin counting at 1 for later ID calculation EXEC sp_xml_removedocument @hdoc --- snip --- UPDATE E SET E.ElementTypeID = M.ElementTypeID FROM Element E INNER JOIN #ElementMetadata M ON E.ElementID = M.ElementID WHERE E.ElementID >= 1 AND M.ElementTypeID >= 1 The following query does the correlation of the negative new element ids to the newly inserted ones: UPDATE #ElementMetadata -- Correlate the new ElementIDs with the input rows SET NewElementID = Scope_Identity() - @@RowCount + DataID WHERE ElementID < 0 Other set-based queries do all the other work of validating that the attributes are allowed, are the correct data type, and inserting, updating, and deleting elements and attributes. I hope this brief run-down is useful to others some day! Converting ADO Recordsets to an XML stream was a huge winner for me as it saved all sorts of time and had a namespace and schema already defined that made the results come out correctly. Using a flatter XML format with 2 inputs was also much easier than sticking to some ideal about having everything in a single XML stream.

Read the article

Sensible Way to Pass Web Data to Sql Server Database

- by Emtucifor

After exploring several different ways to pass web data to a database for update purposes, I'm wondering if XML might be a good strategy. The database is currently SQL 2000. In a few months it will move to SQL 2005 and I will be able to change things if needed, but I need a SQL 2000 solution now. First of all, the database in question uses the EAV model. I know that this kind of database is generally highly frowned on, so for the purposes of this question, please just accept that this is not going to change. The current update method has the web server inserting values (that have all been converted first to their correct underlying types, then to sql_variant) to a temp table. A stored procedure is then run which expects the temp table to exist and it takes care of updating, inserting, or deleting things as needed. So far, only a single element has needed to be updated at a time. But now, there is a requirement to be able to edit multiple elements at once, and also to support hierarchical elements, each of which can have its own list of attributes. Here's some example XML I hand-typed to demonstrate what I'm thinking of. Note that in this database the Entity is Element and an ID of 0 signifies "create" aka an insert of a new item. <Elements> <Element ID="1234"> <Attr ID="221">Value</Attr> <Attr ID="225">287</Attr> <Attr ID="234"> <Element ID="99825"> <Attr ID="7">Value1</Attr> <Attr ID="8">Value2</Attr> <Attr ID="9" Action="delete" /> </Element> <Element ID="99826" Action="delete" /> <Element ID="0" Type="24"> <Attr ID="7">Value4</Attr> <Attr ID="8">Value5</Attr> <Attr ID="9">Value6</Attr> </Element> <Element ID="0" Type="24"> <Attr ID="7">Value7</Attr> <Attr ID="8">Value8</Attr> <Attr ID="9">Value9</Attr> </Element> </Attr> <Rel ID="3827" Action="delete" /> <Rel ID="2284" Role="parent"> <Element ID="3827" /> <Element ID="3829" /> <Attr ID="665">1</Attr> </Rel> <Rel ID="0" Type="23" Role="child"> <Element ID="3830" /> <Attr ID="67" </Rel> </Element> <Element ID="0" Type="87"> <Attr ID="221">Value</Attr> <Attr ID="225">569</Attr> <Attr ID="234"> <Element ID="0" Type="24"> <Attr ID="7">Value10</Attr> <Attr ID="8">Value11</Attr> <Attr ID="9">Value12</Attr> </Element> </Attr> </Element> <Element ID="1235" Action="delete" /> </Elements> Some Attributes are straight value types, such as AttrID 221. But AttrID 234 is a special "multi-value" type that can have a list of elements underneath it, and each one can have one or more values. Types only need to be presented when a new item is created, since the ElementID fully implies the type if it already exists. I'll probably support only passing in changed items (as detected by javascript). And there may be an Action="Delete" on Attr elements as well, since NULLs are treated as "unselected"--sometimes it's very important to know if a Yes/No question has intentionally been answered No or if no one's bothered to say Yes yet. There is also a different kind of data, a Relationship. At this time, those are updated through individual AJAX calls as things are edited in the UI, but I'd like to include those so that changes to relationships can be canceled (right now, once you change it, it's done). So those are really elements, too, but they are called Rel instead of Element. Relationships are implemented as ElementID1 and ElementID2, so the RelID 2284 in the XML above is in the database as: ElementID 2284 ElementID1 1234 ElementID2 3827 Having multiple children in one relationship isn't currently supported, but it would be nice later. Does this strategy and the example XML make sense? Is there a more sensible way? I'm just looking for some broad critique to help save me from going down a bad path. Any aspect that you'd like to comment on would be helpful. The web language happens to be Classic ASP, but that could change to ASP.Net at some point. A persistence engine like Linq or nHibernate is probably not acceptable right now--I just want to get this already working application enhanced without a huge amount of development time. I'll choose the answer that shows experience and has a balance of good warnings about what not to do, confirmations of what I'm planning to do, and recommendations about something else to do. I'll make it as objective as possible. P.S. I'd like to handle unicode characters as well as very long strings (10k +). UPDATE I have had this working for some time and I used the ADO Recordset Save-To-Stream trick to make creating the XML really easy. The result seems to be fairly fast, though if speed ever becomes a problem I may revisit this. In the meantime, my code works to handle any number of elements and attributes on the page at once, including updating, deleting, and creating new items all in one go. I settled on a scheme like so for all my elements: Existing data elements Example: input name e12345_a678 (element 12345, attribute 678), the input value is the value of the attribute. New elements Javascript copies a hidden template of the set of HTML elements needed for the type into the correct location on the page, increments a counter to get a new ID for this item, and prepends the number to the names of the form items. var newid = 0; function metadataAdd(reference, nameid, value) { var t = document.createElement('input'); t.setAttribute('name', nameid); t.setAttribute('id', nameid); t.setAttribute('type', 'hidden'); t.setAttribute('value', value); reference.appendChild(t); } function multiAdd(target, parentelementid, attrid, elementtypeid) { var proto = document.getElementById('a' + attrid + '_proto'); var instance = document.createElement('p'); target.parentNode.parentNode.insertBefore(instance, target.parentNode); var thisid = ++newid; instance.innerHTML = proto.innerHTML.replace(/{prefix}/g, 'n' + thisid + '_'); instance.id = 'n' + thisid; instance.className += ' new'; metadataAdd(instance, 'n' + thisid + '_p', parentelementid); metadataAdd(instance, 'n' + thisid + '_c', attrid); metadataAdd(instance, 'n' + thisid + '_t', elementtypeid); return false; } Example: Template input name _a678 becomes n1_a678 (a new element, the first one on the page, attribute 678). all attributes of this new element are tagged with the same prefix of n1. The next new item will be n2, and so on. Some hidden form inputs are created: n1_t, value is the elementtype of the element to be created n1_p, value is the parent id of the element (if it is a relationship) n1_c, value is the child id of the element (if it is a relationship) Deleting elements A hidden input is created in the form e12345_t with value set to 0. The existing controls displaying that attribute's values are disabled so they are not included in the form post. So "set type to 0" is treated as delete. With this scheme, every item on the page has a unique name and can be distinguished properly, and every action can be represented properly. When the form is posted, here's a sample of building one of the two recordsets used (classic ASP code): Set Data = Server.CreateObject("ADODB.Recordset") Data.Fields.Append "ElementID", adInteger, 4, adFldKeyColumn Data.Fields.Append "AttrID", adInteger, 4, adFldKeyColumn Data.Fields.Append "Value", adLongVarWChar, 2147483647, adFldIsNullable Or adFldMayBeNull Data.CursorLocation = adUseClient Data.CursorType = adOpenDynamic Data.Open This is the recordset for values, the other is for the elements themselves. I step through the posted form and for the element recordset use a Scripting.Dictionary populated with instances of a custom Class that has the properties I need, so that I can add the values piecemeal, since they don't always come in order. New elements are added as negative to distinguish them from regular elements (rather than requiring a separate column to indicate if it is new or addresses an existing element). I use regular expression to tear apart the form keys: "^(e|n)([0-9]{1,10})_(a|p|t|c)([0-9]{0,10})$" Then, adding an attribute looks like this. Data.AddNew ElementID.Value = DataID AttrID.Value = Integerize(Matches(0).SubMatches(3)) AttrValue.Value = Request.Form(Key) Data.Update ElementID, AttrID, and AttrValue are references to the fields of the recordset. This method is hugely faster than using Data.Fields("ElementID").Value each time. I loop through the Dictionary of element updates and ignore any that don't have all the proper information, adding the good ones to the recordset. Then I call my data-updating stored procedure like so: Set Cmd = Server.CreateObject("ADODB.Command") With Cmd Set .ActiveConnection = MyDBConn .CommandType = adCmdStoredProc .CommandText = "DataPost" .Prepared = False .Parameters.Append .CreateParameter("@ElementMetadata", adLongVarWChar, adParamInput, 2147483647, XMLFromRecordset(Element)) .Parameters.Append .CreateParameter("@ElementData", adLongVarWChar, adParamInput, 2147483647, XMLFromRecordset(Data)) End With Result.Open Cmd ' previously created recordset object with options set Here's the function that does the xml conversion: Private Function XMLFromRecordset(Recordset) Dim Stream Set Stream = Server.CreateObject("ADODB.Stream") Stream.Open Recordset.Save Stream, adPersistXML Stream.Position = 0 XMLFromRecordset = Stream.ReadText End Function Just in case the web page needs to know, the SP returns a recordset of any new elements, showing their page value and their created value (so I can see that n1 is now e12346 for example). Here are some key snippets from the stored procedure. Note this is SQL 2000 for now, though I'll be able to switch to 2005 soon: CREATE PROCEDURE [dbo].[DataPost] @ElementMetaData ntext, @ElementData ntext AS DECLARE @hdoc int --- snip --- EXEC sp_xml_preparedocument @hdoc OUTPUT, @ElementMetaData, '<xml xmlns:s="uuid:BDC6E3F0-6DA3-11d1-A2A3-00AA00C14882" xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:rs="urn:schemas-microsoft-com:rowset" xmlns:z="#RowsetSchema" />' INSERT #ElementMetadata (ElementID, ElementTypeID, ElementID1, ElementID2) SELECT * FROM OPENXML(@hdoc, '/xml/rs:data/rs:insert/z:row', 0) WITH ( ElementID int, ElementTypeID int, ElementID1 int, ElementID2 int ) ORDER BY ElementID -- orders negative items (new elements) first so they begin counting at 1 for later ID calculation EXEC sp_xml_removedocument @hdoc --- snip --- UPDATE E SET E.ElementTypeID = M.ElementTypeID FROM Element E INNER JOIN #ElementMetadata M ON E.ElementID = M.ElementID WHERE E.ElementID >= 1 AND M.ElementTypeID >= 1 The following query does the correlation of the negative new element ids to the newly inserted ones: UPDATE #ElementMetadata -- Correlate the new ElementIDs with the input rows SET NewElementID = Scope_Identity() - @@RowCount + DataID WHERE ElementID < 0 Other set-based queries do all the other work of validating that the attributes are allowed, are the correct data type, and inserting, updating, and deleting elements and attributes. I hope this brief run-down is useful to others some day! Converting ADO Recordsets to an XML stream was a huge winner for me as it saved all sorts of time and had a namespace and schema already defined that made the results come out correctly. Using a flatter XML format with 2 inputs was also much easier than sticking to some ideal about having everything in a single XML stream.

Read the article

Nullpointerexcption & abrupt IOStream closure with inheritence and subclasses

- by user1401652

A brief background before so we can communicate on the same wave length. I've had about 8-10 university courses on programming from data structure, to one on all languages, to specific ones such as java & c++. I'm a bit rusty because i usually take 2-3 month breaks from coding. This is a personal project that I started thinking of two years back. Okay down to the details, and a specific question, I'm having problems with my mutator functions. It seems to be that I am trying to access a private variable incorrectly. The question is, am I nesting my classes too much and trying to mutate a base class variable the incorrect way. If so point me in the way of the correct literature, or confirm this is my problem so I can restudy this information. Thanks package GroceryReceiptProgram; import java.io.*; import java.util.Vector; public class Date { private int hour, minute, day, month, year; Date() { try { BufferedReader keyboard = new BufferedReader(new InputStreamReader(System.in)); System.out.println("What's the hour? (Use 1-24 military notation"); hour = Integer.parseInt(keyboard.readLine()); System.out.println("what's the minute? "); minute = Integer.parseInt(keyboard.readLine()); System.out.println("What's the day of the month?"); day = Integer.parseInt(keyboard.readLine()); System.out.println("Which month of the year is it, use an integer"); month = Integer.parseInt(keyboard.readLine()); System.out.println("What year is it?"); year = Integer.parseInt(keyboard.readLine()); keyboard.close(); } catch (IOException e) { System.out.println("Yo houston we have a problem"); } } public void setHour(int hour) { this.hour = hour; } public void setHour() { try { BufferedReader keyboard = new BufferedReader(new InputStreamReader(System.in)); System.out.println("What hour, use military notation?"); this.hour = Integer.parseInt(keyboard.readLine()); keyboard.close(); } catch (NumberFormatException e) { System.out.println(e.toString() + ":doesnt seem to be a number"); } catch (IOException e) { System.out.println(e.toString()); } } public int getHour() { return hour; } public void setMinute(int minute) { this.minute = minute; } public void setMinute() { try (BufferedReader keyboard = new BufferedReader(new InputStreamReader(System.in))) { System.out.println("What minute?"); this.minute = Integer.parseInt(keyboard.readLine()); } catch (NumberFormatException e) { System.out.println(e.toString() + ": doesnt seem to be a number"); } catch (IOException e) { System.out.println(e.toString() + ": minute shall not cooperate"); } catch (NullPointerException e) { System.out.println(e.toString() + ": in the setMinute function of the Date class"); } } public int getMinute() { return minute; } public void setDay(int day) { this.day = day; } public void setDay() { try { BufferedReader keyboard = new BufferedReader(new InputStreamReader(System.in)); System.out.println("What day 0-6?"); this.day = Integer.parseInt(keyboard.readLine()); keyboard.close(); } catch (NumberFormatException e) { System.out.println(e.toString() + ":doesnt seem to be a number"); } catch (IOException e) { System.out.println(e.toString()); } } public int getDay() { return day; } public void setMonth(int month) { this.month = month; } public void setMonth() { try { BufferedReader keyboard = new BufferedReader(new InputStreamReader(System.in)); System.out.println("What month 0-11?"); this.month = Integer.parseInt(keyboard.readLine()); keyboard.close(); } catch (NumberFormatException e) { System.out.println(e.toString() + ":doesnt seem to be a number"); } catch (IOException e) { System.out.println(e.toString()); } } public int getMonth() { return month; } public void setYear(int year) { this.year = year; } public void setYear() { try { BufferedReader keyboard = new BufferedReader(new InputStreamReader(System.in)); System.out.println("What year?"); this.year = Integer.parseInt(keyboard.readLine()); keyboard.close(); } catch (NumberFormatException e) { System.out.println(e.toString() + ":doesnt seem to be a number"); } catch (IOException e) { System.out.println(e.toString()); } } public int getYear() { return year; } public void set() { setMinute(); setHour(); setDay(); setMonth(); setYear(); } public Vector<Integer> get() { Vector<Integer> holder = new Vector<Integer>(5); holder.add(hour); holder.add(minute); holder.add(month); holder.add(day); holder.add(year); return holder; } }; That is the Date class obviously, next is the other base class Location. package GroceryReceiptProgram; import java.io.*; import java.util.Vector; public class Location { String streetName, state, city, country; int zipCode, address; Location() { try { BufferedReader keyboard = new BufferedReader(new InputStreamReader(System.in)); System.out.println("What is the street name"); streetName = keyboard.readLine(); System.out.println("Which state?"); state = keyboard.readLine(); System.out.println("Which city?"); city = keyboard.readLine(); System.out.println("Which country?"); country = keyboard.readLine(); System.out.println("Which zipcode?");//if not u.s. continue around this step zipCode = Integer.parseInt(keyboard.readLine()); System.out.println("What address?"); address = Integer.parseInt(keyboard.readLine()); } catch (IOException e) { System.out.println(e.toString()); } } public void setZipCode(int zipCode) { this.zipCode = zipCode; } public void setZipCode() { try { BufferedReader keyboard = new BufferedReader(new InputStreamReader(System.in)); System.out.println("What zipCode?"); this.zipCode = Integer.parseInt(keyboard.readLine()); keyboard.close(); } catch (NumberFormatException e) { System.out.println(e.toString() + ":doesnt seem to be a number"); } catch (IOException e) { System.out.println(e.toString()); } } public void set() { setAddress(); setCity(); setCountry(); setState(); setStreetName(); setZipCode(); } public int getZipCode() { return zipCode; } public void setAddress(int address) { this.address = address; } public void setAddress() { try { BufferedReader keyboard = new BufferedReader(new InputStreamReader(System.in)); System.out.println("What minute?"); this.address = Integer.parseInt(keyboard.readLine()); keyboard.close(); } catch (NumberFormatException e) { System.out.println(e.toString() + ":doesnt seem to be a number"); } catch (IOException e) { System.out.println(e.toString()); } } public int getAddress() { return address; } public void setStreetName(String streetName) { this.streetName = streetName; } public void setStreetName() { try { BufferedReader keyboard = new BufferedReader(new InputStreamReader(System.in)); System.out.println("What minute?"); this.streetName = keyboard.readLine(); keyboard.close(); } catch (IOException e) { System.out.println(e.toString()); } } public String getStreetName() { return streetName; } public void setState(String state) { this.state = state; } public void setState() { try { BufferedReader keyboard = new BufferedReader(new InputStreamReader(System.in)); System.out.println("What minute?"); this.state = keyboard.readLine(); keyboard.close(); } catch (IOException e) { System.out.println(e.toString()); } } public String getState() { return state; } public void setCity(String city) { this.city = city; } public void setCity() { try { BufferedReader keyboard = new BufferedReader(new InputStreamReader(System.in)); System.out.println("What minute?"); this.city = keyboard.readLine(); keyboard.close(); } catch (IOException e) { System.out.println(e.toString()); } } public String getCity() { return city; } public void setCountry(String country) { this.country = country; } public void setCountry() { try { BufferedReader keyboard = new BufferedReader(new InputStreamReader(System.in)); System.out.println("What minute?"); this.country = keyboard.readLine(); keyboard.close(); } catch (IOException e) { System.out.println(e.toString()); } } public String getCountry() { return country; } }; their parent(What is the proper name?) class package GroceryReceiptProgram; import java.io.*; public class FoodGroup { private int price, count; private Date purchaseDate, expirationDate; private Location location; private String name; public FoodGroup() { try { setPrice(); setCount(); expirationDate.set(); purchaseDate.set(); location.set(); } catch (NullPointerException e) { System.out.println(e.toString() + ": in the constructor of the FoodGroup class"); } } public void setPrice(int price) { this.price = price; } public void setPrice() { try (BufferedReader keyboard = new BufferedReader(new InputStreamReader(System.in))) { System.out.println("What Price?"); price = Integer.parseInt(keyboard.readLine()); } catch (NumberFormatException e) { System.out.println(e.toString() + ":doesnt seem to be a number"); } catch (IOException e) { System.out.println(e.toString() + ": in the FoodGroup class, setPrice function"); } catch (NullPointerException e) { System.out.println(e.toString() + ": in FoodGroup class. SetPrice()"); } } public int getPrice() { return price; } public void setCount(int count) { this.count = count; } public void setCount() { try (BufferedReader keyboard = new BufferedReader(new InputStreamReader(System.in))) { System.out.println("What count?"); count = Integer.parseInt(keyboard.readLine()); } catch (NumberFormatException e) { System.out.println(e.toString() + ":doesnt seem to be a number"); } catch (IOException e) { System.out.println(e.toString() + ": in the FoodGroup class, setCount()"); } catch (NullPointerException e) { System.out.println(e.toString() + ": in FoodGroup class, setCount"); } } public int getCount() { return count; } public void setName(String name) { this.name = name; } public void setName() { try { BufferedReader keyboard = new BufferedReader(new InputStreamReader(System.in)); System.out.println("What minute?"); this.name = keyboard.readLine(); } catch (IOException e) { System.out.println(e.toString()); } } public String getName() { return name; } public void setLocation(Location location) { this.location = location; } public Location getLocation() { return location; } public void setPurchaseDate(Date purchaseDate) { this.purchaseDate = purchaseDate; } public void setPurchaseDate() { this.purchaseDate.set(); } public Date getPurchaseDate() { return purchaseDate; } public void setExpirationDate(Date expirationDate) { this.expirationDate = expirationDate; } public void setExpirationDate() { this.expirationDate.set(); } public Date getExpirationDate() { return expirationDate; } } and finally the main class, so I can get access to all of this work. package GroceryReceiptProgram; public class NewMain { public static void main(String[] args) { FoodGroup test = new FoodGroup(); } } If anyone is further interested, here is a link the UML for this. https://www.dropbox.com/s/1weigjnxih70tbv/GRP.dia

Search Results

Search found 11226 results on 450 pages for 'reverse thinking'.

Page 449/450 | < Previous Page | 445 446 447 448 449 450 | Next Page >

- by Anthony

- by john.rose

- by gotgenes

- by Nemo

- by ZeroDefect

- by Peterson

- by pedalpete

- by Danny Chia

- by Jesse Aldridge

- by Uri

- by Andrew

- by Phil

- by Jonathan Stowell

- by Phil

- by Nicholi

- by Rattenmann

- by Mark Lewis

- by Danny

- by eagle

- by koko

- by Joel

- by James Oloo Onyango

- by Emtucifor

- by Emtucifor

- by user1401652

< Previous Page | 445 446 447 448 449 450 | Next Page >