attribute keys - Page 431

how to debug "deep" crashes in Android?

- by eerok512

Hi All, I've been trying to debug an android crash that is occurring without a Java Stack Trace... Java Stack Trace bugs are very easy for me to fix... but this bug I'm getting seems to be crashing inside the "NDK" or whatever it is the deep internals of Android are called... I've made no modifications to the NDK btw... I just dunno what else to call that layer hehe. Anyway I'm mainly looking for advice on deep-debug methods, rather than help with this specific problem... because I doubt I can post all the source code involved... so really I just need to know how to set breakpoints at the deep layers or whatever other methods there are to trace deep-crashes to their source... so I will briefly describe the bug and then post a LogCat. I have an app with 7 Activities Activity_INTRO Activity_EULA Activity_MAIN Activity_Contact Activity_News Activity_Library Activity_More INTRO is the initiating one... it fades in some company logos... after displaying them for a set time it jumps to the EULA activity... after the user accepts the EULA, it jumps to MAIN... MAIN then creates a TabHost and populates it with the 4 remaining activities now heres the thing... when I click on say, the More tab of the TabHost, the app pauses for a few seconds and then hard-crashes... no java stack trace, but an actual ASM level trace with the registers and IP and stack... the same thing occurs no matter which tab I select, Contact, News, Library, More... all of them crash with the same hard-crash if however I set the manifest to start the app at Activity_MAIN, bypassing the INTRO and EULA, then these crashes do not occur... so something is lingering from those opening activities that is somehow hosing the TabHost'ed Activities... and I'm wondering what the hell that could be... because I'm using finish() on those activites when they need to jump... in fact here is how I'm doing it let me know if you see any bugs: when jumping from INTRO to EULA I do: //Display the EULA Intent newIntent = new Intent (avi, Activity_EULA.class); startActivity (newIntent); finish(); and EULA to MAIN: Intent newIntent = new Intent (this, Activity_Main.class); startActivity (newIntent); finish(); anyway, here is the hard crash log... please let me know if there is some way I can reverse engineer either /system/lib/libcutils.so or /system/lib/libandroid_runtime.so, because I think the crash is happening in one of them... i think its happening in the libandroid_runtime in fact.... anyway on to the log: 12-25 00:56:07.322: INFO/DEBUG(551): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** 12-25 00:56:07.332: INFO/DEBUG(551): Build fingerprint: 'generic/sdk/generic/:1.5/CUPCAKE/150240:eng/test-keys' 12-25 00:56:07.362: INFO/DEBUG(551): pid: 722, tid: 723 >>> com.killerapps.chokes <<< 12-25 00:56:07.362: INFO/DEBUG(551): signal 11 (SIGSEGV), fault addr 00000004 12-25 00:56:07.362: INFO/DEBUG(551): r0 00000004 r1 40021800 r2 00000004 r3 ad3296c5 12-25 00:56:07.372: INFO/DEBUG(551): r4 00000000 r5 00000000 r6 ad342da5 r7 41039fb8 12-25 00:56:07.372: INFO/DEBUG(551): r8 100ffcb0 r9 41039fb0 10 41e014a0 fp 00001071 12-25 00:56:07.382: INFO/DEBUG(551): ip ad35b874 sp 100ffc98 lr ad3296cf pc afb045a8 cpsr 00000010 12-25 00:56:07.552: INFO/DEBUG(551): #00 pc 000045a8 /system/lib/libcutils.so 12-25 00:56:07.572: INFO/DEBUG(551): #01 lr ad3296cf /system/lib/libandroid_runtime.so 12-25 00:56:07.582: INFO/DEBUG(551): stack: 12-25 00:56:07.582: INFO/DEBUG(551): 100ffc58 00000000 12-25 00:56:07.592: INFO/DEBUG(551): 100ffc5c 001c5278 [heap] 12-25 00:56:07.602: INFO/DEBUG(551): 100ffc60 000000da 12-25 00:56:07.602: INFO/DEBUG(551): 100ffc64 0016c778 [heap] 12-25 00:56:07.602: INFO/DEBUG(551): 100ffc68 100ffcc8 12-25 00:56:07.602: INFO/DEBUG(551): 100ffc6c 001c5278 [heap] 12-25 00:56:07.612: INFO/DEBUG(551): 100ffc70 427d1ac0 12-25 00:56:07.612: INFO/DEBUG(551): 100ffc74 000000c1 12-25 00:56:07.612: INFO/DEBUG(551): 100ffc78 40021800 12-25 00:56:07.612: INFO/DEBUG(551): 100ffc7c 000000c2 12-25 00:56:07.612: INFO/DEBUG(551): 100ffc80 00000000 12-25 00:56:07.612: INFO/DEBUG(551): 100ffc84 00000000 12-25 00:56:07.622: INFO/DEBUG(551): 100ffc88 00000000 12-25 00:56:07.622: INFO/DEBUG(551): 100ffc8c 00000000 12-25 00:56:07.622: INFO/DEBUG(551): 100ffc90 df002777 12-25 00:56:07.632: INFO/DEBUG(551): 100ffc94 e3a070ad 12-25 00:56:07.632: INFO/DEBUG(551): #00 100ffc98 00000000 12-25 00:56:07.632: INFO/DEBUG(551): 100ffc9c ad3296cf /system/lib/libandroid_runtime.so 12-25 00:56:07.632: INFO/DEBUG(551): 100ffca0 100ffcd0 12-25 00:56:07.642: INFO/DEBUG(551): 100ffca4 ad342db5 /system/lib/libandroid_runtime.so 12-25 00:56:07.642: INFO/DEBUG(551): 100ffca8 410a79d0 12-25 00:56:07.642: INFO/DEBUG(551): 100ffcac ad00e3b8 /system/lib/libdvm.so 12-25 00:56:07.652: INFO/DEBUG(551): 100ffcb0 410a79d0 12-25 00:56:07.652: INFO/DEBUG(551): 100ffcb4 0016bac0 [heap] 12-25 00:56:07.662: INFO/DEBUG(551): 100ffcb8 ad342da5 /system/lib/libandroid_runtime.so 12-25 00:56:07.662: INFO/DEBUG(551): 100ffcbc 40021800 12-25 00:56:07.662: INFO/DEBUG(551): 100ffcc0 410a79d0 12-25 00:56:07.662: INFO/DEBUG(551): 100ffcc4 afe39dd0 12-25 00:56:07.662: INFO/DEBUG(551): 100ffcc8 100ffcd0 12-25 00:56:07.662: INFO/DEBUG(551): 100ffccc ad040a8d /system/lib/libdvm.so 12-25 00:56:07.672: INFO/DEBUG(551): 100ffcd0 41039fb0 12-25 00:56:07.672: INFO/DEBUG(551): 100ffcd4 420000f8 12-25 00:56:07.672: INFO/DEBUG(551): 100ffcd8 ad342da5 /system/lib/libandroid_runtime.so 12-25 00:56:07.672: INFO/DEBUG(551): 100ffcdc 100ffd48 12-25 00:56:07.852: DEBUG/dalvikvm(722): GC freed 367 objects / 15144 bytes in 210ms 12-25 00:56:08.081: DEBUG/InetAddress(722): www.akillerapp.com: 74.86.47.202 (family 2, proto 6) 12-25 00:56:08.242: DEBUG/dalvikvm(722): GC freed 62 objects / 2328 bytes in 122ms 12-25 00:56:08.771: DEBUG/dalvikvm(722): GC freed 245 objects / 11744 bytes in 179ms 12-25 00:56:09.131: INFO/ActivityManager(577): Process com.killerapps.chokes (pid 722) has died. 12-25 00:56:09.171: INFO/WindowManager(577): WIN DEATH: Window{43719320 com.killerapps.chokes/com.killerapps.chokes.Activity_Main paused=false} 12-25 00:56:09.251: INFO/DEBUG(551): debuggerd committing suicide to free the zombie! 12-25 00:56:09.291: DEBUG/Zygote(553): Process 722 terminated by signal (11) 12-25 00:56:09.311: INFO/DEBUG(781): debuggerd: Jun 30 2009 17:00:51 12-25 00:56:09.331: WARN/InputManagerService(577): Got RemoteException sending setActive(false) notification to pid 722 uid 10020

Read the article

PHP sorting issue with simpleXML

- by tugbucket

Read the article

Adapting a HTML/CSS dropdown menu to multi-level

- by Adam Nygate

Ive been trying to make the original dropdown into multi level for a site im working on. All of my attempts have failed (. For some reason i can only do "margin-right" to align the elements, and this causes some problems. I think it has something to do with the position attribute. Here is my HTML: <ol id="nav"> <li><a href="index.php">Home</a></li> <li class="dropdown_alignedLeft"> <a href="">Products</a> <ul><li class="dropdown_alignedRight"> <a href="">iPoP</a> <ul style="margin-right:-400px; top:0px;-webkit-border-top-right-radius: 5px;border-top-right-radius: 5px;-moz-border-radius-topright: 5px;"><li><a href="customers.php?category=ipop">iPoP - Network Solutions for Vessels</a></li></ul><li class="dropdown_alignedRight"> <a href="">Cameras</a> <ul style="margin-right:-400px; top:0px;-webkit-border-top-right-radius: 5px;border-top-right-radius: 5px;-moz-border-radius-topright: 5px;"><li><a href="customers.php?category=icam">iCam 501 Ultra - Intrinsically Safe Digital Camera with Flash</a></li></ul><li class="dropdown_alignedRight"> <a href="">BNWAS</a> <ul style="margin-right:-400px; top:0px;-webkit-border-top-right-radius: 5px;border-top-right-radius: 5px;-moz-border-radius-topright: 5px;"><li><a href="customers.php?category=bnwas">BNWAS - Bridge Navigation Watch Alarm System</a></li></ul><li class="dropdown_alignedRight"> <a href="">Lighting</a> <ul style="margin-right:-400px; top:0px;-webkit-border-top-right-radius: 5px;border-top-right-radius: 5px;-moz-border-radius-topright: 5px;"><li><a href="customers.php?category=peli">Peli 2690 - Intrinsically Safe LED Head Lamp</a></li></ul><li class="dropdown_alignedRight"> <a href="">Communication</a> <ul style="margin-right:-400px; top:0px;-webkit-border-top-right-radius: 5px;border-top-right-radius: 5px;-moz-border-radius-topright: 5px;"><li><a href="customers.php?category=handy">Ex-Handy 06 - Intrinsically Safe Cell Phone</a></li></ul> </ul> <li class="dropdown_alignedLeft"> <a href="">Customers</a> <ul> <li><a href="customers.php?category=maritime">Maritime</a></li> <li><a href="customers.php?category=non">Non-Maritime</a></li> <li class="dropdown_lastItem"><a href="customers.php?category=organizations">Regulatory Organizations</a></li> </ul> <li><a href="order.php">Product Enquiry</a></li> <li><a href="contact.php">Contact Us</a></li> <li class="dropdown_alignedLeft"> <a href="">Company</a> <ul>  <li><a href="newsandpr.php?category=News">News</a></li> <li class="dropdown_lastItem"><a href="newsandpr.php?category=Press Release">Press Releases</a></li> </ul> </ol> And my CSS: #nav { float:right; margin:15px 0 0; } #nav li { float:left; } #nav li a { display:block; font-family:"PT Sans","Helvetica Neue",Arial,sans-serif; font-size:16px; text-decoration:none; color:#2B95C8; padding:10px 20px 20px; } .dropdown_alignedLeft,.dropdown_alignedRight { position:relative; } #nav .dropdown_alignedLeft>a,#nav .dropdown_alignedRight>a { background:url(../images/dropdown_arrow_blue.png) no-repeat top right; padding:10px 30px 20px 20px; } #nav .dropdown_alignedLeft:hover>a,#nav .dropdown_alignedRight:hover>a { -moz-border-radius-topleft:5px; -moz-border-radius-topright:5px; -moz-border-radius-bottomright:0; -moz-border-radius-bottomleft:0; -webkit-border-top-left-radius:5px; -webkit-border-top-right-radius:5px; -webkit-border-bottom-right-radius:0; -webkit-border-bottom-left-radius:0; border-top-left-radius:5px; border-top-right-radius:5px; border-bottom-right-radius:0; border-bottom-left-radius:0; color:#FFF; background:#2378A1 url(../images/dropdown_arrow_blue.png) no-repeat bottom right; } .dropdown_alignedLeft ul,.dropdown_alignedRight ul { display:none; } #nav .dropdown_alignedLeft:hover>ul,#nav .dropdown_alignedRight:hover>ul { display:block; z-index:100; position:absolute; top:50px; -moz-border-radius-topleft:0; -moz-border-radius-topright:0; -moz-border-radius-bottomright:5px; -moz-border-radius-bottomleft:5px; -webkit-border-top-left-radius:0; -webkit-border-top-right-radius:0; -webkit-border-bottom-right-radius:5px; -webkit-border-bottom-left-radius:5px; border-top-left-radius:0; border-top-right-radius:0; border-bottom-right-radius:5px; border-bottom-left-radius:5px; background:#2378A1; padding:0 0 6px; } #nav .dropdown_alignedRight:hover>ul { top:50px; right:0; text-align:right; } #nav li ul li { float:none; border-bottom:1px dashed #2B95C8; margin:0 20px; } #nav li ul li.dropdown_innerTitle { border:none; font-family:"Helvetica Neue",Arial,sans-serif; font-size:15px; white-space:nowrap; color:#C8DDE7; margin:10px 20px 0; padding:10px 0; } #nav li ul li.dropdown_lastItem { border:none; } #nav li ul li a { font-family:"Helvetica Neue",Arial,sans-serif; font-size:13px; color:#FFF; white-space:nowrap; padding:10px 0 9px; } #nav>li:hover>a,#nav li .current_page { color:#2378A1; background:url(../images/current_page_arrow_blue.png) no-repeat center bottom; } #nav li ul li a:hover { color: #C8DDE7; } For a live version of the menu, please go here: JSFiddle - Live Menu

Read the article

g++ SSE intrinsics dilemma - value from intrinsic "saturates"

- by Sriram

Hi, I wrote a simple program to implement SSE intrinsics for computing the inner product of two large (100000 or more elements) vectors. The program compares the execution time for both, inner product computed the conventional way and using intrinsics. Everything works out fine, until I insert (just for the fun of it) an inner loop before the statement that computes the inner product. Before I go further, here is the code: //this is a sample Intrinsics program to compute inner product of two vectors and compare Intrinsics with traditional method of doing things. #include <iostream> #include <iomanip> #include <xmmintrin.h> #include <stdio.h> #include <time.h> #include <stdlib.h> using namespace std; typedef float v4sf __attribute__ ((vector_size(16))); double innerProduct(float* arr1, int len1, float* arr2, int len2) { //assume len1 = len2. float result = 0.0; for(int i = 0; i < len1; i++) { for(int j = 0; j < len1; j++) { result += (arr1[i] * arr2[i]); } } //float y = 1.23e+09; //cout << "y = " << y << endl; return result; } double sse_v4sf_innerProduct(float* arr1, int len1, float* arr2, int len2) { //assume that len1 = len2. if(len1 != len2) { cout << "Lengths not equal." << endl; exit(1); } /*steps: * 1. load a long-type (4 float) into a v4sf type data from both arrays. * 2. multiply the two. * 3. multiply the same and store result. * 4. add this to previous results. */ v4sf arr1Data, arr2Data, prevSums, multVal, xyz; //__builtin_ia32_xorps(prevSums, prevSums); //making it equal zero. //can explicitly load 0 into prevSums using loadps or storeps (Check). float temp[4] = {0.0, 0.0, 0.0, 0.0}; prevSums = __builtin_ia32_loadups(temp); float result = 0.0; for(int i = 0; i < (len1 - 3); i += 4) { for(int j = 0; j < len1; j++) { arr1Data = __builtin_ia32_loadups(&arr1[i]); arr2Data = __builtin_ia32_loadups(&arr2[i]); //store the contents of two arrays. multVal = __builtin_ia32_mulps(arr1Data, arr2Data); //multiply. xyz = __builtin_ia32_addps(multVal, prevSums); prevSums = xyz; } } //prevSums will hold the sums of 4 32-bit floating point values taken at a time. Individual entries in prevSums also need to be added. __builtin_ia32_storeups(temp, prevSums); //store prevSums into temp. cout << "Values of temp:" << endl; for(int i = 0; i < 4; i++) cout << temp[i] << endl; result += temp[0] + temp[1] + temp[2] + temp[3]; return result; } int main() { clock_t begin, end; int length = 100000; float *arr1, *arr2; double result_Conventional, result_Intrinsic; // printStats("Allocating memory."); arr1 = new float[length]; arr2 = new float[length]; // printStats("End allocation."); srand(time(NULL)); //init random seed. // printStats("Initializing array1 and array2"); begin = clock(); for(int i = 0; i < length; i++) { // for(int j = 0; j < length; j++) { // arr1[i] = rand() % 10 + 1; arr1[i] = 2.5; // arr2[i] = rand() % 10 - 1; arr2[i] = 2.5; // } } end = clock(); cout << "Time to initialize array1 and array2 = " << ((double) (end - begin)) / CLOCKS_PER_SEC << endl; // printStats("Finished initialization."); // printStats("Begin inner product conventionally."); begin = clock(); result_Conventional = innerProduct(arr1, length, arr2, length); end = clock(); cout << "Time to compute inner product conventionally = " << ((double) (end - begin)) / CLOCKS_PER_SEC << endl; // printStats("End inner product conventionally."); // printStats("Begin inner product using Intrinsics."); begin = clock(); result_Intrinsic = sse_v4sf_innerProduct(arr1, length, arr2, length); end = clock(); cout << "Time to compute inner product with intrinsics = " << ((double) (end - begin)) / CLOCKS_PER_SEC << endl; //printStats("End inner product using Intrinsics."); cout << "Results: " << endl; cout << " result_Conventional = " << result_Conventional << endl; cout << " result_Intrinsics = " << result_Intrinsic << endl; return 0; } I use the following g++ invocation to build this: g++ -W -Wall -O2 -pedantic -march=i386 -msse intrinsics_SSE_innerProduct.C -o innerProduct Each of the loops above, in both the functions, runs a total of N^2 times. However, given that arr1 and arr2 (the two floating point vectors) are loaded with a value 2.5, the length of the array is 100,000, the result in both cases should be 6.25e+10. The results I get are: Results: result_Conventional = 6.25e+10 result_Intrinsics = 5.36871e+08 This is not all. It seems that the value returned from the function that uses intrinsics "saturates" at the value above. I tried putting other values for the elements of the array and different sizes too. But it seems that any value above 1.0 for the array contents and any size above 1000 meets with the same value we see above. Initially, I thought it might be because all operations within SSE are in floating point, but floating point should be able to store a number that is of the order of e+08. I am trying to see where I could be going wrong but cannot seem to figure it out. I am using g++ version: g++ (GCC) 4.4.1 20090725 (Red Hat 4.4.1-2). Any help on this is most welcome. Thanks, Sriram.

Read the article

F# - Facebook Hacker Cup - Double Squares

- by Jacob

I'm working on strengthening my F#-fu and decided to tackle the Facebook Hacker Cup Double Squares problem. I'm having some problems with the run-time and was wondering if anyone could help me figure out why it is so much slower than my C# equivalent. There's a good description from another post; Source: Facebook Hacker Cup Qualification Round 2011 A double-square number is an integer X which can be expressed as the sum of two perfect squares. For example, 10 is a double-square because 10 = 3^2 + 1^2. Given X, how can we determine the number of ways in which it can be written as the sum of two squares? For example, 10 can only be written as 3^2 + 1^2 (we don't count 1^2 + 3^2 as being different). On the other hand, 25 can be written as 5^2 + 0^2 or as 4^2 + 3^2. You need to solve this problem for 0 = X = 2,147,483,647. Examples: 10 = 1 25 = 2 3 = 0 0 = 1 1 = 1 My basic strategy (which I'm open to critique on) is to; Create a dictionary (for memoize) of the input numbers initialzed to 0 Get the largest number (LN) and pass it to count/memo function Get the LN square root as int Calculate squares for all numbers 0 to LN and store in dict Sum squares for non repeat combinations of numbers from 0 to LN If sum is in memo dict, add 1 to memo Finally, output the counts of the original numbers. Here is the F# code (See code changes at bottom) I've written that I believe corresponds to this strategy (Runtime: ~8:10); open System open System.Collections.Generic open System.IO /// Get a sequence of values let rec range min max = seq { for num in [min .. max] do yield num } /// Get a sequence starting from 0 and going to max let rec zeroRange max = range 0 max /// Find the maximum number in a list with a starting accumulator (acc) let rec maxNum acc = function | [] -> acc | p::tail when p > acc -> maxNum p tail | p::tail -> maxNum acc tail /// A helper for finding max that sets the accumulator to 0 let rec findMax nums = maxNum 0 nums /// Build a collection of combinations; ie [1,2,3] = (1,1), (1,2), (1,3), (2,2), (2,3), (3,3) let rec combos range = seq { let count = ref 0 for inner in range do for outer in Seq.skip !count range do yield (inner, outer) count := !count + 1 } let rec squares nums = let dict = new Dictionary<int, int>() for s in nums do dict.[s] <- (s * s) dict /// Counts the number of possible double squares for a given number and keeps track of other counts that are provided in the memo dict. let rec countDoubleSquares (num: int) (memo: Dictionary<int, int>) = // The highest relevent square is the square root because it squared plus 0 squared is the top most possibility let maxSquare = System.Math.Sqrt((float)num) // Our relevant squares are 0 to the highest possible square; note the cast to int which shouldn't hurt. let relSquares = range 0 ((int)maxSquare) // calculate the squares up front; let calcSquares = squares relSquares // Build up our square combinations; ie [1,2,3] = (1,1), (1,2), (1,3), (2,2), (2,3), (3,3) for (sq1, sq2) in combos relSquares do let v = calcSquares.[sq1] + calcSquares.[sq2] // Memoize our relevant results if memo.ContainsKey(v) then memo.[v] <- memo.[v] + 1 // return our count for the num passed in memo.[num] // Read our numbers from file. //let lines = File.ReadAllLines("test2.txt") //let nums = [ for line in Seq.skip 1 lines -> Int32.Parse(line) ] // Optionally, read them from straight array let nums = [1740798996; 1257431873; 2147483643; 602519112; 858320077; 1048039120; 415485223; 874566596; 1022907856; 65; 421330820; 1041493518; 5; 1328649093; 1941554117; 4225; 2082925; 0; 1; 3] // Initialize our memoize dictionary let memo = new Dictionary<int, int>() for num in nums do memo.[num] <- 0 // Get the largest number in our set, all other numbers will be memoized along the way let maxN = findMax nums // Do the memoize let maxCount = countDoubleSquares maxN memo // Output our results. for num in nums do printfn "%i" memo.[num] // Have a little pause for when we debug let line = Console.Read() And here is my version in C# (Runtime: ~1:40: using System; using System.Collections.Generic; using System.Diagnostics; using System.IO; using System.Linq; using System.Text; namespace FBHack_DoubleSquares { public class TestInput { public int NumCases { get; set; } public List<int> Nums { get; set; } public TestInput() { Nums = new List<int>(); } public int MaxNum() { return Nums.Max(); } } class Program { static void Main(string[] args) { // Read input from file. //TestInput input = ReadTestInput("live.txt"); // As example, load straight. TestInput input = new TestInput { NumCases = 20, Nums = new List<int> { 1740798996, 1257431873, 2147483643, 602519112, 858320077, 1048039120, 415485223, 874566596, 1022907856, 65, 421330820, 1041493518, 5, 1328649093, 1941554117, 4225, 2082925, 0, 1, 3, } }; var maxNum = input.MaxNum(); Dictionary<int, int> memo = new Dictionary<int, int>(); foreach (var num in input.Nums) { if (!memo.ContainsKey(num)) memo.Add(num, 0); } DoMemoize(maxNum, memo); StringBuilder sb = new StringBuilder(); foreach (var num in input.Nums) { //Console.WriteLine(memo[num]); sb.AppendLine(memo[num].ToString()); } Console.Write(sb.ToString()); var blah = Console.Read(); //File.WriteAllText("out.txt", sb.ToString()); } private static int DoMemoize(int num, Dictionary<int, int> memo) { var highSquare = (int)Math.Floor(Math.Sqrt(num)); var squares = CreateSquareLookup(highSquare); var relSquares = squares.Keys.ToList(); Debug.WriteLine("Starting - " + num.ToString()); Debug.WriteLine("RelSquares.Count = {0}", relSquares.Count); int sum = 0; var index = 0; foreach (var square in relSquares) { foreach (var inner in relSquares.Skip(index)) { sum = squares[square] + squares[inner]; if (memo.ContainsKey(sum)) memo[sum]++; } index++; } if (memo.ContainsKey(num)) return memo[num]; return 0; } private static TestInput ReadTestInput(string fileName) { var lines = File.ReadAllLines(fileName); var input = new TestInput(); input.NumCases = int.Parse(lines[0]); foreach (var lin in lines.Skip(1)) { input.Nums.Add(int.Parse(lin)); } return input; } public static Dictionary<int, int> CreateSquareLookup(int maxNum) { var dict = new Dictionary<int, int>(); int square; foreach (var num in Enumerable.Range(0, maxNum)) { square = num * num; dict[num] = square; } return dict; } } } Thanks for taking a look. UPDATE Changing the combos function slightly will result in a pretty big performance boost (from 8 min to 3:45): /// Old and Busted... let rec combosOld range = seq { let rangeCache = Seq.cache range let count = ref 0 for inner in rangeCache do for outer in Seq.skip !count rangeCache do yield (inner, outer) count := !count + 1 } /// The New Hotness... let rec combos maxNum = seq { for i in 0..maxNum do for j in i..maxNum do yield i,j }

Read the article

What arguments do I send a function being called by a button in python?

- by Jared

I have a UI, in that UI is 4 text fields and 1 int field, then I have a function that calls to another function based on what's inside of the text fields, this function has (self, *args). My function that is being called to takes five arguments and I don't know what to put in it to make it actually work with my UI because python button's send an argument of their own. I have tried self and *args, but it doesn't work. Here is my code, didn't include most of the UI code since it is self explanatory: def crBC(self, IKJoint, FKJoint, bindJoint, xQuan, switch): ''' You should have a controller with an attribute 'ikFkBlend' - The name can be changed after the script executes. Controller should contain an enum - FK/DYN(0), IK(1). Specify the IK joint, then either the dynamic or FK joint, then the bind joint. Then a quantity of joints to pass through and connect. Tested currently on 600 joints (200 x 3), executed in less than a second. Returns nothing. Please open your script editor for details. ''' import itertools # gets children joints of the selected joint chHipIK = cmds.listRelatives(IKJoint, ad = True, type = 'joint') chHipFK = cmds.listRelatives(FKJoint, ad = True, type = 'joint') chHipBind = cmds.listRelatives(bindJoint, ad = True, type = 'joint') # list is built backwards, this reverses the list chHipIK.reverse() chHipFK.reverse() chHipBind.reverse() # appends the initial joint to the list chHipIK.append(IKJoint) chHipFK.append(FKJoint) chHipBind.append(bindJoint) # puts the last joint at the start of the list because the initial joint # was added to the end chHipIK.insert(0, chHipIK.pop()) chHipFK.insert(0, chHipFK.pop()) chHipBind.insert(0, chHipBind.pop()) # pops off the remaining joints in the list the user does not wish to be blended chHipBind[xQuan:] = [] chHipIK[xQuan:] = [] chHipFK[xQuan:] = [] # goes through the bind joints, makes a blend colors for each one, connects # the switch to the blender for a, b, c in itertools.izip(chHipBind, chHipIK, chHipFK): rotBC = cmds.shadingNode('blendColors', asUtility = True, n = a + 'rotate_BC') tranBC = cmds.shadingNode('blendColors', asUtility = True, n = a + 'tran_BC') scaleBC = cmds.shadingNode('blendColors', asUtility = True, n = a + 'scale_BC') cmds.connectAttr(switch + '.ikFkSwitch', rotBC + '.blender') cmds.connectAttr(switch + '.ikFkSwitch', tranBC + '.blender') cmds.connectAttr(switch + '.ikFkSwitch', scaleBC + '.blender') # goes through the ik joints, connects to the blend colors cmds.connectAttr(b + '.rotate', rotBC + '.color1', force = True) cmds.connectAttr(b + '.translate', tranBC + '.color1', force = True) cmds.connectAttr(b + '.scale', scaleBC + '.color1', force = True) # connects FK joints to the blend colors cmds.connectAttr(c + '.rotate', rotBC + '.color2') cmds.connectAttr(c + '.translate', tranBC + '.color2') cmds.connectAttr(c + '.scale', scaleBC + '.color2') # connects blend colors to bind joints cmds.connectAttr(rotBC + '.output', a + '.rotate') cmds.connectAttr(tranBC + '.output', a + '.translate') cmds.connectAttr(scaleBC + '.output', a + '.scale') ------------------- def execCrBC(self, *args): g.crBC(cmds.textField(self.ikJBC, q = True, tx = True), cmds.textField(self.fkJBC, q = True, tx = True), cmds.textField(self.bindJBC, q = True, tx = True), cmds.intField(self.bQBC, q = True, v = True), cmds.textField(self.sCBC, q = True, tx = True)) ------------------- self.bQBC = cmds.intField() cmds.text(l = '') self.sCBC = cmds.textField() cmds.text(l = '') cmds.button(l = 'Help Docs', c = self.crBC.__doc__) cmds.setParent('..') cmds.button(l = 'Create', c = self.execCrBC) Here is the code causing the problem as requested: import maya.cmds as cmds import jtRigUI.createDummyRig as dum import jtRigUI.createSkeleton as sk import jtRigUI.generalUtilities as gu import jtRigUI.createLegRig as lr import jtRigUI.createArmRig as ar class RUI(dum.Dict, dum.Dummy, sk.Skel, sk.FiSkel, lr.LeanLocs, lr.LegRig, ar.ArmRig, gu.Gutils): def __init__(self, charNameUI, gScaleUI, fingButtonGrp, thumbCheckBox, spineButtonGrp, neckButtonGrp, ikJBC, fkJBC, bindJBC, bQBC, sCBC): rigUI = 'rigUI' if cmds.window(rigUI, exists = True): cmds.deleteUI(rigUI) rigUI = cmds.window(rigUI, t = 'JT Rigging UI', sizeable = False, tb = True, mnb = False, mxb = False, menuBar = True, tlb = True, nm = 5) form = cmds.formLayout() tabs = cmds.tabLayout(innerMarginWidth = 1, innerMarginHeight = 1) rigUIMenu = cmds.menu('Help', hm = True) aboutMenu = cmds.menuItem('about') cmds.popupMenu('about', button = 1) deleteUIMenu = cmds.menu('Delete', hm = True) cmds.menuItem('dummySkeleton') cmds.formLayout(form, edit = True, attachForm = ((tabs, 'top', 0), (tabs, 'left', 0), (tabs, 'bottom', 0), (tabs, 'right', 0)), w = 30) tab1 = cmds.rowColumnLayout('Dummy') #cmds.columnLayout(rowSpacing = 10) #cmds.setParent('..') cmds.frameLayout(l = 'A: Dummy Skeleton Setup', w = 400) self.charNameUI = cmds.textFieldGrp (label="Optional Character Name:", ann="Insert a name for the character or leave empty.", tx = '', w = 1) fingJUI = cmds.frameLayout(l = 'B: Number of Fingers', w = 10) cmds.text('\n', h = 5) self.fingButtonGrp = cmds.radioButtonGrp('fingRadio', p = fingJUI, l = 'Fingers: ', sl = 4, w = 1, numberOfRadioButtons = 4, labelArray4 = ['One', 'Two', 'Three', 'Four'], ct2 = ('left', 'left'), cw5 = [60,60,60,60,60]) self.thumbCheckBox = cmds.checkBoxGrp(l = 'Thumb: ', v1 = True) cmds.text('\n', h = 5) spineJUI = cmds.frameLayout(l = 'C: Number of Spine Joints') cmds.text('\n', h = 5) self.spineButtonGrp = cmds.radioButtonGrp('spineRadio', p = spineJUI, l = 'Spine Joints: ', sl = 2, w = 1, numberOfRadioButtons = 3, labelArray3 = ['Three', 'Five', 'Ten'], ct2 = ('left', 'left'), cw4 = [95,95,95,95]) cmds.text('\n', h = 5) neckJUI = cmds.frameLayout(l = 'D: Number of Neck Joints') cmds.text('\n', h = 5) self.neckButtonGrp = cmds.radioButtonGrp('neckRadio', p = neckJUI, l = 'Neck Joints: ', sl = 0, w = 1, numberOfRadioButtons = 3, labelArray3 = ['Two', 'Three', 'Four'], ct2 = ('left', 'left'), cw4 = [95,95,95,95]) cmds.text('\n', h = 5) cmds.setParent('..') cmds.setParent('..') cmds.setParent('..') cmds.frameLayout('E: Creation') cmds.text('SAVE FIRST: CAN NOT UNDO', bgc = (0.2,0.2,0.2)) cmds.button(l = '\nCreate Dummy Skeleton\n', c = self.build) # also have it make char name field grey cmds.text('Elbows and Knees must have bend.', bgc = (0.2,0.2,0.2)) cmds.columnLayout() cmds.setParent('..') cmds.setParent('..') cmds.setParent('..') cmds.setParent('..') tab2 = cmds.rowColumnLayout('Skeleton') cmds.columnLayout(columnAttach = ('both', 5), rowSpacing = 10, columnWidth = 150) cmds.setParent('..') cmds.frameLayout(l = 'A: Skeleton Setup') cmds.text('SAVE FIRST: CAN NOT UNDO', bgc = (0.2,0.2,0.2)) cmds.button(l = '\nConvert to Skeleton - Orient - Set LRA\n', c = self.buildSkel) self.gScaleUI = cmds.textFieldGrp (label="Scale Multiplier:", ann="Scale multipler of Character: basis for all further base controllers", tx = '1.0', w = 1, ed = False, en = False, visible = True) cmds.frameLayout('B: Manual Orientation') cmds.text('You must manually check finger, thumb, leg, foot orientation specifically.\nConfirm rest of joints.\nSpine: X aim, Y point backwards from spine, Z to the side.\nFingers: X is aim, Y points upwards, Z to the side - Spread on Y, curl on Z.\nFoot: Pivots on Y, rolls on Z, leans on X.') cmds.columnLayout() cmds.setParent('..') cmds.frameLayout('C: Finalize Creation of Skeleton') cmds.button(l = '\nFinalize Skeleton\n', c = self.finishS) cmds.setParent('..') cmds.setParent('..') cmds.setParent('..') cmds.setParent('..') tab3 = cmds.rowColumnLayout('Legs') cmds.columnLayout(columnAttach = ('both', 5), rowSpacing = 10, columnWidth = 150) cmds.setParent('..') cmds.frameLayout(l = 'A: Leg Rig Setup') cmds.button(l = '\nGenerate Foot Lean Locators\n', c = self.makeLean) cmds.text('Place on either side of the foot.\nDo not rotate: Automatic orientation in place.') cmds.frameLayout(l = 'B: Rig Legs') cmds.button(l = '\nRig Legs\n', c = self.makeLegs) cmds.setParent('..') cmds.setParent('..') cmds.setParent('..') tab4 = cmds.rowColumnLayout('Arms') cmds.columnLayout(columnAttach = ('both', 5), rowSpacing = 10, columnWidth = 150) cmds.setParent('..') cmds.frameLayout(l = 'A: Arm Rig Setup') cmds.button(l = '\nA: Rig Arms\n', c = self.makeArms) cmds.setParent('..') cmds.setParent('..') tab5 = cmds.rowColumnLayout('Spine and Head') cmds.columnLayout(columnAttach = ('both', 5), rowSpacing = 10, columnWidth = 150) cmds.setParent('..') cmds.frameLayout(l = 'Spine Rig Setup') cmds.setParent('..') cmds.setParent('..') tab6 = cmds.rowColumnLayout('Stretchy IK') cmds.columnLayout(columnAttach = ('both', 5), rowSpacing = 10, columnWidth = 150) cmds.setParent('..') cmds.frameLayout(l = 'Stretchy Setup') cmds.setParent('..') cmds.setParent('..') tab6 = cmds.rowColumnLayout('Extras') cmds.scrollLayout(saw = 600, sah = 600, cr = True) cmds.columnLayout(columnAttach = ('both', 5), rowSpacing = 10, columnWidth = 150) cmds.setParent('..') cmds.frameLayout(l = 'General Utitlities') cmds.text('\nHere are all my general utilities for various things') cmds.frameLayout(l = 'Automatic Blend Colors Creation and Connection') cmds.rowColumnLayout(nc = 5, w = 10) cmds.text('IK Joint:') cmds.text(l = '') cmds.text('FK/Dyn Joint:') cmds.text(l = '') cmds.text('Bind Joint:') self.ikJBC = cmds.textField() cmds.text(l = '') self.fkJBC = cmds.textField() cmds.text(l = '') self.bindJBC = cmds.textField() cmds.text(' \nBlend Quantity:') cmds.text(l = '') cmds.text(' \nSwitch Control:') cmds.text(l = '') cmds.text(l = '') self.bQBC = cmds.intField() cmds.text(l = '') self.sCBC = cmds.textField() cmds.text(l = '') cmds.button(l = 'Help Docs', c = self.crBC.__doc__) cmds.setParent('..') cmds.button(l = 'Create', c = self.execCrBC) cmds.text(l = '') cmds.setParent('..') cmds.frameLayout(l = 'Make Spline IK Curve Stretch And Squash') cmds.rowColumnLayout(nc = 5, w = 10) cmds.text('Curve Name:') cmds.text(l = '') cmds.text('Setup Name:') cmds.text(l = '') cmds.text('Joint Quantity:') self.ikJBC = cmds.textField() cmds.text(l = '') self.fkJBC = cmds.textField() cmds.text(l = '') self.bindJBC = cmds.textField() cmds.text(' \nSwitch Control:') cmds.text(l = '') cmds.text(' \nGlobal Control:') cmds.text(l = '') cmds.text(l = '') self.bQBC = cmds.intField() cmds.text(l = '') self.sCBC = cmds.textField() cmds.text(l = '') cmds.button(l = 'Help Docs', c = self.crBC.__doc__) cmds.setParent('..') cmds.button(l = 'Create', c = self.execCrBC) cmds.setParent('..') cmds.showWindow(rigUI) r = RUI('charNameUI', 'gScaleUI', 'fingButtonGrp', 'thumbCheckBox', 'spineButtonGrp', 'neckButtonGrp', 'ikJBC', 'fkJBC', 'bindJBC', 'bQBC', 'sCBC') # last modified at 6.20 pm 29th June 2011

Read the article

Drupal Ctools Form Wizard in a Block

- by Iamjon

Hi everyone I created a custom module that has a Ctools multi step form. It's basically a copy of http://www.nicklewis.org/using-chaos-tools-form-wizard-build-multistep-forms-drupal-6. The form works. I can see it if I got to the url i made for it. For the life of me I can't get the multistep form to show up in a block. Any clues? /** * Implementation of hook_block() * */ function mycrazymodule_block($op='list', $delta=0, $edit=array()) { switch ($op) { case 'list': $blocks[0]['info'] = t('SFT Getting Started'); $blocks[1]['info'] = t('SFT Contact US'); $blocks[2]['info'] = t('SFT News Letter'); return $blocks; case 'view': switch ($delta){ case '0': $block['subject'] = t('SFT Getting Started Subject'); $block['content'] = mycrazymodule_wizard(); break; case '1': $block['subject'] = t('SFT Contact US Subject'); $block['content'] = t('SFT Contact US content'); break; case '2': $block['subject'] = t('SFT News Letter Subject'); $block['content'] = t('SFT News Letter cONTENT'); break; } return $block; } } /** * Implementation of hook_menu(). */ function mycrazymodule_menu() { $items['hellocowboy'] = array( 'title' = 'Two Step Form', 'page callback' = 'mycrazymodule_wizard', 'access arguments' = array('access content') ); return $items; } /** * menu callback for the multistep form * step is whatever arg one is -- and will refer to the keys listed in * $form_info['order'], and $form_info['forms'] arrays */ function mycrazymodule_wizard() { $step = arg(1); // required includes for wizard $form_state = array(); ctools_include('wizard'); ctools_include('object-cache'); // The array that will hold the two forms and their options $form_info = array( 'id' = 'getting_started', 'path' = "hellocowboy/%step", 'show trail' = FALSE, 'show back' = FALSE, 'show cancel' = false, 'show return' =false, 'next text' = 'Submit', 'next callback' = 'getting_started_add_subtask_next', 'finish callback' = 'getting_started_add_subtask_finish', 'return callback' = 'getting_started_add_subtask_finish', 'order' = array( 'basic' = t('Step 1: Basic Info'), 'lecture' = t('Step 2: Choose Lecture'), ), 'forms' = array( 'basic' = array( 'form id' = 'basic_info_form' ), 'lecture' = array( 'form id' = 'choose_lecture_form' ), ), ); $form_state = array( 'cache name' = NULL, ); // no matter the step, you will load your values from the callback page $getstart = getting_started_get_page_cache(NULL); if (!$getstart) { // set form to first step -- we have no data $step = current(array_keys($form_info['order'])); $getstart = new stdClass(); //create cache ctools_object_cache_set('getting_started', $form_state['cache name'], $getstart); //print_r($getstart); } //THIS IS WHERE WILL STORE ALL FORM DATA $form_state['getting_started_obj'] = $getstart; // and this is the witchcraft that makes it work $output = ctools_wizard_multistep_form($form_info, $step, $form_state); return $output; } function basic_info_form(&$form, &$form_state){ $getstart = &$form_state['getting_started_obj']; $form['firstname'] = array( '#weight' = '0', '#type' = 'textfield', '#title' = t('firstname'), '#size' = 60, '#maxlength' = 255, '#required' = TRUE, ); $form['lastname'] = array( '#weight' = '1', '#type' = 'textfield', '#title' = t('lastname'), '#required' = TRUE, '#size' = 60, '#maxlength' = 255, ); $form['phone'] = array( '#weight' = '2', '#type' = 'textfield', '#title' = t('phone'), '#required' = TRUE, '#size' = 60, '#maxlength' = 255, ); $form['email'] = array( '#weight' = '3', '#type' = 'textfield', '#title' = t('email'), '#required' = TRUE, '#size' = 60, '#maxlength' = 255, ); $form['newsletter'] = array( '#weight' = '4', '#type' = 'checkbox', '#title' = t('I would like to receive the newsletter'), '#required' = TRUE, '#return_value' = 1, '#default_value' = 1, ); $form_state['no buttons'] = TRUE; } function basic_info_form_validate(&$form, &$form_state){ $email = $form_state['values']['email']; $phone = $form_state['values']['phone']; if(valid_email_address($email) != TRUE){ form_set_error('Dude you have an error', t('Where is your email?')); } //if (strlen($phone) 0 && !ereg('^[0-9]{1,3}-[0-9]{3}-[0-9]{3,4}-[0-9]{3,4}$', $phone)) { //form_set_error('Dude the phone', t('Phone number must be in format xxx-xxx-nnnn-nnnn.')); //} } function basic_info_form_submit(&$form, &$form_state){ //Grab the variables $firstname =check_plain ($form_state['values']['firstname']); $lastname = check_plain ($form_state['values']['lastname']); $email = check_plain ($form_state['values']['email']); $phone = check_plain ($form_state['values']['phone']); $newsletter = $form_state['values']['newsletter']; //Send the form and Grab the lead id $leadid = send_first_form($lastname, $firstname, $email,$phone, $newsletter); //Put into form $form_state['getting_started_obj']-firstname = $firstname; $form_state['getting_started_obj']-lastname = $lastname; $form_state['getting_started_obj']-email = $email; $form_state['getting_started_obj']-phone = $phone; $form_state['getting_started_obj']-newsletter = $newsletter; $form_state['getting_started_obj']-leadid = $leadid; } function choose_lecture_form(&$form, &$form_state){ $one = 'event 1' $two = 'event 2' $three = 'event 3' $getstart = &$form_state['getting_started_obj']; $form['lecture'] = array( '#weight' = '5', '#default_value' = 'two', '#options' = array( 'one' = $one, 'two' = $two, 'three' = $three, ), '#type' = 'radios', '#title' = t('Select Workshop'), '#required' = TRUE, ); $form['attendees'] = array( '#weight' = '6', '#default_value' = 'one', '#options' = array( 'one' = t('I will be arriving alone'), 'two' =t('I will be arriving with a guest'), ), '#type' = 'radios', '#title' = t('Attendees'), '#required' = TRUE, ); $form_state['no buttons'] = TRUE; } /** * Same idea as previous steps submit * */ function choose_lecture_form_submit(&$form, &$form_state) { $workshop = $form_state['values']['lecture']; $leadid = $form_state['getting_started_obj']-leadid; $attendees = $form_state['values']['attendees']; $form_state['getting_started_obj']-lecture = $workshop; $form_state['getting_started_obj']-attendees = $attendees; send_second_form($workshop, $attendees, $leadid); } /*----PART 3 CTOOLS CALLBACKS -- these usually don't have to be very unique ---------------------- */ /** * Callback generated when the add page process is finished. * this is where you'd normally save. */ function getting_started_add_subtask_finish(&$form_state) { dpm($form_state); $getstart = &$form_state['getting_started_obj']; drupal_set_message('mycrazymodule '.$getstart-name.' successfully deployed' ); //Get id // Clear the cache ctools_object_cache_clear('getting_started', $form_state['cache name']); $form_state['redirect'] = 'hellocowboy'; } /** * Callback for the proceed step * */ function getting_started_add_subtask_next(&$form_state) { dpm($form_state); $getstart = &$form_state['getting_started_obj']; $cache = ctools_object_cache_set('getting_started', $form_state['cache name'], $getstart); } /*----PART 4 CTOOLS FORM STORAGE HANDLERS -- these usually don't have to be very unique ---------------------- */ /** * Remove an item from the object cache. */ function getting_started_clear_page_cache($name) { ctools_object_cache_clear('getting_started', $name); } /** * Get the cached changes to a given task handler. */ function getting_started_get_page_cache($name) { $cache = ctools_object_cache_get('getting_started', $name); return $cache; } //Salesforce Functions function send_first_form($lastname, $firstname,$email,$phone, $newsletter){ $send = array("LastName" = $lastname , "FirstName" = $firstname, "Email" = $email ,"Phone" = $phone , "Newsletter__c" =$newsletter ); $sf = salesforce_api_connect(); $response = $sf-client-create(array($send), 'Lead'); dpm($response); return $response-id; } function send_second_form($workshop, $attendees, $leadid){ $send = array("Id" = $leadid , "Number_Of_Pepole__c" = "2" ); $sf = salesforce_api_connect(); $response = $sf-client-update(array($send), 'Lead'); dpm($response, 'the final response'); return $response-id; }

Read the article

InputDispatcher Error

- by StarDust

INFO/ActivityManager(68): Process com.example (pid 390) has died. ERROR/InputDispatcher(68): channel '406ed580 com.example/com.example.afeTest (server)' ~ Consumer closed input channel or an error occurred. events=0x8 ERROR/InputDispatcher(68): channel '406ed580 com.example/com.example.afeTest (server)' ~ Channel is unrecoverably broken and will be disposed! ERROR/InputDispatcher(68): Received spurious receive callback for unknown input channel. fd=165, events=0x8 Can anyone tell what may be the reason behind this error? I've ported a native code on the Android-ndk. One thing I noticed regarding fd (that may be some reason :S) My code uses fd_sets which was defined in winsock2.h But I didn't find fd_sets defined in android-ndk. So I had included "select.h" where fd_set is a typedef in the android-ndk: typedef __kernel_fd_set fd_set; Here is the log cat: 04-06 11:15:32.405: INFO/DEBUG(31): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** 04-06 11:15:32.405: INFO/DEBUG(31): Build fingerprint: 'generic/sdk/generic:2.3.3/GRI34/101070:eng/test-keys' 04-06 11:15:32.415: INFO/DEBUG(31): pid: 335, tid: 348 >>> com.example <<< 04-06 11:15:32.426: INFO/DEBUG(31): signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr deadbaad 04-06 11:15:32.426: INFO/DEBUG(31): r0 deadbaad r1 0000000c r2 00000027 r3 00000000 04-06 11:15:32.445: INFO/DEBUG(31): r4 00000080 r5 afd46668 r6 0000a000 r7 00000078 04-06 11:15:32.445: INFO/DEBUG(31): r8 804ab00d r9 002a9778 10 00100000 fp 00000001 04-06 11:15:32.445: INFO/DEBUG(31): ip ffffffff sp 44295d10 lr afd19375 pc afd15ef0 cpsr 00000030 04-06 11:15:32.756: INFO/DEBUG(31): #00 pc 00015ef0 /system/lib/libc.so 04-06 11:15:32.756: INFO/DEBUG(31): #01 pc 00013852 /system/lib/libc.so 04-06 11:15:32.767: INFO/DEBUG(31): code around pc: 04-06 11:15:32.785: INFO/DEBUG(31): afd15ed0 68241c23 d1fb2c00 68dae027 d0042a00 04-06 11:15:32.785: INFO/DEBUG(31): afd15ee0 20014d18 6028447d 48174790 24802227 04-06 11:15:32.785: INFO/DEBUG(31): afd15ef0 f7f57002 2106eb56 ec92f7f6 0563aa01 04-06 11:15:32.796: INFO/DEBUG(31): afd15f00 60932100 91016051 1c112006 e818f7f6 04-06 11:15:32.807: INFO/DEBUG(31): afd15f10 2200a905 f7f62002 f7f5e824 2106eb42 04-06 11:15:32.815: INFO/DEBUG(31): code around lr: 04-06 11:15:32.815: INFO/DEBUG(31): afd19354 b0834a0d 589c447b 26009001 686768a5 04-06 11:15:32.825: INFO/DEBUG(31): afd19364 220ce008 2b005eab 1c28d003 47889901 04-06 11:15:32.836: INFO/DEBUG(31): afd19374 35544306 d5f43f01 2c006824 b003d1ee 04-06 11:15:32.836: INFO/DEBUG(31): afd19384 bdf01c30 000281a8 ffffff88 1c0fb5f0 04-06 11:15:32.846: INFO/DEBUG(31): afd19394 43551c3d a904b087 1c16ac01 604d9004 04-06 11:15:32.856: INFO/DEBUG(31): stack: 04-06 11:15:32.856: INFO/DEBUG(31): 44295cd0 00000408 04-06 11:15:32.867: INFO/DEBUG(31): 44295cd4 afd18407 /system/lib/libc.so 04-06 11:15:32.875: INFO/DEBUG(31): 44295cd8 afd4270c /system/lib/libc.so 04-06 11:15:32.875: INFO/DEBUG(31): 44295cdc afd426b8 /system/lib/libc.so 04-06 11:15:32.885: INFO/DEBUG(31): 44295ce0 00000000 04-06 11:15:32.896: INFO/DEBUG(31): 44295ce4 afd19375 /system/lib/libc.so 04-06 11:15:32.896: INFO/DEBUG(31): 44295ce8 804ab00d /data/data/com.example/lib/libAFE.so 04-06 11:15:32.896: INFO/DEBUG(31): 44295cec afd183d9 /system/lib/libc.so 04-06 11:15:32.906: INFO/DEBUG(31): 44295cf0 00000078 04-06 11:15:32.906: INFO/DEBUG(31): 44295cf4 00000000 04-06 11:15:32.906: INFO/DEBUG(31): 44295cf8 afd46668 04-06 11:15:32.906: INFO/DEBUG(31): 44295cfc 0000a000 [heap] 04-06 11:15:32.916: INFO/DEBUG(31): 44295d00 00000078 04-06 11:15:32.927: INFO/DEBUG(31): 44295d04 afd18677 /system/lib/libc.so 04-06 11:15:32.927: INFO/DEBUG(31): 44295d08 df002777 04-06 11:15:32.945: INFO/DEBUG(31): 44295d0c e3a070ad 04-06 11:15:32.945: INFO/DEBUG(31): #00 44295d10 002c43a0 [heap] 04-06 11:15:32.945: INFO/DEBUG(31): 44295d14 002a9900 [heap] 04-06 11:15:32.956: INFO/DEBUG(31): 44295d18 afd46608 04-06 11:15:32.966: INFO/DEBUG(31): 44295d1c afd11010 /system/lib/libc.so 04-06 11:15:32.976: INFO/DEBUG(31): 44295d20 002c4298 [heap] 04-06 11:15:32.976: INFO/DEBUG(31): 44295d24 fffffbdf 04-06 11:15:33.006: INFO/DEBUG(31): 44295d28 000000da 04-06 11:15:33.006: INFO/DEBUG(31): 44295d2c afd46450 04-06 11:15:33.006: INFO/DEBUG(31): 44295d30 000001b4 04-06 11:15:33.026: INFO/DEBUG(31): 44295d34 afd13857 /system/lib/libc.so 04-06 11:15:33.026: INFO/DEBUG(31): #01 44295d38 afd46450 04-06 11:15:33.035: INFO/DEBUG(31): 44295d3c afd13857 /system/lib/libc.so 04-06 11:15:33.056: INFO/DEBUG(31): 44295d40 804ab00d /data/data/com.example/lib/libAFE.so 04-06 11:15:33.056: INFO/DEBUG(31): 44295d44 44295e8c 04-06 11:15:33.056: INFO/DEBUG(31): 44295d48 804ab00d /data/data/com.example/lib/libAFE.so 04-06 11:15:33.056: INFO/DEBUG(31): 44295d4c 804bfec3 /data/data/com.example/lib/libAFE.so 04-06 11:15:33.056: INFO/DEBUG(31): 44295d50 002c43a0 [heap] 04-06 11:15:33.066: INFO/DEBUG(31): 44295d54 44295e8c 04-06 11:15:33.066: INFO/DEBUG(31): 44295d58 804ab00d /data/data/com.example/lib/libAFE.so 04-06 11:15:33.076: INFO/DEBUG(31): 44295d5c 002a9778 [heap] 04-06 11:15:33.085: INFO/DEBUG(31): 44295d60 00000078 04-06 11:15:33.085: INFO/DEBUG(31): 44295d64 afd14769 /system/lib/libc.so 04-06 11:15:33.085: INFO/DEBUG(31): 44295d68 44295e8c 04-06 11:15:33.085: INFO/DEBUG(31): 44295d6c 805d9763 /data/data/com.example/lib/libAFE.so 04-06 11:15:33.085: INFO/DEBUG(31): 44295d70 44295e8c 04-06 11:15:33.085: INFO/DEBUG(31): 44295d74 8051dc35 /data/data/com.example/lib/libAFE.so 04-06 11:15:33.085: INFO/DEBUG(31): 44295d78 0000003a 04-06 11:15:33.085: INFO/DEBUG(31): 44295d7c 002a9900 [heap] 04-06 11:15:37.126: DEBUG/Zygote(33): Process 335 terminated by signal (11) 04-06 11:15:37.146: INFO/ActivityManager(68): Process com.example (pid 335) has died. 04-06 11:15:37.178: ERROR/InputDispatcher(68): channel '406f03a0 com.example/com.example.afeTest (server)' ~ Consumer closed input channel or an error occurred. events=0x8 04-06 11:15:37.178: ERROR/InputDispatcher(68): channel '406f03a0 com.example/com.example.afeTest (server)' ~ Channel is unrecoverably broken and will be disposed! 04-06 11:15:37.185: INFO/BootReceiver(68): Copying /data/tombstones/tombstone_09 to DropBox (SYSTEM_TOMBSTONE) 04-06 11:15:37.576: DEBUG/dalvikvm(68): GC_FOR_MALLOC freed 266K, 47% free 4404K/8199K, external 3520K/3903K, paused 306ms 04-06 11:15:37.835: DEBUG/dalvikvm(68): GC_FOR_MALLOC freed 203K, 47% free 4457K/8391K, external 3520K/3903K, paused 120ms 04-06 11:15:37.886: INFO/WindowManager(68): WIN DEATH: Window{406f03a0 com.example/com.example.afeTest paused=false} 04-06 11:15:38.095: DEBUG/dalvikvm(68): GC_FOR_MALLOC freed 67K, 47% free 4518K/8391K, external 3511K/3903K, paused 94ms 04-06 11:15:38.095: INFO/dalvikvm-heap(68): Grow heap (frag case) to 10.575MB for 196628-byte allocation 04-06 11:15:38.126: DEBUG/dalvikvm(126): GC_EXPLICIT freed 110K, 51% free 2903K/5895K, external 4701K/5293K, paused 2443ms 04-06 11:15:38.217: DEBUG/dalvikvm(68): GC_FOR_MALLOC freed 1K, 46% free 4708K/8647K, external 3511K/3903K, paused 96ms 04-06 11:15:38.225: INFO/WindowManager(68): WIN DEATH: Window{406f72f8 com.example/com.example.afeTest paused=false} 04-06 11:15:38.405: DEBUG/dalvikvm(68): GC_FOR_MALLOC freed 492K, 50% free 4345K/8647K, external 3511K/3903K, paused 96ms 04-06 11:15:38.485: ERROR/InputDispatcher(68): Received spurious receive callback for unknown input channel. fd=164, events=0x8

Read the article

INDY WebService over SSL contains link with HTTP protocol instead of HTTPS in WSDL

- by user1437820

When creating new SOAP WebService server project using Delphi XE2 the wizard allows to set change port and HTTPS properties. Port is set to 443, HTTPS flag is checked, but when trying to connect to created server it returns incorrect transport (HTTP instead of HTTPS) in WSDL and generates HTTP links on the Service Info Page. The auto-generated page is not so important, but wrong information in WSDL file is a problem. Below you can see the returned WSDL - there are no HTTPS: <?xml version="1.0"?> <definitions xmlns="http://schemas.xmlsoap.org/wsdl/" xmlns:xs="http://www.w3.org/2001/XMLSchema" name="Itest123service" targetNamespace="http://tempuri.org/" xmlns:tns="http://tempuri.org/" xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:mime="http://schemas.xmlsoap.org/wsdl/mime/" xmlns:ns1="urn:test123Intf"> <types> <schema xmlns="http://www.w3.org/2001/XMLSchema" targetNamespace="urn:test123Intf"> <simpleType name="TEnumTest"> <restriction base="string"> <enumeration value="etNone"/> <enumeration value="etAFew"/> <enumeration value="etSome"/> <enumeration value="etAlot"/> </restriction> </simpleType> <complexType name="TDoubleArray"> <complexContent> <restriction base="soapenc:Array"> <sequence/> <attribute ref="soapenc:arrayType" n1:arrayType="xs:double[]" xmlns:n1="http://schemas.xmlsoap.org/wsdl/"/> </restriction> </complexContent> </complexType> <complexType name="TMyEmployee"> <sequence> <element name="LastName" type="xs:string"/> <element name="FirstName" type="xs:string"/> <element name="Salary" type="xs:double"/> </sequence> </complexType> </schema> </types> <message name="echoEnum0Request"> <part name="Value" type="ns1:TEnumTest"/> </message> <message name="echoEnum0Response"> <part name="return" type="ns1:TEnumTest"/> </message> <message name="echoDoubleArray1Request"> <part name="Value" type="ns1:TDoubleArray"/> </message> <message name="echoDoubleArray1Response"> <part name="return" type="ns1:TDoubleArray"/> </message> <message name="echoMyEmployee2Request"> <part name="Value" type="ns1:TMyEmployee"/> </message> <message name="echoMyEmployee2Response"> <part name="return" type="ns1:TMyEmployee"/> </message> <message name="echoDouble3Request"> <part name="Value" type="xs:double"/> </message> <message name="echoDouble3Response"> <part name="return" type="xs:double"/> </message> <portType name="Itest123"> <operation name="echoEnum"> <input message="tns:echoEnum0Request"/> <output message="tns:echoEnum0Response"/> </operation> <operation name="echoDoubleArray"> <input message="tns:echoDoubleArray1Request"/> <output message="tns:echoDoubleArray1Response"/> </operation> <operation name="echoMyEmployee"> <input message="tns:echoMyEmployee2Request"/> <output message="tns:echoMyEmployee2Response"/> </operation> <operation name="echoDouble"> <input message="tns:echoDouble3Request"/> <output message="tns:echoDouble3Response"/> </operation> </portType> <binding name="Itest123binding" type="tns:Itest123"> <binding xmlns="http://schemas.xmlsoap.org/wsdl/soap/" style="rpc" transport="http://schemas.xmlsoap.org/soap/http"/> <operation name="echoEnum"> <operation xmlns="http://schemas.xmlsoap.org/wsdl/soap/" soapAction="urn:test123Intf-Itest123#echoEnum" style="rpc"/> <input> <body xmlns="http://schemas.xmlsoap.org/wsdl/soap/" use="encoded" encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" namespace="urn:test123Intf-Itest123"/> </input> <output> <body xmlns="http://schemas.xmlsoap.org/wsdl/soap/" use="encoded" encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" namespace="urn:test123Intf-Itest123"/> </output> </operation> <operation name="echoDoubleArray"> <operation xmlns="http://schemas.xmlsoap.org/wsdl/soap/" soapAction="urn:test123Intf-Itest123#echoDoubleArray" style="rpc"/> <input> <body xmlns="http://schemas.xmlsoap.org/wsdl/soap/" use="encoded" encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" namespace="urn:test123Intf-Itest123"/> </input> <output> <body xmlns="http://schemas.xmlsoap.org/wsdl/soap/" use="encoded" encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" namespace="urn:test123Intf-Itest123"/> </output> </operation> <operation name="echoMyEmployee"> <operation xmlns="http://schemas.xmlsoap.org/wsdl/soap/" soapAction="urn:test123Intf-Itest123#echoMyEmployee" style="rpc"/> <input> <body xmlns="http://schemas.xmlsoap.org/wsdl/soap/" use="encoded" encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" namespace="urn:test123Intf-Itest123"/> </input> <output> <body xmlns="http://schemas.xmlsoap.org/wsdl/soap/" use="encoded" encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" namespace="urn:test123Intf-Itest123"/> </output> </operation> <operation name="echoDouble"> <operation xmlns="http://schemas.xmlsoap.org/wsdl/soap/" soapAction="urn:test123Intf-Itest123#echoDouble" style="rpc"/> <input> <body xmlns="http://schemas.xmlsoap.org/wsdl/soap/" use="encoded" encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" namespace="urn:test123Intf-Itest123"/> </input> <output> <body xmlns="http://schemas.xmlsoap.org/wsdl/soap/" use="encoded" encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" namespace="urn:test123Intf-Itest123"/> </output> </operation> </binding> <service name="Itest123service"> <port name="Itest123Port" binding="tns:Itest123binding"> <address xmlns="http://schemas.xmlsoap.org/wsdl/soap/" location="http://localhost:443/soap/Itest123"/> </port> </service> </definitions> When I'm trying to import WSDL to soapUI tool to check WebService work I need to change manually binding link to "https://" and only then RPCs will work. I will be very grateful for any idea how to force INDY to return links in WSDL with HTTPS protocol. Thanks in advance!

Read the article

Why is phpseclib producing incompatible certs?

- by chacham15

Why is it that when I try to use a certificate/key pair generated from phpseclib, the OpenSSL server code errors out? Certs/Keys generated from OpenSSL work fine. How do I fix this? Certificate/Key Generation taken straight from phpseclib documentation: <?php include('File/X509.php'); include('Crypt/RSA.php'); // create private key / x.509 cert for stunnel / website $privKey = new Crypt_RSA(); extract($privKey-createKey()); $privKey-loadKey($privatekey); $pubKey = new Crypt_RSA(); $pubKey-loadKey($publickey); $pubKey-setPublicKey(); $subject = new File_X509(); $subject-setDNProp('id-at-organizationName', 'phpseclib demo cert'); //$subject-removeDNProp('id-at-organizationName'); $subject-setPublicKey($pubKey); $issuer = new File_X509(); $issuer-setPrivateKey($privKey); $issuer-setDN($subject-getDN()); $x509 = new File_X509(); //$x509-setStartDate('-1 month'); // default: now //$x509-setEndDate('+1 year'); // default: +1 year $result = $x509-sign($issuer, $subject); echo "the stunnel.pem contents are as follows:\r\n\r\n"; echo $privKey-getPrivateKey(); echo "\r\n"; echo $x509-saveX509($result); echo "\r\n"; ? OpenSSL sample SSL server taken straight from OpenSSL example code: #include <stdio.h #include <unistd.h #include <stdlib.h #include <memory.h #include <errno.h #include <sys/types.h #include <sys/socket.h #include <netinet/in.h #include <arpa/inet.h #include <netdb.h #include <openssl/rsa.h /* SSLeay stuff */ #include <openssl/crypto.h #include <openssl/x509.h #include <openssl/pem.h #include <openssl/ssl.h #include <openssl/err.h #define CHK_NULL(x) if ((x)==NULL) exit (1) #define CHK_ERR(err,s) if ((err)==-1) { perror(s); exit(1); } #define CHK_SSL(err) if ((err)==-1) { ERR_print_errors_fp(stderr); exit(2); } int main (int argc, char *argv[]) { int err; int listen_sd; int sd; struct sockaddr_in sa_serv; struct sockaddr_in sa_cli; size_t client_len; SSL_CTX* ctx; SSL* ssl; X509* client_cert; char* str; char buf [4096]; SSL_METHOD *meth; /* SSL preliminaries. We keep the certificate and key with the context. */ SSL_load_error_strings(); SSLeay_add_ssl_algorithms(); meth = SSLv23_server_method(); ctx = SSL_CTX_new (meth); if (!ctx) { ERR_print_errors_fp(stderr); exit(2); } if (SSL_CTX_use_certificate_file(ctx, argv[1], SSL_FILETYPE_PEM) <= 0) { ERR_print_errors_fp(stderr); exit(3); } if (SSL_CTX_use_PrivateKey_file(ctx, argv[2], SSL_FILETYPE_PEM) <= 0) { ERR_print_errors_fp(stderr); exit(4); } if (!SSL_CTX_check_private_key(ctx)) { fprintf(stderr,"Private key does not match the certificate public key\n"); exit(5); } /* ----------------------------------------------- */ /* Prepare TCP socket for receiving connections */ listen_sd = socket (AF_INET, SOCK_STREAM, 0); CHK_ERR(listen_sd, "socket"); memset (&sa_serv, '\0', sizeof(sa_serv)); sa_serv.sin_family = AF_INET; sa_serv.sin_addr.s_addr = INADDR_ANY; sa_serv.sin_port = htons (1111); /* Server Port number */ err = bind(listen_sd, (struct sockaddr*) &sa_serv, sizeof (sa_serv)); CHK_ERR(err, "bind"); /* Receive a TCP connection. */ err = listen (listen_sd, 5); CHK_ERR(err, "listen"); client_len = sizeof(sa_cli); sd = accept (listen_sd, (struct sockaddr*) &sa_cli, (unsigned int*)&client_len); CHK_ERR(sd, "accept"); close (listen_sd); printf ("Connection from %lx, port %x\n", sa_cli.sin_addr.s_addr, sa_cli.sin_port); /* ----------------------------------------------- */ /* TCP connection is ready. Do server side SSL. */ ssl = SSL_new (ctx); CHK_NULL(ssl); SSL_set_fd (ssl, sd); err = SSL_accept (ssl); CHK_SSL(err); /* Get the cipher - opt */ printf ("SSL connection using %s\n", SSL_get_cipher (ssl)); /* Get client's certificate (note: beware of dynamic allocation) - opt */ client_cert = SSL_get_peer_certificate (ssl); if (client_cert != NULL) { printf ("Client certificate:\n"); str = X509_NAME_oneline (X509_get_subject_name (client_cert), 0, 0); CHK_NULL(str); printf ("\t subject: %s\n", str); OPENSSL_free (str); str = X509_NAME_oneline (X509_get_issuer_name (client_cert), 0, 0); CHK_NULL(str); printf ("\t issuer: %s\n", str); OPENSSL_free (str); /* We could do all sorts of certificate verification stuff here before deallocating the certificate. */ X509_free (client_cert); } else printf ("Client does not have certificate.\n"); /* DATA EXCHANGE - Receive message and send reply. */ err = SSL_read (ssl, buf, sizeof(buf) - 1); CHK_SSL(err); buf[err] = '\0'; printf ("Got %d chars:'%s'\n", err, buf); err = SSL_write (ssl, "I hear you.", strlen("I hear you.")); CHK_SSL(err); /* Clean up. */ close (sd); SSL_free (ssl); SSL_CTX_free (ctx); return 1; } /* EOF - serv.cpp */ This program errors with: (the error is printed out on the call to SSL_write) Connection from 100007f, port a7ff SSL connection using (NONE) Client does not have certificate. Got 0 chars:'' 82673:error:1409E0E5:SSL routines:SSL3_WRITE_BYTES:ssl handshake failure:/SourceCache/OpenSSL098/OpenSSL098-44/src/ssl/s3_pkt.c:539: Here is the relevant code referenced by the error: int ssl3_write_bytes(SSL *s, int type, const void *buf_, int len) { const unsigned char *buf=buf_; unsigned int tot,n,nw; int i; s-rwstate=SSL_NOTHING; tot=s-s3-wnum; s-s3-wnum=0; if (SSL_in_init(s) && !s-in_handshake) { i=s-handshake_func(s); if (i < 0) return(i); if (i == 0) { SSLerr(SSL_F_SSL3_WRITE_BYTES,SSL_R_SSL_HANDSHAKE_FAILURE); return -1; } } ...etc

Read the article

Uploading multiple files using Spring MVC 3.0.2 after HiddenHttpMethodFilter has been enabled

- by Tiny

I'm using Spring version 3.0.2. I need to upload multiple files using the multiple="multiple" attribute of a file browser such as, <input type="file" id="myFile" name="myFile" multiple="multiple"/> (and not using multiple file browsers something like the one stated by this answer, it indeed works I tried). Although no versions of Internet Explorer supports this approach unless an appropriate jQuery plugin/widget is used, I don't care about it right now (since most other browsers support this). This works fine with commons fileupload but in addition to using RequestMethod.POST and RequestMethod.GET methods, I also want to use other request methods supported and suggested by Spring like RequestMethod.PUT and RequestMethod.DELETE in their own appropriate places. For this to be so, I have configured Spring with HiddenHttpMethodFilter which goes fine as this question indicates. but it can upload only one file at a time even though multiple files in the file browser are chosen. In the Spring controller class, a method is mapped as follows. @RequestMapping(method={RequestMethod.POST}, value={"admin_side/Temp"}) public String onSubmit(@RequestParam("myFile") List<MultipartFile> files, @ModelAttribute("tempBean") TempBean tempBean, BindingResult error, Map model, HttpServletRequest request, HttpServletResponse response) throws IOException, FileUploadException { for(MultipartFile file:files) { System.out.println(file.getOriginalFilename()); } } Even with the request parameter @RequestParam("myFile") List<MultipartFile> files which is a List of type MultipartFile (it can always have only one file at a time). I could find a strategy which is likely to work with multiple files on this blog. I have gone through it carefully. The solution below the section SOLUTION 2 – USE THE RAW REQUEST says, If however the client insists on using the same form input name such as ‘files[]‘ or ‘files’ and then populating that name with multiple files then a small hack is necessary as follows. As noted above Spring 2.5 throws an exception if it detects the same form input name of type file more than once. CommonsFileUploadSupport – the class which throws that exception is not final and the method which throws that exception is protected so using the wonders of inheritance and subclassing one can simply fix/modify the logic a little bit as follows. The change I’ve made is literally one word representing one method invocation which enables us to have multiple files incoming under the same form input name. It attempts to override the method protected MultipartParsingResult parseFileItems(List fileItems, String encoding) {} of the abstract class CommonsFileUploadSupport by extending the class CommonsMultipartResolver such as, package multipartResolver; import java.io.UnsupportedEncodingException; import java.util.HashMap; import java.util.Iterator; import java.util.List; import java.util.Map; import javax.servlet.ServletContext; import org.apache.commons.fileupload.FileItem; import org.springframework.util.StringUtils; import org.springframework.web.multipart.MultipartException; import org.springframework.web.multipart.MultipartFile; import org.springframework.web.multipart.commons.CommonsMultipartFile; import org.springframework.web.multipart.commons.CommonsMultipartResolver; final public class MultiCommonsMultipartResolver extends CommonsMultipartResolver { public MultiCommonsMultipartResolver() { } public MultiCommonsMultipartResolver(ServletContext servletContext) { super(servletContext); } @Override @SuppressWarnings("unchecked") protected MultipartParsingResult parseFileItems(List fileItems, String encoding) { Map<String, MultipartFile> multipartFiles = new HashMap<String, MultipartFile>(); Map multipartParameters = new HashMap(); // Extract multipart files and multipart parameters. for (Iterator it = fileItems.iterator(); it.hasNext();) { FileItem fileItem = (FileItem) it.next(); if (fileItem.isFormField()) { String value = null; if (encoding != null) { try { value = fileItem.getString(encoding); } catch (UnsupportedEncodingException ex) { if (logger.isWarnEnabled()) { logger.warn("Could not decode multipart item '" + fileItem.getFieldName() + "' with encoding '" + encoding + "': using platform default"); } value = fileItem.getString(); } } else { value = fileItem.getString(); } String[] curParam = (String[]) multipartParameters.get(fileItem.getFieldName()); if (curParam == null) { // simple form field multipartParameters.put(fileItem.getFieldName(), new String[] { value }); } else { // array of simple form fields String[] newParam = StringUtils.addStringToArray(curParam, value); multipartParameters.put(fileItem.getFieldName(), newParam); } } else { // multipart file field CommonsMultipartFile file = new CommonsMultipartFile(fileItem); if (multipartFiles.put(fileItem.getName(), file) != null) { throw new MultipartException("Multiple files for field name [" + file.getName() + "] found - not supported by MultipartResolver"); } if (logger.isDebugEnabled()) { logger.debug("Found multipart file [" + file.getName() + "] of size " + file.getSize() + " bytes with original filename [" + file.getOriginalFilename() + "], stored " + file.getStorageDescription()); } } } return new MultipartParsingResult(multipartFiles, multipartParameters); } } What happens is that the last line in the method parseFileItems() (the return statement) i.e. return new MultipartParsingResult(multipartFiles, multipartParameters); causes a compile-time error because the first parameter multipartFiles is a type of Map implemented by HashMap but in reality, it requires a parameter of type MultiValueMap<String, MultipartFile> It is a constructor of a static class inside the abstract class CommonsFileUploadSupport, public abstract class CommonsFileUploadSupport { protected static class MultipartParsingResult { public MultipartParsingResult(MultiValueMap<String, MultipartFile> mpFiles, Map<String, String[]> mpParams) { } } } The reason might be - this solution is about the Spring version 2.5 and I'm using the Spring version 3.0.2 which might be inappropriate for this version. I however tried to replace the Map with MultiValueMap in various ways such as the one shown in the following segment of code, MultiValueMap<String, MultipartFile>mul=new LinkedMultiValueMap<String, MultipartFile>(); for(Entry<String, MultipartFile>entry:multipartFiles.entrySet()) { mul.add(entry.getKey(), entry.getValue()); } return new MultipartParsingResult(mul, multipartParameters); but no success. I'm not sure how to replace Map with MultiValueMap and even doing so could work either. After doing this, the browser shows the Http response, HTTP Status 400 - type Status report message description The request sent by the client was syntactically incorrect (). Apache Tomcat/6.0.26 I have tried to shorten the question as possible as I could and I haven't included unnecessary code. How could be made it possible to upload multiple files after Spring has been configured with HiddenHttpMethodFilter? That blog indicates that It is a long standing, high priority bug. If there is no solution regarding the version 3.0.2 (3 or higher) then I have to disable Spring support forever and continue to use commons-fileupolad as suggested by the third solution on that blog omitting the PUT, DELETE and other request methods forever. Just curiously waiting for a solution and/or suggestion. Very little changes to the code in the parseFileItems() method inside the class MultiCommonsMultipartResolver might make it to upload multiple files but I couldn't succeed in my attempts (again with the Spring version 3.0.2 (3 or higher)).

Read the article

Python - pickling fails for numpy.void objects

- by I82Much

>>> idmapfile = open("idmap", mode="w") >>> pickle.dump(idMap, idmapfile) >>> idmapfile.close() >>> idmapfile = open("idmap") >>> unpickled = pickle.load(idmapfile) >>> unpickled == idMap False idMap[1] {1537: (552, 1, 1537, 17.793827056884766, 3), 1540: (4220, 1, 1540, 19.31205940246582, 3), 1544: (592, 1, 1544, 18.129131317138672, 3), 1675: (529, 1, 1675, 18.347782135009766, 3), 1550: (4048, 1, 1550, 19.31205940246582, 3), 1424: (1528, 1, 1424, 19.744396209716797, 3), 1681: (1265, 1, 1681, 19.596025466918945, 3), 1560: (3457, 1, 1560, 20.530569076538086, 3), 1690: (477, 1, 1690, 17.395542144775391, 3), 1691: (554, 1, 1691, 13.446117401123047, 3), 1436: (3010, 1, 1436, 19.596025466918945, 3), 1434: (3183, 1, 1434, 19.744396209716797, 3), 1441: (3570, 1, 1441, 20.589576721191406, 3), 1435: (476, 1, 1435, 19.640911102294922, 3), 1444: (527, 1, 1444, 17.98480224609375, 3), 1478: (1897, 1, 1478, 19.596025466918945, 3), 1575: (614, 1, 1575, 19.371648788452148, 3), 1586: (2189, 1, 1586, 19.31205940246582, 3), 1716: (3470, 1, 1716, 19.158674240112305, 3), 1590: (2278, 1, 1590, 19.596025466918945, 3), 1463: (991, 1, 1463, 19.31205940246582, 3), 1594: (1890, 1, 1594, 19.596025466918945, 3), 1467: (1087, 1, 1467, 19.31205940246582, 3), 1596: (3759, 1, 1596, 19.744396209716797, 3), 1602: (3011, 1, 1602, 20.530569076538086, 3), 1547: (490, 1, 1547, 17.994071960449219, 3), 1605: (658, 1, 1605, 19.31205940246582, 3), 1606: (1794, 1, 1606, 16.964881896972656, 3), 1719: (1826, 1, 1719, 19.596025466918945, 3), 1617: (583, 1, 1617, 11.894925117492676, 3), 1492: (3441, 1, 1492, 20.500667572021484, 3), 1622: (3215, 1, 1622, 19.31205940246582, 3), 1628: (2761, 1, 1628, 19.744396209716797, 3), 1502: (1563, 1, 1502, 19.596025466918945, 3), 1632: (1108, 1, 1632, 15.457141876220703, 3), 1468: (3779, 1, 1468, 19.596025466918945, 3), 1642: (3970, 1, 1642, 19.744396209716797, 3), 1518: (612, 1, 1518, 18.570245742797852, 3), 1647: (854, 1, 1647, 16.964881896972656, 3), 1650: (2099, 1, 1650, 20.439058303833008, 3), 1651: (540, 1, 1651, 18.552841186523438, 3), 1653: (613, 1, 1653, 19.237197875976563, 3), 1532: (537, 1, 1532, 18.885730743408203, 3)} >>> unpickled[1] {1537: (64880, 1638, 56700, -1.0808743559293829e+18, 152), 1540: (64904, 1638, 0, 0.0, 0), 1544: (54472, 1490, 0, 0.0, 0), 1675: (6464, 1509, 0, 0.0, 0), 1550: (43592, 1510, 0, 0.0, 0), 1424: (43616, 1510, 0, 0.0, 0), 1681: (0, 0, 0, 0.0, 0), 1560: (400, 152, 400, 2.1299736657737219e-43, 0), 1690: (408, 152, 408, 2.7201111331839077e+26, 34), 1435: (424, 152, 61512, 1.0122952080313192e-39, 0), 1436: (400, 152, 400, 20.250289916992188, 3), 1434: (424, 152, 62080, 1.0122952080313192e-39, 0), 1441: (400, 152, 400, 12.250144958496094, 3), 1691: (424, 152, 42608, 15.813941955566406, 3), 1444: (400, 152, 400, 19.625289916992187, 3), 1606: (424, 152, 42432, 5.2947192852601414e-22, 41), 1575: (400, 152, 400, 6.2537390010262572e-36, 0), 1586: (424, 152, 42488, 1.0122601755697111e-39, 0), 1716: (400, 152, 400, 6.2537390010262572e-36, 0), 1590: (424, 152, 64144, 1.0126357235581501e-39, 0), 1463: (400, 152, 400, 6.2537390010262572e-36, 0), 1594: (424, 152, 32672, 17.002994537353516, 3), 1467: (400, 152, 400, 19.750289916992187, 3), 1596: (424, 152, 7176, 1.0124003054161436e-39, 0), 1602: (400, 152, 400, 18.500289916992188, 3), 1547: (424, 152, 7000, 1.0124003054161436e-39, 0), 1605: (400, 152, 400, 20.500289916992188, 3), 1478: (424, 152, 42256, -6.0222748507426518e+30, 222), 1719: (400, 152, 400, 6.2537390010262572e-36, 0), 1617: (424, 152, 16472, 1.0124283313854301e-39, 0), 1492: (400, 152, 400, 6.2537390010262572e-36, 0), 1622: (424, 152, 35304, 1.0123190301052127e-39, 0), 1628: (400, 152, 400, 6.2537390010262572e-36, 0), 1502: (424, 152, 63152, 19.627988815307617, 3), 1632: (400, 152, 400, 19.375289916992188, 3), 1468: (424, 152, 38088, 1.0124213248931084e-39, 0), 1642: (400, 152, 400, 6.2537390010262572e-36, 0), 1518: (424, 152, 63896, 1.0127436235399031e-39, 0), 1647: (400, 152, 400, 6.2537390010262572e-36, 0), 1650: (424, 152, 53424, 16.752857208251953, 3), 1651: (400, 152, 400, 19.250289916992188, 3), 1653: (424, 152, 50624, 1.0126497365427934e-39, 0), 1532: (400, 152, 400, 6.2537390010262572e-36, 0)} The keys come out fine, the values are screwed up. I tried same thing loading file in binary mode; didn't fix the problem. Any idea what I'm doing wrong? Edit: Here's the code with binary. Note that the values are different in the unpickled object. >>> idmapfile = open("idmap", mode="wb") >>> pickle.dump(idMap, idmapfile) >>> idmapfile.close() >>> idmapfile = open("idmap", mode="rb") >>> unpickled = pickle.load(idmapfile) >>> unpickled==idMap False >>> unpickled[1] {1537: (12176, 2281, 56700, -1.0808743559293829e+18, 152), 1540: (0, 0, 15934, 2.7457842047810522e+26, 108), 1544: (400, 152, 400, 4.9518498821046956e+27, 53), 1675: (408, 152, 408, 2.7201111331839077e+26, 34), 1550: (456, 152, 456, -1.1349175514578289e+18, 152), 1424: (432, 152, 432, 4.5939047815653343e-40, 11), 1681: (408, 152, 408, 2.1299736657737219e-43, 0), 1560: (376, 152, 376, 2.1299736657737219e-43, 0), 1690: (376, 152, 376, 2.1299736657737219e-43, 0), 1435: (376, 152, 376, 2.1299736657737219e-43, 0), 1436: (376, 152, 376, 2.1299736657737219e-43, 0), 1434: (376, 152, 376, 2.1299736657737219e-43, 0), 1441: (376, 152, 376, 2.1299736657737219e-43, 0), 1691: (376, 152, 376, 2.1299736657737219e-43, 0), 1444: (376, 152, 376, 2.1299736657737219e-43, 0), 1606: (25784, 2281, 376, -3.2883343074537754e+26, 34), 1575: (24240, 2281, 376, 2.1299736657737219e-43, 0), 1586: (24240, 2281, 376, 2.1299736657737219e-43, 0), 1716: (24240, 2281, 376, -3.0093091599657311e-35, 26), 1590: (24240, 2281, 376, 2.1299736657737219e-43, 0), 1463: (24240, 2281, 376, 2.1299736657737219e-43, 0), 1594: (24240, 2281, 376, -4123208450048.0, 196), 1467: (25784, 2281, 376, 2.1299736657737219e-43, 0), 1596: (25784, 2281, 376, 2.1299736657737219e-43, 0), 1602: (25784, 2281, 376, -5.9963281433905448e+26, 76), 1547: (25784, 2281, 376, -218106240.0, 139), 1605: (25784, 2281, 376, -3.7138649803377281e+27, 56), 1478: (376, 152, 376, 2.1299736657737219e-43, 0), 1719: (25784, 2281, 376, 2.1299736657737219e-43, 0), 1617: (25784, 2281, 376, -1.4411779941597184e+17, 237), 1492: (25784, 2281, 376, 2.8596493694487798e-30, 80), 1622: (25784, 2281, 376, 184686084096.0, 93), 1628: (1336, 152, 1336, 3.1691839245470052e+29, 179), 1502: (1272, 152, 1272, -5.2042207205116645e-17, 99), 1632: (1208, 152, 1208, 2.1299736657737219e-43, 0), 1468: (1144, 152, 1144, 2.1299736657737219e-43, 0), 1642: (1080, 152, 1080, 2.1299736657737219e-43, 0), 1518: (1016, 152, 1016, 4.0240902787680023e+35, 145), 1647: (952, 152, 952, -985172619034624.0, 237), 1650: (888, 152, 888, 12094787289088.0, 66), 1651: (824, 152, 824, 2.1299736657737219e-43, 0), 1653: (760, 152, 760, 0.00018310768064111471, 238), 1532: (696, 152, 696, 8.8978061885676389e+26, 125)} OK I've isolated the problem, but don't know why it's so. First, apparently what I'm pickling are not tuples (though they look like it), but instead numpy.void types. Here is a series to illustrate the problem. first = run0.detections[0] >>> first (1, 19, 1578, 82.637763977050781, 1) >>> type(first) <type 'numpy.void'> >>> firstTuple = tuple(first) >>> theFile = open("pickleTest", "w") >>> pickle.dump(first, theFile) >>> theTupleFile = open("pickleTupleTest", "w") >>> pickle.dump(firstTuple, theTupleFile) >>> theFile.close() >>> theTupleFile.close() >>> first (1, 19, 1578, 82.637763977050781, 1) >>> firstTuple (1, 19, 1578, 82.637764, 1) >>> theFile = open("pickleTest", "r") >>> theTupleFile = open("pickleTupleTest", "r") >>> unpickledTuple = pickle.load(theTupleFile) >>> unpickledVoid = pickle.load(theFile) >>> type(unpickledVoid) <type 'numpy.void'> >>> type(unpickledTuple) <type 'tuple'> >>> unpickledTuple (1, 19, 1578, 82.637764, 1) >>> unpickledTuple == firstTuple True >>> unpickledVoid == first False >>> unpickledVoid (7936, 1705, 56700, -1.0808743559293829e+18, 152) >>> first (1, 19, 1578, 82.637763977050781, 1)

Read the article

How Should I Generate Trade Statistics For CouchDB/Rails3 Application?

- by James

My Problem: I am trying to developing a web application for currency traders. The application allows traders to enter or upload information about their trades and I want to calculate a wide variety of statistics based on what the user entered. Now, normally I would use a relational database for this, but I have two requirements that don't fit well with a relational database so I am attempting to use couchdb. Those two problems are: 1) Primarily, I have a companion desktop application that users will be able to work with and replicate to the site using couchdb's awesome replication feature and 2) I would like to allow users to be able to define their own custom things to track about trades and generate results based off of what they enter. The schema less nature of couch seems perfect here, but it may end up being harder than it sounds. (I already know couch requires you to define views in advance and such so I was just planning on sticking all the custom attributes in an array and then emitting the array in the view and further processing from there.) What I Am Doing: Right now I am just emitting each trade in couch keyed by each user's system and querying with the key of the system to get an array of trades per system. Simple. I am not using a reduce function currently to calculate any stats because I couldn't figure out how to get everything I need without getting a reduce overflow error. Here is an example of rows that are getting emitted from couch: {"total_rows":134,"offset":0,"rows":[ {"id":"5b1dcd47221e160d8721feee4ccc64be", "key":["80e40ba2fa43589d57ec3f1d19db41e6","2010/05/14 04:32:37 +0000"], null, "doc":{ "_id":"5b1dcd47221e160d8721feee4ccc64be", "_rev":"1-bc9fe763e2637694df47d6f5efb58e5b", "couchrest-type":"Trade", "system":"80e40ba2fa43589d57ec3f1d19db41e6", "pair":"EUR/USD", "direction":"Buy", "entry":12600, "exit":12700, "stop_loss":12500, "profit_target":12700, "status":"Closed", "slug":"101332132375", "custom_tracking": [{"name":"signal", "value":"Pin Bar"}] "updated_at":"2010/05/14 04:32:37 +0000", "created_at":"2010/05/14 04:32:37 +0000", "result":100}} ]} In my rails 3 controller I am basically just populating an array of trades such as the one above and then extracting out the relevant data into smaller arrays that I can compute my statistics on. Here is my show action for the page that I want to display the stats and all the trades: def show @trades = Trade.by_system(:startkey => [@system.id], :endkey => [@system.id, Time.now ]) @trades.each do |trade| if trade.result > 0 @winning_trades << trade.result elsif trade.result < 0 @losing_trades << trade.result else @breakeven_trades << trade.result end if trade.direction == "Buy" @long_trades << trade.result else @short_trades << trade.result end if trade["custom_tracking"] != nil @custom_tracking << {"result" => trade.result, "variables" => trade["custom_tracking"]} end end end I am omitting some other stuff that is going on, but that is the gist of what I am doing. Then I am calculating stuff in the view layer to produce some results: <% winning_long_trades = @long_trades.reject {|trade| trade <= 0 } %> <% winning_short_trades = @short_trades.reject {|trade| trade <= 0 } %> <ul> <li>Total Trades: <%= @trades.count %></li> <li>Winners: <%= @winning_trades.size %></li> <li>Biggest Winner (Pips): <%= @winning_trades.max %></li> <li>Average Win(Pips): <%= @winning_trades.sum/@winning_trades.size %></li> <li>Losers: <%= @losing_trades.size %></li> <li>Biggest Loser (Pips): <%= @losing_trades.min %></li> <li>Average Loss(Pips): <%= @losing_trades.sum/@losing_trades.size %></li> <li>Breakeven Trades: <%= @breakeven_trades.size %></li> <li>Long Trades: <%= @long_trades.size %></li> <li>Winning Long Trades: <%= winning_long_trades.size %></li> <li>Short Trades: <%= @short_trades.size %></li> <li>Winning Short Trades: <%= winning_short_trades.size %></li> <li>Total Pips: <%= @winning_trades.sum + @losing_trades.sum %></li> <li>Win Rate (%): <%= @winning_trades.size/@trades.count.to_f * 100 %></li> </ul> This produces the following results, which aside from a few things is exactly what I want: Total Trades: 134 Winners: 70 Biggest Winner (Pips): 1488 Average Win(Pips): 440 Losers: 58 Biggest Loser (Pips): -516 Average Loss(Pips): -225 Breakeven Trades: 6 Long Trades: 125 Winning Long Trades: 67 Short Trades: 9 Winning Short Trades: 3 Total Pips: 17819 Win Rate (%): 52.23880597014925 What I Am Wondering- Finally The Actual Questions: I am starting to get really skeptical of how well this method will work when a user has 5,000 trades instead of just 134 like in this example. I anticipate most users will only have somewhere under 200 per year, but some users may have a couple thousand trades per year. Probably no more than 5,000 per year. It seems to work ok now, but the page load times are already getting a tad high for my tastes. (About 800ms to generate the page according to rails logs with about a 250ms of that spent in the view layer.) I will end up caching this page I am sure, but I still need the regenerate the page each time a trade is updated and I can't afford to have this be too slow. Sooo..... Is doing something similar here possible with a straight couchdb reduce function? I am assuming handing this off to couch would possibly help with larger data sets. I couldn't figure out how, but I suppose that doesn't mean it isn't possible. If possible, any hints will be helpful. Could I use a list function if a reduce was not available due to reduce constraints? Are couchdb list functions suitable for this type of calculations? Anyone have any idea of whether or not list functions perform well? Any hints what one would look like for the type of calculations I am trying to achieve? I thought about other options such as running the calculations at the time each trade was saved or nightly if I had to and saving the results to a statistics doc that I could then query so that all the processing was done ahead of time. I would like this to be the last resort because then I can't really filter out trades by time periods dynamically like I would really like to. (I want to have a slider that a user can slide to only show trades from that time period using the startkey and endkey in couchdb if I can.) If I should continue running the calculations inside the rails app at the time of the page view, what can I do to improve my current implementation. I am new to rails, couch and programming in general. I am sure that I could be doing something better here. Do I need to create an array for each stat or is there a better way to do that. I guess I just would really like some advice on how to tackle this problem. I want to keep the page generation time minimal since I anticipate these being some of the highest trafficked pages. My gut is that I will need to offload the statistics calculation to either couch or run the stats in advance of when they are called, but I am not sure. Lastly: Like I mentioned above, one of the primary reasons for using couch is to allow users to define their own things to track per trade. Getting the data into couch is no problem, but how would I be able to take the custom_tracking array and find how many winning trades for each named tracking attribute. If anyone can give me any hints to the possibility of doing this that would be great. Thanks a bunch. Would really appreciate any help. Willing to fork out some $$$ if someone wants to take on the problem for me. (Don't know if that is allowed on stack overflow or not.)

Read the article

mysql_query arguments in PHP

- by Chris Wilson

I'm currently building my first database in MySQL with an interface written in PHP and am using the 'learn-by-doing' approach. The figure below illustrates my database. Table names are at the top, and the attribute names are as they appear in the real database. I am attempting to query the values of each of these attributes using the code seen below the table. I think there is something wrong with my mysql_query() function since I am able to observe the expected behaviour when my form is successfully submitted, but no search results are returned. Can anyone see where I'm going wrong here? Update 1: I've updated the question with my enter script, minus the database login credentials. <html> <head> <title>Search</title> </head> <body> <h1>Search</h1>  <form name = "search" action = "<?=$PHP_SELF?>" method = "get"> Search for <input type = "text" name = "find" /> in <select name = "field"> <option value = "Title">Title</option> <option value = "Description">Description</option> <option value = "City">Location</option> <option value = "Company_name">Employer</option> </select> <input type = "submit" name = "search" value = "Search" /> </form> <form name = "clearsearch" action = "Search.php"> <input type = "submit" value = "Reset search" /> </form> <?php if (isset($_GET["search"])) // Check if form has been submitted correctly { // Check for a search query if($_GET["find"] == "") { echo "<p>You did not enter a search query. Please press the 'Reset search' button and try again"; exit; } echo "<h2>Search results</h2>"; ?> <table align = "left" border = "1" cellspacing = "2" cellpadding = "2"> <tr> <th><font face="Arial, Helvetica, sans-serif">No.</font></th> <th><font face="Arial, Helvetica, sans-serif">Title</font></th> <th><font face="Arial, Helvetica, sans-serif">Employer</font></th> <th><font face="Arial, Helvetica, sans-serif">Description</font></th> <th><font face="Arial, Helvetica, sans-serif">Location</font></th> <th><font face="Arial, Helvetica, sans-serif">Date Posted</font></th> <th><font face="Arial, Helvetica, sans-serif">Application Deadline</font></th> </tr> <? // Connect to the database $username=REDACTED; $password=REDACTED; $host=REDACTED; $database=REDACTED; mysql_connect($host, $username, $password); @mysql_select_db($database) or die (mysql_error()); // Perform the search $find = mysql_real_escape_string($find); $query = "SELECT job.Title, job.Description, employer.Company_name, address.City, job.Date_posted, job.Application_deadline WHERE ( Title = '{$_GET['find']}' OR Company_name = '{$_GET['find']}' OR Date_posted = '{$_GET['find']}' OR Application_deadline = '{$_GET['find']}' ) AND job.employer_id_job = employer.employer_id AND job.address_id_job = address.address_id"; if (!$query) { die ('Invalid query:' .mysql_error()); } $result = mysql_query($query); $num = mysql_numrows($result); $count = 0; while ($count < $num) { $title = mysql_result ($result, $count, "Title"); $date_posted = mysql_result ($result, $count, "Date_posted"); $application_deadline = mysql_result ($result, $count, "Application_deadline"); $description = mysql_result ($result, $count, "Description"); $company = mysql_result ($result, $count, "Company_name"); $city = mysql_result ($result, $count, "City"); ?> <tr> <td><font face = "Arial, Helvetica, sans-serif"><? echo $count + 1; ?></font></td> <td><font face = "Arial, Helvetica, sans-serif"><? echo $title; ?></font></td> <td><font face = "Arial, Helvetica, sans-serif"><? echo $company; ?></font></td> <td><font face = "Arial, Helvetica, sans-serif"><? echo $description; ?></font></td> <td><font face = "Arial, Helvetica, sans-serif"><? echo $date_posted; ?></font></td> <td><font face = "Arial, Helvetica, sans-serif"><? echo $application_deadline; ?></font></td> <td><font face = "Arial, Helvetica, sans-serif"><? echo $education_level; ?></font></td> <td><font face = "Arial, Helvetica, sans-serif"><? echo $years_of_experience; ?></font></td> <? $count ++; } } ?> </body> </html>

Read the article

Tips on how to refactor this unwieldy upvote/downvote code

- by bob_cobb

Basically this code is for an upvote/downvote system and I'm basically Incrementing the count by 1 when voting up Decrementing the count by 1 when voting down If the number of downvotes upvotes, we'll assume it's a negative score, so the count stays 0 Reverting the count back to what it originally was when clicking upvote twice or downvote twice Never go below 0 (by showing negative numbers); Basically it's the same scoring scheme reddit uses, and I tried to get some ideas from the source which was minified and kind of hard to grok: a.fn.vote = function(b, c, e, j) { if (reddit.logged && a(this).hasClass("arrow")) { var k = a(this).hasClass("up") ? 1 : a(this).hasClass("down") ? -1 : 0, v = a(this).all_things_by_id(), p = v.children().not(".child").find(".arrow"), q = k == 1 ? "up" : "upmod"; p.filter("." + q).removeClass(q).addClass(k == 1 ? "upmod" : "up"); q = k == -1 ? "down" : "downmod"; p.filter("." + q).removeClass(q).addClass(k == -1 ? "downmod" : "down"); reddit.logged && (v.each(function() { var b = a(this).find(".entry:first, .midcol:first"); k > 0 ? b.addClass("likes").removeClass("dislikes unvoted") : k < 0 ? b.addClass("dislikes").removeClass("likes unvoted") : b.addClass("unvoted").removeClass("likes dislikes") }), a.defined(j) || (j = v.filter(":first").thing_id(), b += e ? "" : "-" + j, a.request("vote", {id: j,dir: k,vh: b}))); c && c(v, k) } }; I'm trying to look for a pattern, but there are a bunch of edge cases that I've been adding in, and it's still a little off. My code (and fiddle): $(function() { var down = $('.vote-down'); var up = $('.vote-up'); var direction = up.add(down); var largeCount = $('#js-large-count'); var totalUp = $('#js-total-up'); var totalDown = $('#js-total-down'); var totalUpCount = parseInt(totalUp.text(), 10); var totalDownCount = parseInt(totalDown.text(), 10); var castVote = function(submissionId, voteType) { /* var postURL = '/vote'; $.post(postURL, { submissionId : submissionId, voteType : voteType } , function (data){ if (data.response === 'success') { totalDown.text(data.downvotes); totalUp.text(data.upvotes); } }, 'json'); */ alert('voted!'); }; $(direction).on('click', direction, function () { // The submission ID var $that = $(this), submissionId = $that.attr('id'), voteType = $that.attr('dir'), // what direction was voted? [up or down] isDown = $that.hasClass('down'), isUp = $that.hasClass('up'), curVotes = parseInt($that.parent().find('div.count').text(), 10); // current vote castVote(submissionId, voteType); // Voted up on submission if (voteType === 'up') { var alreadyVotedUp = $that.hasClass('likes'), upCount = $that.next('div.count'), dislikes = $that.nextAll('a').first(); // next anchor attr if (alreadyVotedUp) { // Clicked the up arrow and previously voted up $that.toggleClass('likes up'); if (totalUpCount > totalDownCount) { upCount.text(curVotes - 1); largeCount.text(curVotes - 1); } else { upCount.text(0); largeCount.text(0); } upCount.css('color', '#555').hide().fadeIn(); largeCount.hide().fadeIn(); } else if (dislikes.hasClass('dislikes')) { // Voted down now are voting up if (totalDownCount > totalUpCount) { upCount.text(0); largeCount.text(0); } else if (totalUpCount > totalDownCount) { console.log(totalDownCount); console.log(totalUpCount); if (totalDownCount === 0) { upCount.text(curVotes + 1); largeCount.text(curVotes + 1); } else { upCount.text(curVotes + 2); largeCount.text(curVotes + 2); } } else { upCount.text(curVotes + 1); largeCount.text(curVotes + 1); } dislikes.toggleClass('down dislikes'); upCount.css('color', '#296394').hide().fadeIn(200); largeCount.hide().fadeIn(); } else { if (totalDownCount > totalUpCount) { upCount.text(0); largeCount.text(0); } else { // They clicked the up arrow and haven't voted up yet upCount.text(curVotes + 1); largeCount.text(curVotes + 1).hide().fadeIn(200); upCount.css('color', '#296394').hide().fadeIn(200); } } // Change arrow to dark blue if (isUp) { $that.toggleClass('up likes'); } } // Voted down on submission if (voteType === 'down') { var alreadyVotedDown = $that.hasClass('dislikes'), downCount = $that.prev('div.count'); // Get previous anchor attribute var likes = $that.prevAll('a').first(); if (alreadyVotedDown) { if (curVotes === 0) { if (totalDownCount > totalUp) { downCount.text(curVotes); largeCount.text(curVotes); } else { if (totalUpCount < totalDownCount || totalUpCount == totalDownCount) { downCount.text(0); largeCount.text(0); } else { downCount.text((totalUpCount - totalUpCount) + 1); largeCount.text((totalUpCount - totalUpCount) + 1); } } } else { downCount.text(curVotes + 1); largeCount.text(curVotes + 1); } $that.toggleClass('down dislikes'); downCount.css('color', '#555').hide().fadeIn(200); largeCount.hide().fadeIn(); } else if (likes.hasClass('likes')) { // They voted up from 0, and now are voting down if (curVotes <= 1) { downCount.text(0); largeCount.text(0); } else { // They voted up, now they are voting down (from a number > 0) downCount.text(curVotes - 2); largeCount.text(curVotes - 2); } likes.toggleClass('up likes'); downCount.css('color', '#ba2a2a').hide().fadeIn(200); largeCount.hide().fadeIn(200); } else { if (curVotes > 0) { downCount.text(curVotes - 1); largeCount.text(curVotes - 1); } else { downCount.text(curVotes); largeCount.text(curVotes); } downCount.css('color', '#ba2a2a').hide().fadeIn(200); largeCount.hide().fadeIn(200); } // Change the arrow to red if (isDown) { $that.toggleClass('down dislikes'); } } return false; }); });? Pretty convoluted, right? Is there a way to do something similar but in about 1/3 of the code I've written? After attempting to re-write it, I find myself doing the same thing so I just gave up halfway through and decided to ask for some help (fiddle of most recent).

Read the article

Weird vps server issue

- by anon-user0

I have an unmanaged linux vps Ubuntu 11.10 (Oneiric Ocelot). I have LNMP installed. Also php-fpm php-apc, varnish, memcache. I have (or rather had) several live sites on it. under normal load the server uses ~700 mb memory. But since last night its using only 20mb~ memory and a lot of the services seems to be down (according to htop) I only see nginx working and mysql starts up and goes does every few minutes on a loop. Here are some information on the server that might help you help me: root@server:~# uname -a Linux server 2.6.18-308.el5.028stab099.3 #1 SMP Wed Mar 7 15:56:00 MSK 2012 i686 i686 i386 GNU/Linux - root@server:~# ifconfig -a lo Link encap:Local Loopback LOOPBACK MTU:16436 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) venet0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 inet addr:127.0.0.2 P-t-P:127.0.0.2 Bcast:0.0.0.0 Mask:255.255.255.255 UP BROADCAST POINTOPOINT RUNNING NOARP MTU:1500 Metric:1 RX packets:12515 errors:0 dropped:0 overruns:0 frame:0 TX packets:9541 errors:0 dropped:1 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:7191214 (7.1 MB) TX bytes:536726 (536.7 KB) venet0:0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 inet addr:176.31.158.78 P-t-P:176.31.158.78 Bcast:0.0.0.0 Mask:255.255.255.255 UP BROADCAST POINTOPOINT RUNNING NOARP MTU:1500 Metric:1 - root@server:~# netstat -l Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 *:http-alt *:* LISTEN tcp 0 0 *:ssh *:* LISTEN tcp6 0 0 [::]:http-alt [::]:* LISTEN tcp6 0 0 [::]:ssh [::]:* LISTEN Active UNIX domain sockets (only servers) Proto RefCnt Flags Type State I-Node Path unix 2 [ ACC ] STREAM LISTENING 9307368 @/com/ubuntu/upstart - htop: http://i.stack.imgur.com/NHKYX.png EDIT: Stressed. mind was not working adding log: root@server:~# less /var/log/syslog Jun 27 05:27:42 server syslogd 1.5.0#6ubuntu1: restart. Jun 27 05:39:01 server CRON[9298]: (root) CMD ([ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -depth -mindepth 1 -maxdepth 1 -type f -cmin +$(/usr/lib/php5/maxlifetime) -delete) Jun 27 05:40:01 server CRON[9463]: (smmsp) CMD (test -x /etc/init.d/sendmail && /usr/share/sendmail/sendmail cron-msp) Jun 27 05:46:21 server sm-msp-queue[9480]: q5R1R7Ue004056: to=root, ctladdr=root (0/0), delay=00:19:14, xdelay=00:06:18, mailer=relay, pri=122407, relay=[127.0.0.1] [127.0.0.1], dsn=4.0.0, stat=Deferred: Connection timed out with [127.0.0.1] Jun 27 05:52:39 server sm-msp-queue[9480]: q5QMk7S9009582: to=root, ctladdr=root (0/0), delay=03:06:32, xdelay=00:06:18, mailer=relay, pri=842407, relay=[127.0.0.1] [127.0.0.1], dsn=4.0.0, stat=Deferred: Connection timed out with [127.0.0.1] Jun 27 06:00:01 server CRON[15671]: (smmsp) CMD (test -x /etc/init.d/sendmail && /usr/share/sendmail/sendmail cron-msp) Jun 27 06:06:22 server sm-msp-queue[15690]: q5R1R7Ue004056: to=root, ctladdr=root (0/0), delay=00:39:15, xdelay=00:06:18, mailer=relay, pri=212407, relay=[127.0.0.1] [127.0.0.1], dsn=4.0.0, stat=Deferred: Connection timed out with [127.0.0.1] Jun 27 06:09:01 server CRON[18114]: (root) CMD ([ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -depth -mindepth 1 -maxdepth 1 -type f -cmin +$(/usr/lib/php5/maxlifetime) -delete) Jun 27 06:12:40 server sm-msp-queue[15690]: q5QMk7S9009582: to=root, ctladdr=root (0/0), delay=03:26:33, xdelay=00:06:18, mailer=relay, pri=932407, relay=[127.0.0.1] [127.0.0.1], dsn=4.0.0, stat=Deferred: Connection timed out with [127.0.0.1] Jun 27 06:20:02 server CRON[21888]: (smmsp) CMD (test -x /etc/init.d/sendmail && /usr/share/sendmail/sendmail cron-msp) Jun 27 06:26:22 server sm-msp-queue[21907]: q5R1R7Ue004056: to=root, ctladdr=root (0/0), delay=00:59:15, xdelay=00:06:18, mailer=relay, pri=302407, relay=[127.0.0.1] [127.0.0.1], dsn=4.0.0, stat=Deferred: Connection timed out with [127.0.0.1] Jun 27 06:27:02 server CRON[24021]: (root) CMD (cd / && run-parts --report /etc/cron.hourly) Jun 27 06:32:40 server sm-msp-queue[21907]: q5QMk7S9009582: to=root, ctladdr=root (0/0), delay=03:46:33, xdelay=00:06:18, mailer=relay, pri=1022407, relay=[127.0.0.1] [127.0.0.1], dsn=4.0.0, stat=Deferred: Connection timed out with [127.0.0.1] Jun 27 06:39:01 server CRON[27941]: (root) CMD ([ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -depth -mindepth 1 -maxdepth 1 -type f -cmin +$(/usr/lib/php5/maxlifetime) -delete) Jun 27 06:40:02 server CRON[28110]: (smmsp) CMD (test -x /etc/init.d/sendmail && /usr/share/sendmail/sendmail cron-msp) Jun 27 06:46:22 server sm-msp-queue[28125]: q5R1R7Ue004056: to=root, ctladdr=root (0/0), delay=01:19:15, xdelay=00:06:18, mailer=relay, pri=392407, relay=[127.0.0.1] [127.0.0.1], dsn=4.0.0, stat=Deferred: Connection timed out with [127.0.0.1] Jun 27 06:52:40 server sm-msp-queue[28125]: q5QMk7S9009582: to=root, ctladdr=root (0/0), delay=04:06:33, xdelay=00:06:18, mailer=relay, pri=1112407, relay=[127.0.0.1] [127.0.0.1], dsn=4.0.0, stat=Deferred: Connection timed out with [127.0.0.1] Jun 27 06:52:40 server sm-msp-queue[28125]: q5QMk7S9009582: q5R2e4uo028125: sender notify: Warning: could not send message for past 4 hours Jun 27 06:52:44 server sm-msp-queue[28125]: q5R2e4uo028125: to=root, delay=00:00:04, xdelay=00:00:04, mailer=relay, pri=33690, relay=[127.0.0.1] [127.0.0.1], dsn=4.0.0, stat=Deferred: Connection timed out with [127.0.0.1] Jun 27 07:00:02 server CRON[1543]: (smmsp) CMD (test -x /etc/init.d/sendmail && /usr/share/sendmail/sendmail cron-msp) Jun 27 07:06:21 server sm-msp-queue[1560]: q5R2e4uo028125: to=root, delay=00:13:41, xdelay=00:06:18, mailer=relay, pri=123690, relay=[127.0.0.1] [127.0.0.1], dsn=4.0.0, stat=Deferred: Connection timed out with [127.0.0.1] Jun 27 07:09:01 server CRON[3986]: (root) CMD ([ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -depth -mindepth 1 -maxdepth 1 -type f -cmin +$(/usr/lib/php5/maxlifetime) -delete) Jun 27 07:12:39 server sm-msp-queue[1560]: q5R1R7Ue004056: to=root, ctladdr=root (0/0), delay=01:45:32, xdelay=00:06:18, mailer=relay, pri=482407, relay=[127.0.0.1] [127.0.0.1], dsn=4.0.0, stat=Deferred: Connection timed out with [127.0.0.1] Jun 27 07:18:57 server sm-msp-queue[1560]: q5QMk7S9009582: to=root, ctladdr=root (0/0), delay=04:32:50, xdelay=00:06:18, mailer=relay, pri=1202407, relay=[127.0.0.1] [127.0.0.1], dsn=4.0.0, stat=Deferred: Connection timed out with [127.0.0.1] Jun 27 07:20:02 server CRON[7760]: (smmsp) CMD (test -x /etc/init.d/sendmail && /usr/share/sendmail/sendmail cron-msp) Jun 27 07:26:22 server sm-msp-queue[7775]: q5R2e4uo028125: to=root, delay=00:33:42, xdelay=00:06:18, mailer=relay, pri=213690, relay=[127.0.0.1] [127.0.0.1], dsn=4.0.0, stat=Deferred: Connection timed out with [127.0.0.1] Jun 27 07:27:01 server CRON[9887]: (root) CMD (cd / && run-parts --report /etc/cron.hourly) Jun 27 07:32:40 server sm-msp-queue[7775]: q5R1R7Ue004056: to=root, ctladdr=root (0/0), delay=02:05:33, xdelay=00:06:18, mailer=relay, pri=572407, relay=[127.0.0.1] [127.0.0.1], dsn=4.0.0, stat=Deferred: Connection timed out with [127.0.0.1] Jun 27 07:38:58 server sm-msp-queue[7775]: q5QMk7S9009582: to=root, ctladdr=root (0/0), delay=04:52:51, xdelay=00:06:18, mailer=relay, pri=1292407, relay=[127.0.0.1] [127.0.0.1], dsn=4.0.0, stat=Deferred: Connection timed out with [127.0.0.1] Jun 27 07:39:01 server CRON[13813]: (root) CMD ([ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -depth -mindepth : root@server:~# df -h Filesystem Size Used Avail Use% Mounted on /dev/simfs 20G 2.3G 18G 12% / - Jun 26 16:22:41 server varnishd[1413]: Child (32425) died signal=3 Jun 26 16:22:41 server varnishd[1413]: child (21687) Started Jun 26 16:22:41 server varnishd[1413]: Child (21687) said Child starts Jun 26 16:22:41 server varnishd[1413]: Child (21687) said SMF.s0 mmap'ed 1073741824 bytes of 1073741824 Jun 26 16:34:28 server -- MARK -- Jun 26 16:54:29 server -- MARK -- Jun 26 17:14:29 server -- MARK -- Jun 26 17:34:29 server -- MARK -- Jun 26 17:54:29 server -- MARK -- Jun 26 18:14:29 server -- MARK -- Jun 26 18:34:29 server -- MARK -- Jun 26 18:54:29 server -- MARK -- Jun 26 19:14:29 server -- MARK -- Jun 26 19:34:29 server -- MARK -- Jun 26 19:54:29 server -- MARK -- Jun 26 20:14:29 server -- MARK -- Jun 26 20:34:29 server -- MARK -- Jun 26 20:48:12 server exiting on signal 15 Jun 26 20:51:58 server syslogd 1.5.0#6ubuntu1: restart. Jun 26 20:52:01 server varnishd[1324]: Platform: Linux,2.6.18-308.el5.028stab099.3,i686,-sfile,-smalloc,-hcritbit Jun 26 21:11:58 server -- MARK -- Jun 26 21:31:58 server -- MARK -- Jun 26 21:51:58 server -- MARK -- Jun 26 22:11:58 server -- MARK -- Jun 26 22:31:58 server -- MARK -- Jun 26 22:51:58 server -- MARK -- Jun 26 23:11:58 server -- MARK -- Jun 26 23:31:58 server -- MARK -- Jun 26 23:51:58 server -- MARK -- Jun 27 00:11:58 server -- MARK -- Jun 27 00:23:42 server exiting on signal 15 Jun 27 02:21:10 server syslogd 1.5.0#6ubuntu1: restart. Jun 27 02:21:12 server varnishd[1341]: Platform: Linux,2.6.18-308.el5.028stab099.3,i686,-sfile,-smalloc,-hcritbit Jun 27 02:41:10 server -- MARK -- Jun 27 02:46:41 server syslogd 1.5.0#6ubuntu1: restart. Jun 27 03:20:44 server syslogd 1.5.0#6ubuntu1: restart. Jun 27 03:20:46 server varnishd[1238]: Platform: Linux,2.6.18-308.el5.028stab099.3,i686,-sfile,-smalloc,-hcritbit Jun 27 03:20:46 server varnishd[1238]: child (1239) Started Jun 27 03:20:46 server varnishd[1238]: Child (1239) said Child starts Jun 27 03:20:46 server varnishd[1238]: Child (1239) said SMF.s0 mmap'ed 1073741824 bytes of 1073741824 Jun 27 03:32:52 server exiting on signal 15 Jun 27 03:33:16 server syslogd 1.5.0#6ubuntu1: restart. Jun 27 03:33:31 server varnishd[1372]: Platform: Linux,2.6.18-308.el5.028stab099.3,i686,-sfile,-smalloc,-hcritbit Jun 27 03:53:16 server -- MARK -- Jun 27 04:13:16 server -- MARK -- Jun 27 04:33:16 server -- MARK -- Jun 27 04:53:16 server -- MARK -- Jun 27 05:13:16 server -- MARK -- Jun 27 05:27:42 server syslogd 1.5.0#6ubuntu1: restart. Jun 27 05:53:17 server -- MARK -- Jun 27 06:13:17 server -- MARK -- Jun 27 06:33:17 server -- MARK -- Jun 27 06:53:17 server -- MARK -- Jun 27 07:13:17 server -- MARK -- Jun 27 07:33:17 server -- MARK -- Jun 27 07:53:17 server -- MARK -- Jun 27 08:13:17 server -- MARK -- Jun 27 08:33:17 server -- MARK -- Jun 27 08:53:17 server -- MARK -- Jun 27 09:13:17 server -- MARK -- Jun 27 09:33:17 server -- MARK -- Jun 27 09:53:17 server -- MARK -- Jun 27 10:13:17 server -- MARK -- Jun 27 10:33:17 server -- MARK -- Jun 27 10:53:17 server -- MARK -- Jun 27 11:13:17 server -- MARK -- Jun 27 11:33:17 server -- MARK -- Jun 27 11:53:18 server -- MARK -- Jun 27 12:13:18 server -- MARK -- Jun 27 12:33:18 server -- MARK -- Jun 27 12:53:18 server -- MARK -- Jun 27 13:13:18 server -- MARK -- Jun 27 13:33:18 server -- MARK -- Jun 27 13:53:18 server -- MARK -- Jun 27 14:13:18 server -- MARK -- Jun 27 14:33:18 server -- MARK -- Jun 27 14:53:18 server -- MARK -- -- root@server:~# cat /var/log/nginx/error.log 2012/06/27 03:32:54 [alert] 1199#0: worker process 1203 exited on signal 9 2012/06/27 03:32:54 [alert] 1199#0: worker process 1200 exited on signal 9 2012/06/27 03:32:54 [alert] 1199#0: worker process 1201 exited on signal 9 2012/06/27 03:32:54 [alert] 1199#0: worker process 1202 exited on signal 9 root@server:~# cat /var/log/nginx/access.log 31.210.99.87 - - [27/Jun/2012:09:09:08 +0400] "GET /w00tw00t.at.ISC.SANS.DFind:) HTTP/1.1" 400 172 "-" "-" 88.191.138.103 - - [27/Jun/2012:13:27:08 +0400] "GET /cms/cmx.jsp HTTP/1.1" 301 184 "-" "-" 88.191.138.103 - - [27/Jun/2012:13:27:08 +0400] "GET /iesvc/iesvc.jsp HTTP/1.1" 301 184 "-" "-" 88.191.138.103 - - [27/Jun/2012:13:27:08 +0400] "GET /cmd2/index.jsp HTTP/1.1" 301 184 "-" "-" 88.191.138.103 - - [27/Jun/2012:13:27:09 +0400] "GET /cmd/index.jsp HTTP/1.1" 301 184 "-" "-" 58.97.147.197 - - [27/Jun/2012:17:17:19 +0400] "GET / HTTP/1.1" 301 184 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.56 Safari/536.5" 58.97.147.197 - - [27/Jun/2012:17:17:37 +0400] "GET / HTTP/1.1" 301 184 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.56 Safari/536.5" 58.97.147.197 - - [27/Jun/2012:17:17:38 +0400] "-" 400 0 "-" "-" 58.97.147.197 - - [27/Jun/2012:17:17:38 +0400] "-" 400 0 "-" "-" 58.97.147.197 - - [27/Jun/2012:17:17:48 +0400] "-" 400 0 "-" "-" - root@server:~# cat /var/log/daemon.log Jun 26 20:48:10 server xinetd[1177]: Exiting... Jun 26 20:51:58 server xinetd[1174]: Reading included configuration file: /etc/xinetd.d/daytime [file=/etc/xinetd.d/daytime] [line=28] Jun 26 20:51:58 server xinetd[1174]: Reading included configuration file: /etc/xinetd.d/discard [file=/etc/xinetd.d/discard] [line=26] Jun 26 20:51:58 server xinetd[1174]: Reading included configuration file: /etc/xinetd.d/echo [file=/etc/xinetd.d/echo] [line=25] Jun 26 20:51:58 server xinetd[1174]: Reading included configuration file: /etc/xinetd.d/time [file=/etc/xinetd.d/time] [line=26] Jun 26 20:51:58 server xinetd[1174]: removing chargen Jun 26 20:51:58 server xinetd[1174]: removing chargen Jun 26 20:51:58 server xinetd[1174]: removing daytime Jun 26 20:51:58 server xinetd[1174]: removing daytime Jun 26 20:51:58 server xinetd[1174]: removing discard Jun 26 20:51:58 server xinetd[1174]: removing discard Jun 26 20:51:58 server xinetd[1174]: removing echo Jun 26 20:51:58 server xinetd[1174]: removing echo Jun 26 20:51:58 server xinetd[1174]: removing time Jun 26 20:51:58 server xinetd[1174]: removing time Jun 26 20:51:58 server xinetd[1174]: xinetd Version 2.3.14 started with libwrap loadavg options compiled in. Jun 26 20:51:58 server xinetd[1174]: Started working: 0 available services Jun 26 20:52:01 server vnstatd[1330]: vnStat daemon 1.11 started. Jun 26 20:52:01 server vnstatd[1330]: Monitoring: venet0 Jun 27 00:23:41 server xinetd[1174]: Exiting... Jun 27 02:21:12 server vnstatd[1349]: vnStat daemon 1.11 started. Jun 27 02:21:12 server vnstatd[1349]: Monitoring: venet0 Jun 27 03:20:44 server xinetd[1166]: attribute: disable should not be in default section [file=/etc/xinetd.conf] [line=12] Jun 27 03:20:44 server xinetd[1166]: Reading included configuration file: /etc/xinetd.d/chargen [file=/etc/xinetd.conf] [line=15] Jun 27 03:20:44 server xinetd[1166]: Reading included configuration file: /etc/xinetd.d/daytime [file=/etc/xinetd.d/daytime] [line=28] Jun 27 03:20:44 server xinetd[1166]: Reading included configuration file: /etc/xinetd.d/discard [file=/etc/xinetd.d/discard] [line=26] Jun 27 03:20:44 server xinetd[1166]: Reading included configuration file: /etc/xinetd.d/echo [file=/etc/xinetd.d/echo] [line=25] Jun 27 03:20:44 server xinetd[1166]: Reading included configuration file: /etc/xinetd.d/time [file=/etc/xinetd.d/time] [line=26] Jun 27 03:20:44 server xinetd[1166]: removing chargen Jun 27 03:20:44 server xinetd[1166]: removing chargen Jun 27 03:20:44 server xinetd[1166]: removing daytime Jun 27 03:20:44 server xinetd[1166]: removing daytime Jun 27 03:20:44 server xinetd[1166]: removing discard Jun 27 03:20:44 server xinetd[1166]: removing discard Jun 27 03:20:44 server xinetd[1166]: removing echo Jun 27 03:20:44 server xinetd[1166]: removing echo Jun 27 03:20:44 server xinetd[1166]: removing time Jun 27 03:20:44 server xinetd[1166]: removing time Jun 27 03:20:44 server xinetd[1166]: xinetd Version 2.3.14 started with libwrap loadavg options compiled in. Jun 27 03:20:44 server xinetd[1166]: Started working: 0 available services Jun 27 03:20:46 server vnstatd[1249]: vnStat daemon 1.11 started. Jun 27 03:20:46 server vnstatd[1249]: Monitoring: venet0 Jun 27 03:32:41 server xinetd[1166]: Exiting... Jun 27 03:33:32 server vnstatd[1380]: vnStat daemon 1.11 started. Jun 27 03:33:32 server vnstatd[1380]: Monitoring: venet0 root@server:~# - Anything else you need let me know

Read the article

The Incremental Architect’s Napkin - #5 - Design functions for extensibility and readability

- by Ralf Westphal

Originally posted on: http://geekswithblogs.net/theArchitectsNapkin/archive/2014/08/24/the-incremental-architectrsquos-napkin---5---design-functions-for.aspx The functionality of programs is entered via Entry Points. So what we´re talking about when designing software is a bunch of functions handling the requests represented by and flowing in through those Entry Points. Designing software thus consists of at least three phases: Analyzing the requirements to find the Entry Points and their signatures Designing the functionality to be executed when those Entry Points get triggered Implementing the functionality according to the design aka coding I presume, you´re familiar with phase 1 in some way. And I guess you´re proficient in implementing functionality in some programming language. But in my experience developers in general are not experienced in going through an explicit phase 2. “Designing functionality? What´s that supposed to mean?” you might already have thought. Here´s my definition: To design functionality (or functional design for short) means thinking about… well, functions. You find a solution for what´s supposed to happen when an Entry Point gets triggered in terms of functions. A conceptual solution that is, because those functions only exist in your head (or on paper) during this phase. But you may have guess that, because it´s “design” not “coding”. And here is, what functional design is not: It´s not about logic. Logic is expressions (e.g. +, -, && etc.) and control statements (e.g. if, switch, for, while etc.). Also I consider calling external APIs as logic. It´s equally basic. It´s what code needs to do in order to deliver some functionality or quality. Logic is what´s doing that needs to be done by software. Transformations are either done through expressions or API-calls. And then there is alternative control flow depending on the result of some expression. Basically it´s just jumps in Assembler, sometimes to go forward (if, switch), sometimes to go backward (for, while, do). But calling your own function is not logic. It´s not necessary to produce any outcome. Functionality is not enhanced by adding functions (subroutine calls) to your code. Nor is quality increased by adding functions. No performance gain, no higher scalability etc. through functions. Functions are not relevant to functionality. Strange, isn´t it. What they are important for is security of investment. By introducing functions into our code we can become more productive (re-use) and can increase evolvability (higher unterstandability, easier to keep code consistent). That´s no small feat, however. Evolvable code can hardly be overestimated. That´s why to me functional design is so important. It´s at the core of software development. To sum this up: Functional design is on a level of abstraction above (!) logical design or algorithmic design. Functional design is only done until you get to a point where each function is so simple you are very confident you can easily code it. Functional design an logical design (which mostly is coding, but can also be done using pseudo code or flow charts) are complementary. Software needs both. If you start coding right away you end up in a tangled mess very quickly. Then you need back out through refactoring. Functional design on the other hand is bloodless without actual code. It´s just a theory with no experiments to prove it. But how to do functional design? An example of functional design Let´s assume a program to de-duplicate strings. The user enters a number of strings separated by commas, e.g. a, b, a, c, d, b, e, c, a. And the program is supposed to clear this list of all doubles, e.g. a, b, c, d, e. There is only one Entry Point to this program: the user triggers the de-duplication by starting the program with the string list on the command line C:\>deduplicate "a, b, a, c, d, b, e, c, a" a, b, c, d, e …or by clicking on a GUI button. This leads to the Entry Point function to get called. It´s the program´s main function in case of the batch version or a button click event handler in the GUI version. That´s the physical Entry Point so to speak. It´s inevitable. What then happens is a three step process: Transform the input data from the user into a request. Call the request handler. Transform the output of the request handler into a tangible result for the user. Or to phrase it a bit more generally: Accept input. Transform input into output. Present output. This does not mean any of these steps requires a lot of effort. Maybe it´s just one line of code to accomplish it. Nevertheless it´s a distinct step in doing the processing behind an Entry Point. Call it an aspect or a responsibility - and you will realize it most likely deserves a function of its own to satisfy the Single Responsibility Principle (SRP). Interestingly the above list of steps is already functional design. There is no logic, but nevertheless the solution is described - albeit on a higher level of abstraction than you might have done yourself. But it´s still on a meta-level. The application to the domain at hand is easy, though: Accept string list from command line De-duplicate Present de-duplicated strings on standard output And this concrete list of processing steps can easily be transformed into code:static void Main(string[] args) { var input = Accept_string_list(args); var output = Deduplicate(input); Present_deduplicated_string_list(output); } Instead of a big problem there are three much smaller problems now. If you think each of those is trivial to implement, then go for it. You can stop the functional design at this point. But maybe, just maybe, you´re not so sure how to go about with the de-duplication for example. Then just implement what´s easy right now, e.g.private static string Accept_string_list(string[] args) { return args[0]; } private static void Present_deduplicated_string_list( string[] output) { var line = string.Join(", ", output); Console.WriteLine(line); } Accept_string_list() contains logic in the form of an API-call. Present_deduplicated_string_list() contains logic in the form of an expression and an API-call. And then repeat the functional design for the remaining processing step. What´s left is the domain logic: de-duplicating a list of strings. How should that be done? Without any logic at our disposal during functional design you´re left with just functions. So which functions could make up the de-duplication? Here´s a suggestion: De-duplicate Parse the input string into a true list of strings. Register each string in a dictionary/map/set. That way duplicates get cast away. Transform the data structure into a list of unique strings. Processing step 2 obviously was the core of the solution. That´s where real creativity was needed. That´s the core of the domain. But now after this refinement the implementation of each step is easy again:private static string[] Parse_string_list(string input) { return input.Split(',') .Select(s => s.Trim()) .ToArray(); } private static Dictionary<string,object> Compile_unique_strings(string[] strings) { return strings.Aggregate( new Dictionary<string, object>(), (agg, s) => { agg[s] = null; return agg; }); } private static string[] Serialize_unique_strings( Dictionary<string,object> dict) { return dict.Keys.ToArray(); } With these three additional functions Main() now looks like this:static void Main(string[] args) { var input = Accept_string_list(args); var strings = Parse_string_list(input); var dict = Compile_unique_strings(strings); var output = Serialize_unique_strings(dict); Present_deduplicated_string_list(output); } I think that´s very understandable code: just read it from top to bottom and you know how the solution to the problem works. It´s a mirror image of the initial design: Accept string list from command line Parse the input string into a true list of strings. Register each string in a dictionary/map/set. That way duplicates get cast away. Transform the data structure into a list of unique strings. Present de-duplicated strings on standard output You can even re-generate the design by just looking at the code. Code and functional design thus are always in sync - if you follow some simple rules. But about that later. And as a bonus: all the functions making up the process are small - which means easy to understand, too. So much for an initial concrete example. Now it´s time for some theory. Because there is method to this madness ;-) The above has only scratched the surface. Introducing Flow Design Functional design starts with a given function, the Entry Point. Its goal is to describe the behavior of the program when the Entry Point is triggered using a process, not an algorithm. An algorithm consists of logic, a process on the other hand consists just of steps or stages. Each processing step transforms input into output or a side effect. Also it might access resources, e.g. a printer, a database, or just memory. Processing steps thus can rely on state of some sort. This is different from Functional Programming, where functions are supposed to not be stateful and not cause side effects.[1] In its simplest form a process can be written as a bullet point list of steps, e.g. Get data from user Output result to user Transform data Parse data Map result for output Such a compilation of steps - possibly on different levels of abstraction - often is the first artifact of functional design. It can be generated by a team in an initial design brainstorming. Next comes ordering the steps. What should happen first, what next etc.? Get data from user Parse data Transform data Map result for output Output result to user That´s great for a start into functional design. It´s better than starting to code right away on a given function using TDD. Please get me right: TDD is a valuable practice. But it can be unnecessarily hard if the scope of a functionn is too large. But how do you know beforehand without investing some thinking? And how to do this thinking in a systematic fashion? My recommendation: For any given function you´re supposed to implement first do a functional design. Then, once you´re confident you know the processing steps - which are pretty small - refine and code them using TDD. You´ll see that´s much, much easier - and leads to cleaner code right away. For more information on this approach I call “Informed TDD” read my book of the same title. Thinking before coding is smart. And writing down the solution as a bunch of functions possibly is the simplest thing you can do, I´d say. It´s more according to the KISS (Keep It Simple, Stupid) principle than returning constants or other trivial stuff TDD development often is started with. So far so good. A simple ordered list of processing steps will do to start with functional design. As shown in the above example such steps can easily be translated into functions. Moving from design to coding thus is simple. However, such a list does not scale. Processing is not always that simple to be captured in a list. And then the list is just text. Again. Like code. That means the design is lacking visuality. Textual representations need more parsing by your brain than visual representations. Plus they are limited in their “dimensionality”: text just has one dimension, it´s sequential. Alternatives and parallelism are hard to encode in text. In addition the functional design using numbered lists lacks data. It´s not visible what´s the input, output, and state of the processing steps. That´s why functional design should be done using a lightweight visual notation. No tool is necessary to draw such designs. Use pen and paper; a flipchart, a whiteboard, or even a napkin is sufficient. Visualizing processes The building block of the functional design notation is a functional unit. I mostly draw it like this: Something is done, it´s clear what goes in, it´s clear what comes out, and it´s clear what the processing step requires in terms of state or hardware. Whenever input flows into a functional unit it gets processed and output is produced and/or a side effect occurs. Flowing data is the driver of something happening. That´s why I call this approach to functional design Flow Design. It´s about data flow instead of control flow. Control flow like in algorithms is of no concern to functional design. Thinking about control flow simply is too low level. Once you start with control flow you easily get bogged down by tons of details. That´s what you want to avoid during design. Design is supposed to be quick, broad brush, abstract. It should give overview. But what about all the details? As Robert C. Martin rightly said: “Programming is abot detail”. Detail is a matter of code. Once you start coding the processing steps you designed you can worry about all the detail you want. Functional design does not eliminate all the nitty gritty. It just postpones tackling them. To me that´s also an example of the SRP. Function design has the responsibility to come up with a solution to a problem posed by a single function (Entry Point). And later coding has the responsibility to implement the solution down to the last detail (i.e. statement, API-call). TDD unfortunately mixes both responsibilities. It´s just coding - and thereby trying to find detailed implementations (green phase) plus getting the design right (refactoring). To me that´s one reason why TDD has failed to deliver on its promise for many developers. Using functional units as building blocks of functional design processes can be depicted very easily. Here´s the initial process for the example problem: For each processing step draw a functional unit and label it. Choose a verb or an “action phrase” as a label, not a noun. Functional design is about activities, not state or structure. Then make the output of an upstream step the input of a downstream step. Finally think about the data that should flow between the functional units. Write the data above the arrows connecting the functional units in the direction of the data flow. Enclose the data description in brackets. That way you can clearly see if all flows have already been specified. Empty brackets mean “no data is flowing”, but nevertheless a signal is sent. A name like “list” or “strings” in brackets describes the data content. Use lower case labels for that purpose. A name starting with an upper case letter like “String” or “Customer” on the other hand signifies a data type. If you like, you also can combine descriptions with data types by separating them with a colon, e.g. (list:string) or (strings:string[]). But these are just suggestions from my practice with Flow Design. You can do it differently, if you like. Just be sure to be consistent. Flows wired-up in this manner I call one-dimensional (1D). Each functional unit just has one input and/or one output. A functional unit without an output is possible. It´s like a black hole sucking up input without producing any output. Instead it produces side effects. A functional unit without an input, though, does make much sense. When should it start to work? What´s the trigger? That´s why in the above process even the first processing step has an input. If you like, view such 1D-flows as pipelines. Data is flowing through them from left to right. But as you can see, it´s not always the same data. It get´s transformed along its passage: (args) becomes a (list) which is turned into (strings). The Principle of Mutual Oblivion A very characteristic trait of flows put together from function units is: no functional units knows another one. They are all completely independent of each other. Functional units don´t know where their input is coming from (or even when it´s gonna arrive). They just specify a range of values they can process. And they promise a certain behavior upon input arriving. Also they don´t know where their output is going. They just produce it in their own time independent of other functional units. That means at least conceptually all functional units work in parallel. Functional units don´t know their “deployment context”. They now nothing about the overall flow they are place in. They are just consuming input from some upstream, and producing output for some downstream. That makes functional units very easy to test. At least as long as they don´t depend on state or resources. I call this the Principle of Mutual Oblivion (PoMO). Functional units are oblivious of others as well as an overall context/purpose. They are just parts of a whole focused on a single responsibility. How the whole is built, how a larger goal is achieved, is of no concern to the single functional units. By building software in such a manner, functional design interestingly follows nature. Nature´s building blocks for organisms also follow the PoMO. The cells forming your body do not know each other. Take a nerve cell “controlling” a muscle cell for example:[2] The nerve cell does not know anything about muscle cells, let alone the specific muscel cell it is “attached to”. Likewise the muscle cell does not know anything about nerve cells, let a lone a specific nerve cell “attached to” it. Saying “the nerve cell is controlling the muscle cell” thus only makes sense when viewing both from the outside. “Control” is a concept of the whole, not of its parts. Control is created by wiring-up parts in a certain way. Both cells are mutually oblivious. Both just follow a contract. One produces Acetylcholine (ACh) as output, the other consumes ACh as input. Where the ACh is going, where it´s coming from neither cell cares about. Million years of evolution have led to this kind of division of labor. And million years of evolution have produced organism designs (DNA) which lead to the production of these different cell types (and many others) and also to their co-location. The result: the overall behavior of an organism. How and why this happened in nature is a mystery. For our software, though, it´s clear: functional and quality requirements needs to be fulfilled. So we as developers have to become “intelligent designers” of “software cells” which we put together to form a “software organism” which responds in satisfying ways to triggers from it´s environment. My bet is: If nature gets complex organisms working by following the PoMO, who are we to not apply this recipe for success to our much simpler “machines”? So my rule is: Wherever there is functionality to be delivered, because there is a clear Entry Point into software, design the functionality like nature would do it. Build it from mutually oblivious functional units. That´s what Flow Design is about. In that way it´s even universal, I´d say. Its notation can also be applied to biology: Never mind labeling the functional units with nouns. That´s ok in Flow Design. You´ll do that occassionally for functional units on a higher level of abstraction or when their purpose is close to hardware. Getting a cockroach to roam your bedroom takes 1,000,000 nerve cells (neurons). Getting the de-duplication program to do its job just takes 5 “software cells” (functional units). Both, though, follow the same basic principle. Translating functional units into code Moving from functional design to code is no rocket science. In fact it´s straightforward. There are two simple rules: Translate an input port to a function. Translate an output port either to a return statement in that function or to a function pointer visible to that function. The simplest translation of a functional unit is a function. That´s what you saw in the above example. Functions are mutually oblivious. That why Functional Programming likes them so much. It makes them composable. Which is the reason, nature works according to the PoMO. Let´s be clear about one thing: There is no dependency injection in nature. For all of an organism´s complexity no DI container is used. Behavior is the result of smooth cooperation between mutually oblivious building blocks. Functions will often be the adequate translation for the functional units in your designs. But not always. Take for example the case, where a processing step should not always produce an output. Maybe the purpose is to filter input. Here the functional unit consumes words and produces words. But it does not pass along every word flowing in. Some words are swallowed. Think of a spell checker. It probably should not check acronyms for correctness. There are too many of them. Or words with no more than two letters. Such words are called “stop words”. In the above picture the optionality of the output is signified by the astrisk outside the brackets. It means: Any number of (word) data items can flow from the functional unit for each input data item. It might be none or one or even more. This I call a stream of data. Such behavior cannot be translated into a function where output is generated with return. Because a function always needs to return a value. So the output port is translated into a function pointer or continuation which gets passed to the subroutine when called:[3]void filter_stop_words( string word, Action<string> onNoStopWord) { if (...check if not a stop word...) onNoStopWord(word); } If you want to be nitpicky you might call such a function pointer parameter an injection. And technically you´re right. Conceptually, though, it´s not an injection. Because the subroutine is not functionally dependent on the continuation. Firstly continuations are procedures, i.e. subroutines without a return type. Remember: Flow Design is about unidirectional data flow. Secondly the name of the formal parameter is chosen in a way as to not assume anything about downstream processing steps. onNoStopWord describes a situation (or event) within the functional unit only. Translating output ports into function pointers helps keeping functional units mutually oblivious in cases where output is optional or produced asynchronically. Either pass the function pointer to the function upon call. Or make it global by putting it on the encompassing class. Then it´s called an event. In C# that´s even an explicit feature.class Filter { public void filter_stop_words( string word) { if (...check if not a stop word...) onNoStopWord(word); } public event Action<string> onNoStopWord; } When to use a continuation and when to use an event dependens on how a functional unit is used in flows and how it´s packed together with others into classes. You´ll see examples further down the Flow Design road. Another example of 1D functional design Let´s see Flow Design once more in action using the visual notation. How about the famous word wrap kata? Robert C. Martin has posted a much cited solution including an extensive reasoning behind his TDD approach. So maybe you want to compare it to Flow Design. The function signature given is:string WordWrap(string text, int maxLineLength) {...} That´s not an Entry Point since we don´t see an application with an environment and users. Nevertheless it´s a function which is supposed to provide a certain functionality. The text passed in has to be reformatted. The input is a single line of arbitrary length consisting of words separated by spaces. The output should consist of one or more lines of a maximum length specified. If a word is longer than a the maximum line length it can be split in multiple parts each fitting in a line. Flow Design Let´s start by brainstorming the process to accomplish the feat of reformatting the text. What´s needed? Words need to be assembled into lines Words need to be extracted from the input text The resulting lines need to be assembled into the output text Words too long to fit in a line need to be split Does sound about right? I guess so. And it shows a kind of priority. Long words are a special case. So maybe there is a hint for an incremental design here. First let´s tackle “average words” (words not longer than a line). Here´s the Flow Design for this increment: The the first three bullet points turned into functional units with explicit data added. As the signature requires a text is transformed into another text. See the input of the first functional unit and the output of the last functional unit. In between no text flows, but words and lines. That´s good to see because thereby the domain is clearly represented in the design. The requirements are talking about words and lines and here they are. But note the asterisk! It´s not outside the brackets but inside. That means it´s not a stream of words or lines, but lists or sequences. For each text a sequence of words is output. For each sequence of words a sequence of lines is produced. The asterisk is used to abstract from the concrete implementation. Like with streams. Whether the list of words gets implemented as an array or an IEnumerable is not important during design. It´s an implementation detail. Does any processing step require further refinement? I don´t think so. They all look pretty “atomic” to me. And if not… I can always backtrack and refine a process step using functional design later once I´ve gained more insight into a sub-problem. Implementation The implementation is straightforward as you can imagine. The processing steps can all be translated into functions. Each can be tested easily and separately. Each has a focused responsibility. And the process flow becomes just a sequence of function calls: Easy to understand. It clearly states how word wrapping works - on a high level of abstraction. And it´s easy to evolve as you´ll see. Flow Design - Increment 2 So far only texts consisting of “average words” are wrapped correctly. Words not fitting in a line will result in lines too long. Wrapping long words is a feature of the requested functionality. Whether it´s there or not makes a difference to the user. To quickly get feedback I decided to first implement a solution without this feature. But now it´s time to add it to deliver the full scope. Fortunately Flow Design automatically leads to code following the Open Closed Principle (OCP). It´s easy to extend it - instead of changing well tested code. How´s that possible? Flow Design allows for extension of functionality by inserting functional units into the flow. That way existing functional units need not be changed. The data flow arrow between functional units is a natural extension point. No need to resort to the Strategy Pattern. No need to think ahead where extions might need to be made in the future. I just “phase in” the remaining processing step: Since neither Extract words nor Reformat know of their environment neither needs to be touched due to the “detour”. The new processing step accepts the output of the existing upstream step and produces data compatible with the existing downstream step. Implementation - Increment 2 A trivial implementation checking the assumption if this works does not do anything to split long words. The input is just passed on: Note how clean WordWrap() stays. The solution is easy to understand. A developer looking at this code sometime in the future, when a new feature needs to be build in, quickly sees how long words are dealt with. Compare this to Robert C. Martin´s solution:[4] How does this solution handle long words? Long words are not even part of the domain language present in the code. At least I need considerable time to understand the approach. Admittedly the Flow Design solution with the full implementation of long word splitting is longer than Robert C. Martin´s. At least it seems. Because his solution does not cover all the “word wrap situations” the Flow Design solution handles. Some lines would need to be added to be on par, I guess. But even then… Is a difference in LOC that important as long as it´s in the same ball park? I value understandability and openness for extension higher than saving on the last line of code. Simplicity is not just less code, it´s also clarity in design. But don´t take my word for it. Try Flow Design on larger problems and compare for yourself. What´s the easier, more straightforward way to clean code? And keep in mind: You ain´t seen all yet ;-) There´s more to Flow Design than described in this chapter. In closing I hope I was able to give you a impression of functional design that makes you hungry for more. To me it´s an inevitable step in software development. Jumping from requirements to code does not scale. And it leads to dirty code all to quickly. Some thought should be invested first. Where there is a clear Entry Point visible, it´s functionality should be designed using data flows. Because with data flows abstraction is possible. For more background on why that´s necessary read my blog article here. For now let me point out to you - if you haven´t already noticed - that Flow Design is a general purpose declarative language. It´s “programming by intention” (Shalloway et al.). Just write down how you think the solution should work on a high level of abstraction. This breaks down a large problem in smaller problems. And by following the PoMO the solutions to those smaller problems are independent of each other. So they are easy to test. Or you could even think about getting them implemented in parallel by different team members. Flow Design not only increases evolvability, but also helps becoming more productive. All team members can participate in functional design. This goes beyon collective code ownership. We´re talking collective design/architecture ownership. Because with Flow Design there is a common visual language to talk about functional design - which is the foundation for all other design activities. PS: If you like what you read, consider getting my ebook “The Incremental Architekt´s Napkin”. It´s where I compile all the articles in this series for easier reading. I like the strictness of Function Programming - but I also find it quite hard to live by. And it certainly is not what millions of programmers are used to. Also to me it seems, the real world is full of state and side effects. So why give them such a bad image? That´s why functional design takes a more pragmatic approach. State and side effects are ok for processing steps - but be sure to follow the SRP. Don´t put too much of it into a single processing step. ? Image taken from www.physioweb.org ? My code samples are written in C#. C# sports typed function pointers called delegates. Action is such a function pointer type matching functions with signature void someName(T t). Other languages provide similar ways to work with functions as first class citizens - even Java now in version 8. I trust you find a way to map this detail of my translation to your favorite programming language. I know it works for Java, C++, Ruby, JavaScript, Python, Go. And if you´re using a Functional Programming language it´s of course a no brainer. ? Taken from his blog post “The Craftsman 62, The Dark Path”. ?

Read the article

Using HTML 5 SessionState to save rendered Page Content

- by Rick Strahl

HTML 5 SessionState and LocalStorage are very useful and super easy to use to manage client side state. For building rich client side or SPA style applications it's a vital feature to be able to cache user data as well as HTML content in order to swap pages in and out of the browser's DOM. What might not be so obvious is that you can also use the sessionState and localStorage objects even in classic server rendered HTML applications to provide caching features between pages. These APIs have been around for a long time and are supported by most relatively modern browsers and even all the way back to IE8, so you can use them safely in your Web applications. SessionState and LocalStorage are easy The APIs that make up sessionState and localStorage are very simple. Both object feature the same API interface which is a simple, string based key value store that has getItem, setItem, removeitem, clear and key methods. The objects are also pseudo array objects and so can be iterated like an array with a length property and you have array indexers to set and get values with. Basic usage for storing and retrieval looks like this (using sessionStorage, but the syntax is the same for localStorage - just switch the objects):// set var lastAccess = new Date().getTime(); if (sessionStorage) sessionStorage.setItem("myapp_time", lastAccess.toString()); // retrieve in another page or on a refresh var time = null; if (sessionStorage) time = sessionStorage.getItem("myapp_time"); if (time) time = new Date(time * 1); else time = new Date(); sessionState stores data that is browser session specific and that has a liftetime of the active browser session or window. Shut down the browser or tab and the storage goes away. localStorage uses the same API interface, but the lifetime of the data is permanently stored in the browsers storage area until deleted via code or by clearing out browser cookies (not the cache). Both sessionStorage and localStorage space is limited. The spec is ambiguous about this - supposedly sessionStorage should allow for unlimited size, but it appears that most WebKit browsers support only 2.5mb for either object. This means you have to be careful what you store especially since other applications might be running on the same domain and also use the storage mechanisms. That said 2.5mb worth of character data is quite a bit and would go a long way. The easiest way to get a feel for how sessionState and localStorage work is to look at a simple example. You can go check out the following example online in Plunker: http://plnkr.co/edit/0ICotzkoPjHaWa70GlRZ?p=preview which looks like this: Plunker is an online HTML/JavaScript editor that lets you write and run Javascript code and similar to JsFiddle, but a bit cleaner to work in IMHO (thanks to John Papa for turning me on to it). The sample has two text boxes with counts that update session/local storage every time you click the related button. The counts are 'cached' in Session and Local storage. The point of these examples is that both counters survive full page reloads, and the LocalStorage counter survives a complete browser shutdown and restart. Go ahead and try it out by clicking the Reload button after updating both counters and then shutting down the browser completely and going back to the same URL (with the same browser). What you should see is that reloads leave both counters intact at the counted values, while a browser restart will leave only the local storage counter intact. The code to deal with the SessionStorage (and LocalStorage not shown here) in the example is isolated into a couple of wrapper methods to simplify the code: function getSessionCount() { var count = 0; if (sessionStorage) { var count = sessionStorage.getItem("ss_count"); count = !count ? 0 : count * 1; } $("#txtSession").val(count); return count; } function setSessionCount(count) { if (sessionStorage) sessionStorage.setItem("ss_count", count.toString()); } These two functions essentially load and store a session counter value. The two key methods used here are: sessionStorage.getItem(key); sessionStorage.setItem(key,stringVal); Note that the value given to setItem and return by getItem has to be a string. If you pass another type you get an error. Don't let that limit you though - you can easily enough store JSON data in a variable so it's quite possible to pass complex objects and store them into a single sessionStorage value:var user = { name: "Rick", id="ricks", level=8 } sessionStorage.setItem("app_user",JSON.stringify(user)); to retrieve it:var user = sessionStorage.getItem("app_user"); if (user) user = JSON.parse(user); Simple! If you're using the Chrome Developer Tools (F12) you can also check out the session and local storage state on the Resource tab: You can also use this tool to refresh or remove entries from storage. What we just looked at is a purely client side implementation where a couple of counters are stored. For rich client centric AJAX applications sessionStorage and localStorage provide a very nice and simple API to store application state while the application is running. But you can also use these storage mechanisms to manage server centric HTML applications when you combine server rendering with some JavaScript to perform client side data caching. You can both store some state information and data on the client (ie. store a JSON object and carry it forth between server rendered HTML requests) or you can use it for good old HTTP based caching where some rendered HTML is saved and then restored later. Let's look at the latter with a real life example. Why do I need Client-side Page Caching for Server Rendered HTML? I don't know about you, but in a lot of my existing server driven applications I have lists that display a fair amount of data. Typically these lists contain links to then drill down into more specific data either for viewing or editing. You can then click on a link and go off to a detail page that provides more concise content. So far so good. But now you're done with the detail page and need to get back to the list, so you click on a 'bread crumbs trail' or an application level 'back to list' button and… …you end up back at the top of the list - the scroll position, the current selection in some cases even filters conditions - all gone with the wind. You've left behind the state of the list and are starting from scratch in your browsing of the list from the top. Not cool! Sound familiar? This a pretty common scenario with server rendered HTML content where it's so common to display lists to drill into, only to lose state in the process of returning back to the original list. Look at just about any traditional forums application, or even StackOverFlow to see what I mean here. Scroll down a bit to look at a post or entry, drill in then use the bread crumbs or tab to go back… In some cases returning to the top of a list is not a big deal. On StackOverFlow that sort of works because content is turning around so quickly you probably want to actually look at the top posts. Not always though - if you're browsing through a list of search topics you're interested in and drill in there's no way back to that position. Essentially anytime you're actively browsing the items in the list, that's when state becomes important and if it's not handled the user experience can be really disrupting. Content Caching If you're building client centric SPA style applications this is a fairly easy to solve problem - you tend to render the list once and then update the page content to overlay the detail content, only hiding the list temporarily until it's used again later. It's relatively easy to accomplish this simply by hiding content on the page and later making it visible again. But if you use server rendered content, hanging on to all the detail like filters, selections and scroll position is not quite as easy. Or is it??? This is where sessionStorage comes in handy. What if we just save the rendered content of a previous page, and then restore it when we return to this page based on a special flag that tells us to use the cached version? Let's see how we can do this. A real World Use Case Recently my local ISP asked me to help out with updating an ancient classifieds application. They had a very busy, local classifieds app that was originally an ASP classic application. The old app was - wait for it: frames based - and even though I lobbied against it, the decision was made to keep the frames based layout to allow rapid browsing of the hundreds of posts that are made on a daily basis. The primary reason they wanted this was precisely for the ability to quickly browse content item by item. While I personally hate working with Frames, I have to admit that the UI actually works well with the frames layout as long as you're running on a large desktop screen. You can check out the frames based desktop site here: http://classifieds.gorge.net/ However when I rebuilt the app I also added a secondary view that doesn't use frames. The main reason for this of course was for mobile displays which work horribly with frames. So there's a somewhat mobile friendly interface to the interface, which ditches the frames and uses some responsive design tweaking for mobile capable operation: http://classifeds.gorge.net/mobile (or browse the base url with your browser width under 800px) Here's what the mobile, non-frames view looks like: As you can see this means that the list of classifieds posts now is a list and there's a separate page for drilling down into the item. And of course… originally we ran into that usability issue I mentioned earlier where the browse, view detail, go back to the list cycle resulted in lost list state. Originally in mobile mode you scrolled through the list, found an item to look at and drilled in to display the item detail. Then you clicked back to the list and BAM - you've lost your place. Because there are so many items added on a daily basis the full list is never fully loaded, but rather there's a "Load Additional Listings" entry at the button. Not only did we originally lose our place when coming back to the list, but any 'additionally loaded' items are no longer there because the list was now rendering as if it was the first page hit. The additional listings, and any filters, the selection of an item all were lost. Major Suckage! Using Client SessionStorage to cache Server Rendered Content To work around this problem I decided to cache the rendered page content from the list in SessionStorage. Anytime the list renders or is updated with Load Additional Listings, the page HTML is cached and stored in Session Storage. Any back links from the detail page or the login or write entry forms then point back to the list page with a back=true query string parameter. If the server side sees this parameter it doesn't render the part of the page that is cached. Instead the client side code retrieves the data from the sessionState cache and simply inserts it into the page. It sounds pretty simple, and the overall the process is really easy, but there are a few gotchas that I'll discuss in a minute. But first let's look at the implementation. Let's start with the server side here because that'll give a quick idea of the doc structure. As I mentioned the server renders data from an ASP.NET MVC view. On the list page when returning to the list page from the display page (or a host of other pages) looks like this: https://classifieds.gorge.net/list?back=True The query string value is a flag, that indicates whether the server should render the HTML. Here's what the top level MVC Razor view for the list page looks like:@model MessageListViewModel @{ ViewBag.Title = "Classified Listing"; bool isBack = !string.IsNullOrEmpty(Request.QueryString["back"]); } <form method="post" action="@Url.Action("list")"> <div id="SizingContainer"> @if (!isBack) { @Html.Partial("List_CommandBar_Partial", Model) <div id="PostItemContainer" class="scrollbox" xstyle="-webkit-overflow-scrolling: touch;"> @Html.Partial("List_Items_Partial", Model) @if (Model.RequireLoadEntry) { <div class="postitem loadpostitems" style="padding: 15px;"> <div id="LoadProgress" class="smallprogressright"></div> <div class="control-progress"> Load additional listings... </div> </div> } </div> } </div> </form> As you can see the query string triggers a conditional block that if set is simply not rendered. The content inside of #SizingContainer basically holds the entire page's HTML sans the headers and scripts, but including the filter options and menu at the top. In this case this makes good sense - in other situations the fact that the menu or filter options might be dynamically updated might make you only cache the list rather than essentially the entire page. In this particular instance all of the content works and produces the proper result as both the list along with any filter conditions in the form inputs are restored. Ok, let's move on to the client. On the client there are two page level functions that deal with saving and restoring state. Like the counter example I showed earlier, I like to wrap the logic to save and restore values from sessionState into a separate function because they are almost always used in several places.page.saveData = function(id) { if (!sessionStorage) return; var data = { id: id, scroll: $("#PostItemContainer").scrollTop(), html: $("#SizingContainer").html() }; sessionStorage.setItem("list_html",JSON.stringify(data)); }; page.restoreData = function() { if (!sessionStorage) return; var data = sessionStorage.getItem("list_html"); if (!data) return null; return JSON.parse(data); }; The data that is saved is an object which contains an ID which is the selected element when the user clicks and a scroll position. These two values are used to reset the scroll position when the data is used from the cache. Finally the html from the #SizingContainer element is stored, which makes for the bulk of the document's HTML. In this application the HTML captured could be a substantial bit of data. If you recall, I mentioned that the server side code renders a small chunk of data initially and then gets more data if the user reads through the first 50 or so items. The rest of the items retrieved can be rather sizable. Other than the JSON deserialization that's Ok. Since I'm using SessionStorage the storage space has no immediate limits. Next is the core logic to handle saving and restoring the page state. At first though this would seem pretty simple, and in some cases it might be, but as the following code demonstrates there are a few gotchas to watch out for. Here's the relevant code I use to save and restore:$( function() { … var isBack = getUrlEncodedKey("back", location.href); if (isBack) { // remove the back key from URL setUrlEncodedKey("back", "", location.href); var data = page.restoreData(); // restore from sessionState if (!data) { // no data - force redisplay of the server side default list window.location = "list"; return; } $("#SizingContainer").html(data.html); var el = $(".postitem[data-id=" + data.id + "]"); $(".postitem").removeClass("highlight"); el.addClass("highlight"); $("#PostItemContainer").scrollTop(data.scroll); setTimeout(function() { el.removeClass("highlight"); }, 2500); } else if (window.noFrames) page.saveData(null); // save when page loads $("#SizingContainer").on("click", ".postitem", function() { var id = $(this).attr("data-id"); if (!id) return true; if (window.noFrames) page.saveData(id); var contentFrame = window.parent.frames["Content"]; if (contentFrame) contentFrame.location.href = "show/" + id; else window.location.href = "show/" + id; return false; }); … The code starts out by checking for the back query string flag which triggers restoring from the client cache. If cached the cached data structure is read from sessionStorage. It's important here to check if data was returned. If the user had back=true on the querystring but there is no cached data, he likely bookmarked this page or otherwise shut down the browser and came back to this URL. In that case the server didn't render any detail and we have no cached data, so all we can do is redirect to the original default list view using window.location. If we continued the page would render no data - so make sure to always check the cache retrieval result. Always! If there is data the it's loaded and the data.html data is restored back into the document by simply injecting the HTML back into the document's #SizingContainer element:$("#SizingContainer").html(data.html); It's that simple and it's quite quick even with a fully loaded list of additional items and on a phone. The actual HTML data is stored to the cache on every page load initially and then again when the user clicks on an element to navigate to a particular listing. The former ensures that the client cache always has something in it, and the latter updates with additional information for the selected element. For the click handling I use a data-id attribute on the list item (.postitem) in the list and retrieve the id from that. That id is then used to navigate to the actual entry as well as storing that Id value in the saved cached data. The id is used to reset the selection by searching for the data-id value in the restored elements. The overall process of this save/restore process is pretty straight forward and it doesn't require a bunch of code, yet it yields a huge improvement in the usability of the site on mobile devices (or anybody who uses the non-frames view). Some things to watch out for As easy as it conceptually seems to simply store and retrieve cached content, you have to be quite aware what type of content you are caching. The code above is all that's specific to cache/restore cycle and it works, but it took a few tweaks to the rest of the script code and server code to make it all work. There were a few gotchas that weren't immediately obvious. Here are a few things to pay attention to: Event Handling Logic Timing of manipulating DOM events Inline Script Code Bookmarking to the Cache Url when no cache exists Do you have inline script code in your HTML? That script code isn't going to run if you restore from cache and simply assign or it may not run at the time you think it would normally in the DOM rendering cycle. JavaScript Event Hookups The biggest issue I ran into with this approach almost immediately is that originally I had various static event handlers hooked up to various UI elements that are now cached. If you have an event handler like:$("#btnSearch").click( function() {…}); that works fine when the page loads with server rendered HTML, but that code breaks when you now load the HTML from cache. Why? Because the elements you're trying to hook those events to may not actually be there - yet. Luckily there's an easy workaround for this by using deferred events. With jQuery you can use the .on() event handler instead:$("#SelectionContainer").on("click","#btnSearch", function() {…}); which monitors a parent element for the events and checks for the inner selector elements to handle events on. This effectively defers to runtime event binding, so as more items are added to the document bindings still work. For any cached content use deferred events. Timing of manipulating DOM Elements Along the same lines make sure that your DOM manipulation code follows the code that loads the cached content into the page so that you don't manipulate DOM elements that don't exist just yet. Ideally you'll want to check for the condition to restore cached content towards the top of your script code, but that can be tricky if you have components or other logic that might not all run in a straight line. Inline Script Code Here's another small problem I ran into: I use a DateTime Picker widget I built a while back that relies on the jQuery date time picker. I also created a helper function that allows keyboard date navigation into it that uses JavaScript logic. Because MVC's limited 'object model' the only way to embed widget content into the page is through inline script. This code broken when I inserted the cached HTML into the page because the script code was not available when the component actually got injected into the page. As the last bullet - it's a matter of timing. There's no good work around for this - in my case I pulled out the jQuery date picker and relied on native <input type="date" /> logic instead - a better choice these days anyway, especially since this view is meant to be primarily to serve mobile devices which actually support date input through the browser (unlike desktop browsers of which only WebKit seems to support it). Bookmarking Cached Urls When you cache HTML content you have to make a decision whether you cache on the client and also not render that same content on the server. In the Classifieds app I didn't render server side content so if the user comes to the page with back=True and there is no cached content I have to a have a Plan B. Typically this happens when somebody ends up bookmarking the back URL. The easiest and safest solution for this scenario is to ALWAYS check the cache result to make sure it exists and if not have a safe URL to go back to - in this case to the plain uncached list URL which amounts to effectively redirecting. This seems really obvious in hindsight, but it's easy to overlook and not see a problem until much later, when it's not obvious at all why the page is not rendering anything. Don't use <body> to replace Content Since we're practically replacing all the HTML in the page it may seem tempting to simply replace the HTML content of the <body> tag. Don't. The body tag usually contains key things that should stay in the page and be there when it loads. Specifically script tags and elements and possibly other embedded content. It's best to create a top level DOM element specifically as a placeholder container for your cached content and wrap just around the actual content you want to replace. In the app above the #SizingContainer is that container. Other Approaches The approach I've used for this application is kind of specific to the existing server rendered application we're running and so it's just one approach you can take with caching. However for server rendered content caching this is a pattern I've used in a few apps to retrofit some client caching into list displays. In this application I took the path of least resistance to the existing server rendering logic. Here are a few other ways that come to mind: Using Partial HTML Rendering via AJAXInstead of rendering the page initially on the server, the page would load empty and the client would render the UI by retrieving the respective HTML and embedding it into the page from a Partial View. This effectively makes the initial rendering and the cached rendering logic identical and removes the server having to decide whether this request needs to be rendered or not (ie. not checking for a back=true switch). All the logic related to caching is made on the client in this case. Using JSON Data and Client RenderingThe hardcore client option is to do the whole UI SPA style and pull data from the server and then use client rendering or databinding to pull the data down and render using templates or client side databinding with knockout/angular et al. As with the Partial Rendering approach the advantage is that there's no difference in the logic between pulling the data from cache or rendering from scratch other than the initial check for the cache request. Of course if the app is a full on SPA app, then caching may not be required even - the list could just stay in memory and be hidden and reactivated. I'm sure there are a number of other ways this can be handled as well especially using AJAX. AJAX rendering might simplify the logic, but it also complicates search engine optimization since there's no content loaded initially. So there are always tradeoffs and it's important to look at all angles before deciding on any sort of caching solution in general. State of the Session SessionState and LocalStorage are easy to use in client code and can be integrated even with server centric applications to provide nice caching features of content and data. In this post I've shown a very specific scenario of storing HTML content for the purpose of remembering list view data and state and making the browsing experience for lists a bit more friendly, especially if there's dynamically loaded content involved. If you haven't played with sessionStorage or localStorage I encourage you to give it a try. There's a lot of cool stuff that you can do with this beyond the specific scenario I've covered here… Resources Overview of localStorage (also applies to sessionStorage) Web Storage Compatibility Modernizr Test Suite© Rick Strahl, West Wind Technologies, 2005-2013Posted in JavaScript HTML5 ASP.NET MVC Tweet !function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0];if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src="//platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs"); (function() { var po = document.createElement('script'); po.type = 'text/javascript'; po.async = true; po.src = 'https://apis.google.com/js/plusone.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s); })();

Read the article

Java Cloud Service Integration to REST Service

- by Jani Rautiainen

Service (JCS) provides a platform to develop and deploy business applications in the cloud. In Fusion Applications Cloud deployments customers do not have the option to deploy custom applications developed with JDeveloper to ensure the integrity and supportability of the hosted application service. Instead the custom applications can be deployed to the JCS and integrated to the Fusion Application Cloud instance. This series of articles will go through the features of JCS, provide end-to-end examples on how to develop and deploy applications on JCS and how to integrate them with the Fusion Applications instance. In this article a custom application integrating with REST service will be implemented. We will use REST services provided by Taleo as an example; however the same approach will work with any REST service. In this example the data from the REST service is used to populate a dynamic table. Pre-requisites Access to Cloud instance In order to deploy the application access to a JCS instance is needed, a free trial JCS instance can be obtained from Oracle Cloud site. To register you will need a credit card even if the credit card will not be charged. To register simply click "Try it" and choose the "Java" option. The confirmation email will contain the connection details. See this video for example of the registration.Once the request is processed you will be assigned 2 service instances; Java and Database. Applications deployed to the JCS must use Oracle Database Cloud Service as their underlying database. So when JCS instance is created a database instance is associated with it using a JDBC data source.The cloud services can be monitored and managed through the web UI. For details refer to Getting Started with Oracle Cloud. JDeveloper JDeveloper contains Cloud specific features related to e.g. connection and deployment. To use these features download the JDeveloper from JDeveloper download site by clicking the "Download JDeveloper 11.1.1.7.1 for ADF deployment on Oracle Cloud" link, this version of JDeveloper will have the JCS integration features that will be used in this article. For versions that do not include the Cloud integration features the Oracle Java Cloud Service SDK or the JCS Java Console can be used for deployment. For details on installing and configuring the JDeveloper refer to the installation guideFor details on SDK refer to Using the Command-Line Interface to Monitor Oracle Java Cloud Service and Using the Command-Line Interface to Manage Oracle Java Cloud Service. Access to a local database The database associated with the JCS instance cannot be connected to with JDBC. Since creating ADFbc business component requires a JDBC connection we will need access to a local database. 3rd party libraries This example will use some 3rd party libraries for implementing the REST service call and processing the input / output content. Other libraries may also be used, however these are tested to work. Jersey 1.x Jersey library will be used as a client to make the call to the REST service. JCS documentation for supported specifications states: Java API for RESTful Web Services (JAX-RS) 1.1 So Jersey 1.x will be used. Download the single-JAR Jersey bundle; in this example Jersey 1.18 JAR bundle is used. Json-simple Jjson-simple library will be used to process the json objects. Download the JAR file; in this example json-simple-1.1.1.jar is used. Accessing data in Taleo Before implementing the application it is beneficial to familiarize oneself with the data in Taleo. Easiest way to do this is by using a RESTClient on your browser. Once added to the browser you can access the UI: The client can be used to call the REST services to test the URLs and data before adding them into the application. First derive the base URL for the service this can be done with: Method: GET URL: https://tbe.taleo.net/MANAGER/dispatcher/api/v1/serviceUrl/<company name> The response will contain the base URL to be used for the service calls for the company. Next obtain authentication token with: Method: POST URL: https://ch.tbe.taleo.net/CH07/ats/api/v1/login?orgCode=<company>&userName=<user name>&password=<password> The response includes an authentication token that can be used for few hours to authenticate with the service: { "response": { "authToken": "webapi26419680747505890557" }, "status": { "detail": {}, "success": true } } To authenticate the service calls navigate to "Headers -> Custom Header": And add a new request header with: Name: Cookie Value: authToken=webapi26419680747505890557 Once authentication token is defined the tool can be used to invoke REST services; for example: Method: GET URL: https://ch.tbe.taleo.net/CH07/ats/api/v1/object/candidate/search.xml?status=16 This data will be used on the application to be created. For details on the Taleo REST services refer to the Taleo Business Edition REST API Guide. Create Application First Fusion Web Application is created and configured. Start JDeveloper and click "New Application": Application Name: JcsRestDemo Application Package Prefix: oracle.apps.jcs.test Application Template: Fusion Web Application (ADF) Configure Local Cloud Connection Follow the steps documented in the "Java Cloud Service ADF Web Application" article to configure a local database connection needed to create the ADFbc objects. Configure Libraries Add the 3rd party libraries into the class path. Create the following directory and copy the jar files into it: <JDEV_USER_HOME>/JcsRestDemo/lib Select the "Model" project, navigate "Application -> Project Properties -> Libraries and Classpath -> Add JAR / Directory" and add the 2 3rd party libraries: Accessing Data from Taleo To access data from Taleo using the REST service the 3rd party libraries will be used. 2 Java classes are implemented, one representing the Candidate object and another for accessing the Taleo repository Candidate Candidate object is a POJO object used to represent the candidate data obtained from the Taleo repository. The data obtained will be used to populate the ADFbc object used to display the data on the UI. The candidate object contains simply the variables we obtain using the REST services and the getters / setters for them: Navigate "New -> General -> Java -> Java Class", enter "Candidate" as the name and create it in the package "oracle.apps.jcs.test.model". Copy / paste the following as the content: import oracle.jbo.domain.Number; public class Candidate { private Number candId; private String firstName; private String lastName; public Candidate() { super(); } public Candidate(Number candId, String firstName, String lastName) { super(); this.candId = candId; this.firstName = firstName; this.lastName = lastName; } public void setCandId(Number candId) { this.candId = candId; } public Number getCandId() { return candId; } public void setFirstName(String firstName) { this.firstName = firstName; } public String getFirstName() { return firstName; } public void setLastName(String lastName) { this.lastName = lastName; } public String getLastName() { return lastName; } } Taleo Repository Taleo repository class will interact with the Taleo REST services. The logic will query data from Taleo and populate Candidate objects with the data. The Candidate object will then be used to populate the ADFbc object used to display data on the UI. Navigate "New -> General -> Java -> Java Class", enter "TaleoRepository" as the name and create it in the package "oracle.apps.jcs.test.model". Copy / paste the following as the content (for details of the implementation refer to the documentation in the code): import com.sun.jersey.api.client.Client; import com.sun.jersey.api.client.ClientResponse; import com.sun.jersey.api.client.WebResource; import com.sun.jersey.core.util.MultivaluedMapImpl; import java.io.StringReader; import java.util.ArrayList; import java.util.Iterator; import java.util.List; import java.util.Map; import javax.ws.rs.core.MediaType; import javax.ws.rs.core.MultivaluedMap; import oracle.jbo.domain.Number; import org.json.simple.JSONArray; import org.json.simple.JSONObject; import org.json.simple.parser.JSONParser; /** * This class interacts with the Taleo REST services */ public class TaleoRepository { /** * Connection information needed to access the Taleo services */ String _company = null; String _userName = null; String _password = null; /** * Jersey client used to access the REST services */ Client _client = null; /** * Parser for processing the JSON objects used as * input / output for the services */ JSONParser _parser = null; /** * The base url for constructing the REST URLs. This is obtained * from Taleo with a service call */ String _baseUrl = null; /** * Authentication token obtained from Taleo using a service call. * The token can be used to authenticate on subsequent * service calls. The token will expire in 4 hours */ String _authToken = null; /** * Static url that can be used to obtain the url used to construct * service calls for a given company */ private static String _taleoUrl = "https://tbe.taleo.net/MANAGER/dispatcher/api/v1/serviceUrl/"; /** * Default constructor for the repository * Authentication details are passed as parameters and used to generate * authentication token. Note that each service call will * generate its own token. This is done to avoid dealing with the expiry * of the token. Also only 20 tokens are allowed per user simultaneously. * So instead for each call there is login / logout. * * @param company the company for which the service calls are made * @param userName the user name to authenticate with * @param password the password to authenticate with. */ public TaleoRepository(String company, String userName, String password) { super(); _company = company; _userName = userName; _password = password; _client = Client.create(); _parser = new JSONParser(); _baseUrl = getBaseUrl(); } /** * This obtains the base url for a company to be used * to construct the urls for service calls * @return base url for the service calls */ private String getBaseUrl() { String result = null; if (null != _baseUrl) { result = _baseUrl; } else { try { String company = _company; WebResource resource = _client.resource(_taleoUrl + company); ClientResponse response = resource.type(MediaType.APPLICATION_FORM_URLENCODED_TYPE).get(ClientResponse.class); String entity = response.getEntity(String.class); JSONObject jsonObject = (JSONObject)_parser.parse(new StringReader(entity)); JSONObject jsonResponse = (JSONObject)jsonObject.get("response"); result = (String)jsonResponse.get("URL"); } catch (Exception ex) { ex.printStackTrace(); } } return result; } /** * Generates authentication token, that can be used to authenticate on * subsequent service calls. Note that each service call will * generate its own token. This is done to avoid dealing with the expiry * of the token. Also only 20 tokens are allowed per user simultaneously. * So instead for each call there is login / logout. * @return authentication token that can be used to authenticate on * subsequent service calls */ private String login() { String result = null; try { MultivaluedMap<String, String> formData = new MultivaluedMapImpl(); formData.add("orgCode", _company); formData.add("userName", _userName); formData.add("password", _password); WebResource resource = _client.resource(_baseUrl + "login"); ClientResponse response = resource.type(MediaType.APPLICATION_FORM_URLENCODED_TYPE).post(ClientResponse.class, formData); String entity = response.getEntity(String.class); JSONObject jsonObject = (JSONObject)_parser.parse(new StringReader(entity)); JSONObject jsonResponse = (JSONObject)jsonObject.get("response"); result = (String)jsonResponse.get("authToken"); } catch (Exception ex) { throw new RuntimeException("Unable to login ", ex); } if (null == result) throw new RuntimeException("Unable to login "); return result; } /** * Releases a authentication token. Each call to login must be followed * by call to logout after the processing is done. This is required as * the tokens are limited to 20 per user and if not released the tokens * will only expire after 4 hours. * @param authToken */ private void logout(String authToken) { WebResource resource = _client.resource(_baseUrl + "logout"); resource.header("cookie", "authToken=" + authToken).post(ClientResponse.class); } /** * This method is used to obtain a list of candidates using a REST * service call. At this example the query is hard coded to query * based on status. The url constructed to access the service is: * <_baseUrl>/object/candidate/search.xml?status=16 * @return List of candidates obtained with the service call */ public List<Candidate> getCandidates() { List<Candidate> result = new ArrayList<Candidate>(); try { // First login, note that in finally block we must have logout _authToken = "authToken=" + login(); /** * Construct the URL, the resulting url will be: * <_baseUrl>/object/candidate/search.xml?status=16 */ MultivaluedMap<String, String> formData = new MultivaluedMapImpl(); formData.add("status", "16"); JSONArray searchResults = (JSONArray)getTaleoResource("object/candidate/search", "searchResults", formData); /** * Process the results, the resulting JSON object is something like * this (simplified for readability): * * { * "response": * { * "searchResults": * [ * { * "candidate": * { * "candId": 211, * "firstName": "Mary", * "lastName": "Stochi", * logic here will find the candidate object(s), obtain the desired * data from them, construct a Candidate object based on the data * and add it to the results. */ for (Object object : searchResults) { JSONObject temp = (JSONObject)object; JSONObject candidate = (JSONObject)findObject(temp, "candidate"); Long candIdTemp = (Long)candidate.get("candId"); Number candId = (null == candIdTemp ? null : new Number(candIdTemp)); String firstName = (String)candidate.get("firstName"); String lastName = (String)candidate.get("lastName"); result.add(new Candidate(candId, firstName, lastName)); } } catch (Exception ex) { ex.printStackTrace(); } finally { if (null != _authToken) logout(_authToken); } return result; } /** * Convenience method to construct url for the service call, invoke the * service and obtain a resource from the response * @param path the path for the service to be invoked. This is combined * with the base url to construct a url for the service * @param resource the key for the object in the response that will be * obtained * @param parameters any parameters used for the service call. The call * is slightly different depending whether parameters exist or not. * @return the resource from the response for the service call */ private Object getTaleoResource(String path, String resource, MultivaluedMap<String, String> parameters) { Object result = null; try { WebResource webResource = _client.resource(_baseUrl + path); ClientResponse response = null; if (null == parameters) response = webResource.header("cookie", _authToken).get(ClientResponse.class); else response = webResource.queryParams(parameters).header("cookie", _authToken).get(ClientResponse.class); String entity = response.getEntity(String.class); JSONObject jsonObject = (JSONObject)_parser.parse(new StringReader(entity)); result = findObject(jsonObject, resource); } catch (Exception ex) { ex.printStackTrace(); } return result; } /** * Convenience method to recursively find a object with an key * traversing down from a given root object. This will traverse a * JSONObject / JSONArray recursively to find a matching key, if found * the object with the key is returned. * @param root root object which contains the key searched for * @param key the key for the object to search for * @return the object matching the key */ private Object findObject(Object root, String key) { Object result = null; if (root instanceof JSONObject) { JSONObject rootJSON = (JSONObject)root; if (rootJSON.containsKey(key)) { result = rootJSON.get(key); } else { Iterator children = rootJSON.entrySet().iterator(); while (children.hasNext()) { Map.Entry entry = (Map.Entry)children.next(); Object child = entry.getValue(); if (child instanceof JSONObject || child instanceof JSONArray) { result = findObject(child, key); if (null != result) break; } } } } else if (root instanceof JSONArray) { JSONArray rootJSON = (JSONArray)root; for (Object child : rootJSON) { if (child instanceof JSONObject || child instanceof JSONArray) { result = findObject(child, key); if (null != result) break; } } } return result; } } Creating Business Objects While JCS application can be created without a local database, the local database is required when using ADFbc objects even if database objects are not referred. For this example we will create a "Transient" view object that will be programmatically populated based the data obtained from Taleo REST services. Creating ADFbc objects Choose the "Model" project and navigate "New -> Business Tier : ADF Business Components : View Object". On the "Initialize Business Components Project" choose the local database connection created in previous step. On Step 1 enter "JcsRestDemoVO" on the "Name" and choose "Rows populated programmatically, not based on query": On step 2 create the following attributes: CandId Type: Number Updatable: Always Key Attribute: checked Name Type: String Updatable: Always On steps 3 and 4 accept defaults and click "Next". On step 5 check the "Application Module" checkbox and enter "JcsRestDemoAM" as the name: Click "Finish" to generate the objects. Populating the VO To display the data on the UI the "transient VO" is populated programmatically based on the data obtained from the Taleo REST services. Open the "JcsRestDemoVOImpl.java". Copy / paste the following as the content (for details of the implementation refer to the documentation in the code): import java.sql.ResultSet; import java.util.List; import java.util.ListIterator; import oracle.jbo.server.ViewObjectImpl; import oracle.jbo.server.ViewRowImpl; import oracle.jbo.server.ViewRowSetImpl; // --------------------------------------------------------------------- // --- File generated by Oracle ADF Business Components Design Time. // --- Tue Feb 18 09:40:25 PST 2014 // --- Custom code may be added to this class. // --- Warning: Do not modify method signatures of generated methods. // --------------------------------------------------------------------- public class JcsRestDemoVOImpl extends ViewObjectImpl { /** * This is the default constructor (do not remove). */ public JcsRestDemoVOImpl() { } @Override public void executeQuery() { /** * For some reason we need to reset everything, otherwise * 2nd entry to the UI screen may fail with * "java.util.NoSuchElementException" in createRowFromResultSet * call to "candidates.next()". I am not sure why this is happening * as the Iterator is new and "hasNext" is true at the point * of the execution. My theory is that since the iterator object is * exactly the same the VO cache somehow reuses the iterator including * the pointer that has already exhausted the iterable elements on the * previous run. Working around the issue * here by cleaning out everything on the VO every time before query * is executed on the VO. */ getViewDef().setQuery(null); getViewDef().setSelectClause(null); setQuery(null); this.reset(); this.clearCache(); super.executeQuery(); } /** * executeQueryForCollection - overridden for custom java data source support. */ protected void executeQueryForCollection(Object qc, Object[] params, int noUserParams) { /** * Integrate with the Taleo REST services using TaleoRepository class. * A list of candidates matching a hard coded query is obtained. */ TaleoRepository repository = new TaleoRepository(<company>, <username>, <password>); List<Candidate> candidates = repository.getCandidates(); /** * Store iterator for the candidates as user data on the collection. * This will be used in createRowFromResultSet to create rows based on * the custom iterator. */ ListIterator<Candidate> candidatescIterator = candidates.listIterator(); setUserDataForCollection(qc, candidatescIterator); super.executeQueryForCollection(qc, params, noUserParams); } /** * hasNextForCollection - overridden for custom java data source support. */ protected boolean hasNextForCollection(Object qc) { boolean result = false; /** * Determines whether there are candidates for which to create a row */ ListIterator<Candidate> candidates = (ListIterator<Candidate>)getUserDataForCollection(qc); result = candidates.hasNext(); /** * If all candidates to be created indicate that processing is done */ if (!result) { setFetchCompleteForCollection(qc, true); } return result; } /** * createRowFromResultSet - overridden for custom java data source support. */ protected ViewRowImpl createRowFromResultSet(Object qc, ResultSet resultSet) { /** * Obtain the next candidate from the collection and create a row * for it. */ ListIterator<Candidate> candidates = (ListIterator<Candidate>)getUserDataForCollection(qc); ViewRowImpl row = createNewRowForCollection(qc); try { Candidate candidate = candidates.next(); row.setAttribute("CandId", candidate.getCandId()); row.setAttribute("Name", candidate.getFirstName() + " " + candidate.getLastName()); } catch (Exception e) { e.printStackTrace(); } return row; } /** * getQueryHitCount - overridden for custom java data source support. */ public long getQueryHitCount(ViewRowSetImpl viewRowSet) { /** * For this example this is not implemented rather we always return 0. */ return 0; } } Creating UI Choose the "ViewController" project and navigate "New -> Web Tier : JSF : JSF Page". On the "Create JSF Page" enter "JcsRestDemo" as name and ensure that the "Create as XML document (*.jspx)" is checked. Open "JcsRestDemo.jspx" and navigate to "Data Controls -> JcsRestDemoAMDataControl -> JcsRestDemoVO1" and drag & drop the VO to the "<af:form> " as a "ADF Read-only Table": Accept the defaults in "Edit Table Columns". To execute the query navigate to to "Data Controls -> JcsRestDemoAMDataControl -> JcsRestDemoVO1 -> Operations -> Execute" and drag & drop the operation to the "<af:form> " as a "Button": Deploying to JCS Follow the same steps as documented in previous article"Java Cloud Service ADF Web Application". Once deployed the application can be accessed with URL: https://java-[identity domain].java.[data center].oraclecloudapps.com/JcsRestDemo-ViewController-context-root/faces/JcsRestDemo.jspx The UI displays a list of candidates obtained from the Taleo REST Services: Summary In this article we learned how to integrate with REST services using Jersey library in JCS. In future articles various other integration techniques will be covered.

Read the article

Part 2–Load Testing In The Cloud

- by Tarun Arora

Welcome to Part 2, In Part 1 we discussed the advantages of creating a Test Rig in the cloud, the Azure edge and the Test Rig Topology we want to get to. In Part 2, Let’s start by understanding the components of Azure we’ll be making use of followed by manually putting them together to create the test rig, so… let’s get down dirty start setting up the Test Rig. What Components of Azure will I be using for building the Test Rig in the Cloud? To run the Test Agents we’ll make use of Windows Azure Compute and to enable communication between Test Controller and Test Agents we’ll make use of Windows Azure Connect. Azure Connect The Test Controller is on premise and the Test Agents are in the cloud (How will they talk?). To enable communication between the two, we’ll make use of Windows Azure Connect. With Windows Azure Connect, you can use a simple user interface to configure IPsec protected connections between computers or virtual machines (VMs) in your organization’s network, and roles running in Windows Azure. With this you can now join Windows Azure role instances to your domain, so that you can use your existing methods for domain authentication, name resolution, or other domain-wide maintenance actions. For more details refer to an overview of Windows Azure connect. A very useful video explaining everything you wanted to know about Windows Azure connect. Azure Compute Windows Azure compute provides developers a platform to host and manage applications in Microsoft’s data centres across the globe. A Windows Azure application is built from one or more components called ‘roles.’ Roles come in three different types: Web role, Worker role, and Virtual Machine (VM) role, we’ll be using the Worker role to set up the Test Agents. A very nice blog post discussing the difference between the 3 role types. Developers are free to use the .NET framework or other software that runs on Windows with the Worker role or Web role. Developers can also create applications using languages such as PHP and Java. More on Windows Azure Compute. Each Windows Azure compute instance represents a virtual server... Virtual Machine Size CPU Cores Memory Cost Per Hour Extra Small Shared 768 MB $0.04 Small 1 1.75 GB $0.12 Medium 2 3.50 GB $0.24 Large 4 7.00 GB $0.48 Extra Large 8 14.00 GB $0.96 You might want to review the Windows Azure Pricing FAQ. Let’s Get Started building the Test Rig… Configuration Machine Role Comments VM – 1 Domain Controller for Playpit.com On Premise VM – 2 TFS, Test Controller On Premise VM – 3 Test Agent Cloud In this blog post I would assume that you have the domain, Team Foundation Server and Test Controller Installed and set up already. If not, please refer to the TFS 2010 Installation Guide and this walkthrough on MSDN to set up your Test Controller. You can also download a preconfigured TFS 2010 VM from Brian Keller's blog, Brian also has some great hands on Labs on TFS 2010 that you may want to explore. I. Lets start building VM – 3: The Test Agent Download the Windows Azure SDK and Tools Open Visual Studio and create a new Windows Azure Project using the Cloud Template Choose the Worker Role for reasons explained in the earlier post The WorkerRole.cs implements the Run() and OnStart() methods, no code changes required. You should be able to compile the project and run it in the compute emulator (The compute emulator should have been installed as part of the Windows Azure Toolkit) on your local machine. We will only be making changes to WindowsAzureProject, open ServiceDefinition.csdef. Ensure that the vmsize is small (remember the cost chart above). Import the “Connect” module. I am importing the Connect module because I need to join the Worker role VM to the Playpit domain. <?xml version="1.0" encoding="utf-8"?> <ServiceDefinition name="WindowsAzureProject2" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceDefinition"> <WorkerRole name="WorkerRole1" vmsize="Small"> <Imports> <Import moduleName="Diagnostics" /> <Import moduleName="Connect"/> </Imports> </WorkerRole> </ServiceDefinition> Go to the ServiceConfiguration.Cloud.cscfg and note that settings with key ‘Microsoft.WindowsAzure.Plugins.Connect.%%%%’ have been added to the configuration file. This is because you decided to import the connect module. See the config below. <?xml version="1.0" encoding="utf-8"?> <ServiceConfiguration serviceName="WindowsAzureProject2" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration" osFamily="1" osVersion="*"> <Role name="WorkerRole1"> <Instances count="1" /> <ConfigurationSettings> <Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString" value="UseDevelopmentStorage=true" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.ActivationToken" value="" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.Refresh" value="" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.WaitForConnectivity" value="" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.Upgrade" value="" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.EnableDomainJoin" value="" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainFQDN" value="" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainControllerFQDN" value="" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainAccountName" value="" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainPassword" value="" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainOU" value="" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.Administrators" value="" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainSiteName" value="" /> </ConfigurationSettings> </Role> </ServiceConfiguration> Let’s go step by step and understand all the highlighted parameters and where you can find the values for them. osFamily – By default this is set to 1 (Windows Server 2008 SP2). Change this to 2 if you want the Windows Server 2008 R2 operating system. The Advantage of using osFamily = “2” is that you get Powershell 2.0 rather than Powershell 1.0. In Powershell 2.0 you could simply use “powershell -ExecutionPolicy Unrestricted ./myscript.ps1” and it will work while in Powershell 1.0 you will have to change the registry key by including the following in your command file “reg add HKLM\Software\Microsoft\PowerShell\1\ShellIds\Microsoft.PowerShell /v ExecutionPolicy /d Unrestricted /f” before you can execute any power shell. The other reason you might want to move to os2 is if you wanted IIS 7.5. Activation Token – To enable communication between the on premise machine and the Windows Azure Worker role VM both need to have the same token. Log on to Windows Azure Management Portal, click on Connect, click on Get Activation Token, this should give you the activation token, copy the activation token to the clipboard and paste it in the configuration file. Note – Later in the blog I’ll be showing you how to install connect on the on premise machine. EnableDomainJoin – Set the value to true, ofcourse we want to join the on windows azure worker role VM to the domain. DomainFQDN, DomainControllerFQDN, DomainAccountName, DomainPassword, DomainOU, Administrators – This information is specific to your domain. I have extracted this information from the ‘service manager’ and ‘Active Directory Users and Computers’. Also, i created a new Domain-OU namely ‘CloudInstances’ so all my cloud instances joined to my domain show up here, this is optional. You can encrypt the DomainPassword – refer to the instructions here. Or hold fire, I’ll be covering that when i come to certificates and encryption in the coming section. Now once you have filled all this information up, the configuration file should look something like below, <?xml version="1.0" encoding="utf-8"?> <ServiceConfiguration serviceName="WindowsAzureProject2" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration" osFamily="2" osVersion="*"> <Role name="WorkerRole1"> <Instances count="1" /> <ConfigurationSettings> <Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString" value="UseDevelopmentStorage=true" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.ActivationToken" value="45f55fea-f194-4fbc-b36e-25604faac784" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.Refresh" value="" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.WaitForConnectivity" value="" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.Upgrade" value="" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.EnableDomainJoin" value="true" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainFQDN" value="play.pit.com" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainControllerFQDN" value="WIN-KUDQMQFGQOL.play.pit.com" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainAccountName" value="playpit\Administrator" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainPassword" value="************************" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainOU" value="OU=CloudInstances, DC=Play, DC=Pit, DC=com" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.Administrators" value="Playpit\Administrator" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainSiteName" value="" /> </ConfigurationSettings> </Role> </ServiceConfiguration> Next we will be enabling the Remote Desktop module in to the ServiceDefinition.csdef, we could make changes manually or allow a beautiful wizard to help us make changes. I prefer the second option. So right click on the Windows Azure project and choose Publish Now once you get the publish wizard, if you haven’t already you would be asked to import your Windows Azure subscription, this is simply the Msdn subscription activation key xml. Once you have done click Next to go to the Settings page and check ‘Enable Remote Desktop for all roles’. As soon as you do that you get another pop up asking you the details for the user that you would be logging in with (make sure you enter a reasonable expiry date, you do not want the user account to expire today). Notice the more information tag at the bottom, click that to get access to the certificate section. See screen shot below. From the drop down select the option to create a new certificate In the pop up window enter the friendly name for your certificate. In my case I entered ‘WAC – Test Rig’ and click ok. This will create a new certificate for you. Click on the view button to see the certificate details. Do you see the Thumbprint, this is the value that will go in the config file (very important). Now click on the Copy to File button to copy the certificate, we will need to import the certificate to the windows Azure Management portal later. So, make sure you save it a safe location. Click Finish and enter details of the user you would like to create with permissions for remote desktop access, once you have entered the details on the ‘Remote desktop configuration’ screen click on Ok. From the Publish Windows Azure Wizard screen press Cancel. Cancel because we don’t want to publish the role just yet and Yes because we want to save all the changes in the config file. Now if you go to the ServiceDefinition.csdef file you will see that the RemoteAccess and RemoteForwarder roles have been imported for you. <?xml version="1.0" encoding="utf-8"?> <ServiceDefinition name="WindowsAzureProject2" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceDefinition"> <WorkerRole name="WorkerRole1" vmsize="Small"> <Imports> <Import moduleName="Diagnostics" /> <Import moduleName="Connect" /> <Import moduleName="RemoteAccess" /> <Import moduleName="RemoteForwarder" /> </Imports> </WorkerRole> </ServiceDefinition> Now go to the ServiceConfiguration.Cloud.cscfg file and you see a whole bunch for setting “Microsoft.WindowsAzure.Plugins.RemoteAccess.%%%” values added for you. <?xml version="1.0" encoding="utf-8"?> <ServiceConfiguration serviceName="WindowsAzureProject2" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration" osFamily="2" osVersion="*"> <Role name="WorkerRole1"> <Instances count="1" /> <ConfigurationSettings> <Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString" value="UseDevelopmentStorage=true" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.ActivationToken" value="45f55fea-f194-4fbc-b36e-25604faac784" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.Refresh" value="" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.WaitForConnectivity" value="" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.Upgrade" value="" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.EnableDomainJoin" value="true" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainFQDN" value="play.pit.com" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainControllerFQDN" value="WIN-KUDQMQFGQOL.play.pit.com" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainAccountName" value="playpit\Administrator" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainPassword" value="************************" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainOU" value="OU=CloudInstances, DC=Play, DC=Pit, DC=com" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.Administrators" value="Playpit\Administrator" /> <Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainSiteName" value="" /> <Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.Enabled" value="true" /> <Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.AccountUsername" value="Administrator" /> <Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.AccountEncryptedPassword" value="MIIBnQYJKoZIhvcNAQcDoIIBjjCCAYoCAQAxggFOMIIBSgIBADAyMB4xHDAaBgNVBAMME1dpbmRvd 3MgQXp1cmUgVG9vbHMCEGa+B46voeO5T305N7TSG9QwDQYJKoZIhvcNAQEBBQAEggEABg4ol5Xol66Ip6QKLbAPWdmD4ae ADZ7aKj6fg4D+ATr0DXBllZHG5Umwf+84Sj2nsPeCyrg3ZDQuxrfhSbdnJwuChKV6ukXdGjX0hlowJu/4dfH4jTJC7sBWS AKaEFU7CxvqYEAL1Hf9VPL5fW6HZVmq1z+qmm4ecGKSTOJ20Fptb463wcXgR8CWGa+1w9xqJ7UmmfGeGeCHQ4QGW0IDSBU6ccg vzF2ug8/FY60K1vrWaCYOhKkxD3YBs8U9X/kOB0yQm2Git0d5tFlIPCBT2AC57bgsAYncXfHvPesI0qs7VZyghk8LVa9g5IqaM Cp6cQ7rmY/dLsKBMkDcdBHuCTAzBgkqhkiG9w0BBwEwFAYIKoZIhvcNAwcECDRVifSXbA43gBApNrp40L1VTVZ1iGag+3O1" /> <Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.AccountExpiration" value="2012-11-27T23:59:59.0000000+00:00" /> <Setting name="Microsoft.WindowsAzure.Plugins.RemoteForwarder.Enabled" value="true" /> </ConfigurationSettings> <Certificates> <Certificate name="Microsoft.WindowsAzure.Plugins.RemoteAccess.PasswordEncryption" thumbprint="AA23016CF0BDFC344400B5B82706B608B92E4217" thumbprintAlgorithm="sha1" /> </Certificates> </Role> </ServiceConfiguration> Okay let’s look at them one at a time, Enabled - Yes, we would like to enable Remote Access. AccountUserName – This is the user name you entered while you were on the publish windows azure role screen, as detailed above. AccountEncrytedPassword – Try and decode that, the certificate is used to encrypt the password you specified for the user account. Remember earlier i said, either use the instructions or wait and i’ll be showing you encryption, now the user account i am using for rdp has the same password as my domain password, so i can simply copy the value of the AccountEncryptedPassword to the DomainPassword as well. AccountExpiration – This is the expiration as you specified in the wizard earlier, make sure your account does not expire today. Remote Forwarder – Check out the documentation, below is how I understand it, -- One role in an application that implements a remote desktop connection must import the RemoteForwarder module. The two modules work together to enable the remote desktop connections to role instances. -- If you have multiple roles defined in the service model, it does not matter which role you add the RemoteForwarder module to, but you must add it to only one of the role definitions. Certificate – Remember the certificate thumbprint from the wizard, the on premise machine and windows azure role machine that need to speak to each other must have the same thumbprint. More on that when we install Windows Azure connect Endpoints on the on premise machine. As i said earlier, in this blog post, I’ll be showing you the manual process so i won’t be scripting any star up tasks to install the test agent or register the test agent with the TFS Server. I’ll be showing you all this cool stuff in the next blog post, that’s because it’s important to understand the manual side of it, it becomes easier for you to troubleshoot in case something fails. Having said that, the changes we have made are sufficient to spin up the Windows Azure Worker Role aka Test Agent VM, have it connected with the play.pit.com domain and have remote access enabled on it. Before we deploy the Test Agent VM we need to set up Windows Azure Connect on the TFS Server. II. Windows Azure Connect: Setting up Connect on VM – 2 i.e. TFS & Test Controller Glad you made it so far, now to enable communication between the on premise TFS/Test Controller and Azure-ed Test Agent we need to enable communication. We have set up the Azure connect module in the Test Agent configuration, now the connect end points need to be enabled on the on premise machines, let’s have a look at how we can do this. Log on to VM – 2 running the TFS Server and Test Controller Log on to the Windows Azure Management Portal and click on Virtual Network Click on Virtual Network, if you already have a subscription you should see the below screen shot, if not, you would be asked to complete the subscription first Click on Install Local Endpoints from the top left on the panel and you get a url appended with a token id in it, remember the token i showed you earlier, in theory the token you get here should match the token you added to the Test Agent config file. Copy the url to the clip board and paste it in IE explorer (important, the installation at present only works out of IE and you need to have cookies enabled in order to complete the installation). As stated in the pop up, you can NOT download and run the software later, you need to run it as is, since it contains a token. Once the installation completes you should see the Windows Azure connect icon in the system tray. Right click the Azure Connect icon, choose Diagnostics and refer to this link for diagnostic detail terminology. NOTE – Unfortunately I could not see the Windows Azure connect icon in the system tray, a bit of binging with Google revealed that the azure connect icon is only shown when the ‘Windows Azure Connect Endpoint’ Service is started. So go to services.msc and make sure that the service is started, if not start it, unfortunately again, the service did not start for me on a manual start and i realised that one of the dependant services was disabled, you can look at the service dependencies and start them and then start windows azure connect. Bottom line, you need to start Windows Azure connect service before you can proceed. Please refer here on MSDN for more on Troubleshooting Windows Azure connect. (Follow the next step as well) Now go back to the Windows Azure Management Portal and from Groups and Roles create a new group, lets call it ‘Test Rig’. Make sure you add the VM – 2 (the TFS Server VM where you just installed the endpoint). Now if you go back to the Azure Connect icon in the system tray and click ‘Refresh Policy’ you will notice that the disconnected status of the icon should change to ready for connection. III. Importing Certificate in to Windows Azure Management Portal But before that you need to import the certificate you created in Step I in to the Windows Azure Management Portal. Log on to the Windows Azure Management Portal and click on ‘Hosted Services, Storage Accounts & CDN’ and then ‘Management Certificates’ followed by Add Certificates as shown in the screen shot below Browse to the location where you saved the certificate earlier, remember… Refer to Step I in case you forgot. Now you should be able to see the imported certificate here, make sure the thumbprint of the certificate matches the one you inserted in the config files IV. Publish Windows Azure Worker Role aka Test Agent Having completed I, II and III, you are ready to publish the Test Agent VM – 3 to the cloud. Go to Visual Studio and right click the Windows Azure project and select Publish. Verify the infomration in the wizard, from the advanced settings tab, you can also enabled capture of intellitrace or profiling information. Click Next and Click Publish! From the view menu bar select the Windows Azure Activity Log window. Now you should be able to see the deployment progress in real time. In the Windows Azure Management Portal, you should also be able to see the progress of creation of a new Worker Role. Once the deployment is complete you should be able to RDP (go to run prompt type mstsc and in the pop up the machine name) in to the Test Agent Worker Role VM from the Playpit network using the domain admin user account. In case you are unable to log in to the Test Agent using the domain admin user account it means the process of joining the Test Agent to the domain has failed! But the good news is, because you imported the connect module, you can connect to the Test Agent machine using Windows Azure Management Portal and troubleshoot the reason for failure, you will be able to log in with the user name and password you specified in the config file for the keys ‘RemoteAccess.AccountUsername, RemoteAccess.EncryptedPassword (just that enter the password unencrypted)’, fix it or manually join the machine to the domain. Once you have managed to Join the Test Agent VM to the Domain move to the next step. So, log in to the Test Agent Worker Role VM with the Playpit Domain Administrator and verify that you can log in, the machine is connected to the domain and the connect service is successfully running. If yes, give your self a pat on the back, you are 80% mission accomplished! Go to the Windows Azure Management Portal and click on Virtual Network, click on Groups and Roles and click on Test Rig, click Edit Group, the edit the Test Rig group you created earlier. In the Connect to section, click on Add to select the worker role you have just deployed. Also, check the ‘Allow connections between endpoints in the group’ with this you will enable to communication between test controller and test agents and test agents/test agents. Click Save. Now, you are ready to deploy the Test Agent software on the Worker Role Test Agent VM and configure it to work with the Test Controller. V. Configuring VM – 3: Installing Test Agent and Associating Test Agent to Controller Log in to the Worker Role Test Agent VM that you have just successfully deployed, make sure you log in with the domain administrator account. Download the All Agents software from MSDN, ‘en_visual_studio_agents_2010_x86_x64_dvd_509679.iso’, extract the iso and navigate to where you have extracted the iso. In my case, i have extracted the iso to “C:\Resources\Temp\VsAgentSetup”. Open the Test Agent folder and double click on setup.exe. Once you have installed the Test Agent you should reach the configuration window. If you face any issues installing TFS Test Agent on the VM, refer to the walkthrough on MSDN. Once you have successfully installed the Test Agent software you will need to configure the test agent. Right click the test agent configuration tool and run as a different user. i.e. an Administrator. This is really to run the configuration wizard with elevated privileges (you might have UAC block something's otherwise). In the run options, you can select ‘service’ you do not need to run the agent as interactive un less you are running coded UI tests. I have specified the domain administrator to connect to the TFS Test Controller. In real life, i would never do that, i would create a separate test user service account for this purpose. But for the blog post, we are using the most powerful user so that any policies or restrictions don’t block you. Click the Apply Settings button and you should be all green! If not, the summary usually gives helpful error messages that you can resolve and proceed. As per my experience, you may run in to either a permission or a firewall blocking communication issue. And now the moment of truth! Go to VM –2 open up Visual Studio and from the Test Menu select Manage Test Controller Mission Accomplished! You should be able to see the Test Agent that you have just configured here, VI. Creating and Running Load Tests on your brand new Azure-ed Test Rig I have various blog posts on Performance Testing with Visual Studio Ultimate, you can follow the links and videos below, Blog Posts: - Part 1 – Performance Testing using Visual Studio 2010 Ultimate - Part 2 – Performance Testing using Visual Studio 2010 Ultimate - Part 3 – Performance Testing using Visual Studio 2010 Ultimate Videos: - Test Tools Configuration & Settings in Visual Studio - Why & How to Record Web Performance Tests in Visual Studio Ultimate - Goal Driven Load Testing using Visual Studio Ultimate Now that you have created your load tests, there is one last change you need to make before you can run the tests on your Azure Test Rig, create a new Test settings file, and change the Test Execution method to ‘Remote Execution’ and select the test controller you have configured the Worker Role Test Agent against in our case VM – 2 So, go on, fire off a test run and see the results of the test being executed on the Azur-ed Test Rig. Review and What’s next? A quick recap of the benefits of running the Test Rig in the cloud and what i will be covering in the next blog post AND I would love to hear your feedback! Advantages Utilizing the power of Azure compute to run a heavy virtual user load. Benefiting from the Azure flexibility, destroy Test Agents when not in use, takes < 25 minutes to spin up a new Test Agent. Most important test Network Latency, (network latency and speed of connection are two different things – usually network latency is very hard to test), by placing the Test Agents in Microsoft Data centres around the globe, one can actually test the lag in transferring the bytes not because of a slow connection but because the page has been requested from the other side of the globe. Next Steps The process of spinning up the Test Agents in windows Azure is not 100% automated. I am working on the Worker process and power shell scripts to make the role deployment, unattended install of test agent software and registration of the test agent to the test controller automated. In the next blog post I will show you how to make the complete process unattended and automated. Remember to subscribe to http://feeds.feedburner.com/TarunArora. Hope you enjoyed this post, I would love to hear your feedback! If you have any recommendations on things that I should consider or any questions or feedback, feel free to leave a comment. See you in Part III. Share this post : CodeProject

Read the article

Node.js Adventure - Storage Services and Service Runtime

- by Shaun

When I described on how to host a Node.js application on Windows Azure, one of questions might be raised about how to consume the vary Windows Azure services, such as the storage, service bus, access control, etc.. Interact with windows azure services is available in Node.js through the Windows Azure Node.js SDK, which is a module available in NPM. In this post I would like to describe on how to use Windows Azure Storage (a.k.a. WAS) as well as the service runtime. Consume Windows Azure Storage Let’s firstly have a look on how to consume WAS through Node.js. As we know in the previous post we can host Node.js application on Windows Azure Web Site (a.k.a. WAWS) as well as Windows Azure Cloud Service (a.k.a. WACS). In theory, WAWS is also built on top of WACS worker roles with some more features. Hence in this post I will only demonstrate for hosting in WACS worker role. The Node.js code can be used when consuming WAS when hosted on WAWS. But since there’s no roles in WAWS, the code for consuming service runtime mentioned in the next section cannot be used for WAWS node application. We can use the solution that I created in my last post. Alternatively we can create a new windows azure project in Visual Studio with a worker role, add the “node.exe” and “index.js” and install “express” and “node-sqlserver” modules, make all files as “Copy always”. In order to use windows azure services we need to have Windows Azure Node.js SDK, as knows as a module named “azure” which can be installed through NPM. Once we downloaded and installed, we need to include them in our worker role project and make them as “Copy always”. You can use my “Copy all always” tool mentioned in my last post to update the currently worker role project file. You can also find the source code of this tool here. The source code of Windows Azure SDK for Node.js can be found in its GitHub page. It contains two parts. One is a CLI tool which provides a cross platform command line package for Mac and Linux to manage WAWS and Windows Azure Virtual Machines (a.k.a. WAVM). The other is a library for managing and consuming vary windows azure services includes tables, blobs, queues, service bus and the service runtime. I will not cover all of them but will only demonstrate on how to use tables and service runtime information in this post. You can find the full document of this SDK here. Back to Visual Studio and open the “index.js”, let’s continue our application from the last post, which was working against Windows Azure SQL Database (a.k.a. WASD). The code should looks like this. 1: var express = require("express"); 2: var sql = require("node-sqlserver"); 3: 4: var connectionString = "Driver={SQL Server Native Client 10.0};Server=tcp:ac6271ya9e.database.windows.net,1433;Database=synctile;Uid=shaunxu@ac6271ya9e;Pwd={PASSWORD};Encrypt=yes;Connection Timeout=30;"; 5: var port = 80; 6: 7: var app = express(); 8: 9: app.configure(function () { 10: app.use(express.bodyParser()); 11: }); 12: 13: app.get("/", function (req, res) { 14: sql.open(connectionString, function (err, conn) { 15: if (err) { 16: console.log(err); 17: res.send(500, "Cannot open connection."); 18: } 19: else { 20: conn.queryRaw("SELECT * FROM [Resource]", function (err, results) { 21: if (err) { 22: console.log(err); 23: res.send(500, "Cannot retrieve records."); 24: } 25: else { 26: res.json(results); 27: } 28: }); 29: } 30: }); 31: }); 32: 33: app.get("/text/:key/:culture", function (req, res) { 34: sql.open(connectionString, function (err, conn) { 35: if (err) { 36: console.log(err); 37: res.send(500, "Cannot open connection."); 38: } 39: else { 40: var key = req.params.key; 41: var culture = req.params.culture; 42: var command = "SELECT * FROM [Resource] WHERE [Key] = '" + key + "' AND [Culture] = '" + culture + "'"; 43: conn.queryRaw(command, function (err, results) { 44: if (err) { 45: console.log(err); 46: res.send(500, "Cannot retrieve records."); 47: } 48: else { 49: res.json(results); 50: } 51: }); 52: } 53: }); 54: }); 55: 56: app.get("/sproc/:key/:culture", function (req, res) { 57: sql.open(connectionString, function (err, conn) { 58: if (err) { 59: console.log(err); 60: res.send(500, "Cannot open connection."); 61: } 62: else { 63: var key = req.params.key; 64: var culture = req.params.culture; 65: var command = "EXEC GetItem '" + key + "', '" + culture + "'"; 66: conn.queryRaw(command, function (err, results) { 67: if (err) { 68: console.log(err); 69: res.send(500, "Cannot retrieve records."); 70: } 71: else { 72: res.json(results); 73: } 74: }); 75: } 76: }); 77: }); 78: 79: app.post("/new", function (req, res) { 80: var key = req.body.key; 81: var culture = req.body.culture; 82: var val = req.body.val; 83: 84: sql.open(connectionString, function (err, conn) { 85: if (err) { 86: console.log(err); 87: res.send(500, "Cannot open connection."); 88: } 89: else { 90: var command = "INSERT INTO [Resource] VALUES ('" + key + "', '" + culture + "', N'" + val + "')"; 91: conn.queryRaw(command, function (err, results) { 92: if (err) { 93: console.log(err); 94: res.send(500, "Cannot retrieve records."); 95: } 96: else { 97: res.send(200, "Inserted Successful"); 98: } 99: }); 100: } 101: }); 102: }); 103: 104: app.listen(port); Now let’s create a new function, copy the records from WASD to table service. 1. Delete the table named “resource”. 2. Create a new table named “resource”. These 2 steps ensures that we have an empty table. 3. Load all records from the “resource” table in WASD. 4. For each records loaded from WASD, insert them into the table one by one. 5. Prompt to user when finished. In order to use table service we need the storage account and key, which can be found from the developer portal. Just select the storage account and click the Manage Keys button. Then create two local variants in our Node.js application for the storage account name and key. Since we need to use WAS we need to import the azure module. Also I created another variant stored the table name. In order to work with table service I need to create the storage client for table service. This is very similar as the Windows Azure SDK for .NET. As the code below I created a new variant named “client” and use “createTableService”, specified my storage account name and key. 1: var azure = require("azure"); 2: var storageAccountName = "synctile"; 3: var storageAccountKey = "/cOy9L7xysXOgPYU9FjDvjrRAhaMX/5tnOpcjqloPNDJYucbgTy7MOrAW7CbUg6PjaDdmyl+6pkwUnKETsPVNw=="; 4: var tableName = "resource"; 5: var client = azure.createTableService(storageAccountName, storageAccountKey); Now create a new function for URL “/was/init” so that we can trigger it through browser. Then in this function we will firstly load all records from WASD. 1: app.get("/was/init", function (req, res) { 2: // load all records from windows azure sql database 3: sql.open(connectionString, function (err, conn) { 4: if (err) { 5: console.log(err); 6: res.send(500, "Cannot open connection."); 7: } 8: else { 9: conn.queryRaw("SELECT * FROM [Resource]", function (err, results) { 10: if (err) { 11: console.log(err); 12: res.send(500, "Cannot retrieve records."); 13: } 14: else { 15: if (results.rows.length > 0) { 16: // begin to transform the records into table service 17: } 18: } 19: }); 20: } 21: }); 22: }); When we succeed loaded all records we can start to transform them into table service. First I need to recreate the table in table service. This can be done by deleting and creating the table through table client I had just created previously. 1: app.get("/was/init", function (req, res) { 2: // load all records from windows azure sql database 3: sql.open(connectionString, function (err, conn) { 4: if (err) { 5: console.log(err); 6: res.send(500, "Cannot open connection."); 7: } 8: else { 9: conn.queryRaw("SELECT * FROM [Resource]", function (err, results) { 10: if (err) { 11: console.log(err); 12: res.send(500, "Cannot retrieve records."); 13: } 14: else { 15: if (results.rows.length > 0) { 16: // begin to transform the records into table service 17: // recreate the table named 'resource' 18: client.deleteTable(tableName, function (error) { 19: client.createTableIfNotExists(tableName, function (error) { 20: if (error) { 21: error["target"] = "createTableIfNotExists"; 22: res.send(500, error); 23: } 24: else { 25: // transform the records 26: } 27: }); 28: }); 29: } 30: } 31: }); 32: } 33: }); 34: }); As you can see, the azure SDK provide its methods in callback pattern. In fact, almost all modules in Node.js use the callback pattern. For example, when I deleted a table I invoked “deleteTable” method, provided the name of the table and a callback function which will be performed when the table had been deleted or failed. Underlying, the azure module will perform the table deletion operation in POSIX async threads pool asynchronously. And once it’s done the callback function will be performed. This is the reason we need to nest the table creation code inside the deletion function. If we perform the table creation code after the deletion code then they will be invoked in parallel. Next, for each records in WASD I created an entity and then insert into the table service. Finally I send the response to the browser. Can you find a bug in the code below? I will describe it later in this post. 1: app.get("/was/init", function (req, res) { 2: // load all records from windows azure sql database 3: sql.open(connectionString, function (err, conn) { 4: if (err) { 5: console.log(err); 6: res.send(500, "Cannot open connection."); 7: } 8: else { 9: conn.queryRaw("SELECT * FROM [Resource]", function (err, results) { 10: if (err) { 11: console.log(err); 12: res.send(500, "Cannot retrieve records."); 13: } 14: else { 15: if (results.rows.length > 0) { 16: // begin to transform the records into table service 17: // recreate the table named 'resource' 18: client.deleteTable(tableName, function (error) { 19: client.createTableIfNotExists(tableName, function (error) { 20: if (error) { 21: error["target"] = "createTableIfNotExists"; 22: res.send(500, error); 23: } 24: else { 25: // transform the records 26: for (var i = 0; i < results.rows.length; i++) { 27: var entity = { 28: "PartitionKey": results.rows[i][1], 29: "RowKey": results.rows[i][0], 30: "Value": results.rows[i][2] 31: }; 32: client.insertEntity(tableName, entity, function (error) { 33: if (error) { 34: error["target"] = "insertEntity"; 35: res.send(500, error); 36: } 37: else { 38: console.log("entity inserted"); 39: } 40: }); 41: } 42: // send the 43: console.log("all done"); 44: res.send(200, "All done!"); 45: } 46: }); 47: }); 48: } 49: } 50: }); 51: } 52: }); 53: }); Now we can publish it to the cloud and have a try. But normally we’d better test it at the local emulator first. In Node.js SDK there are three build-in properties which provides the account name, key and host address for local storage emulator. We can use them to initialize our table service client. We also need to change the SQL connection string to let it use my local database. The code will be changed as below. 1: // windows azure sql database 2: //var connectionString = "Driver={SQL Server Native Client 10.0};Server=tcp:ac6271ya9e.database.windows.net,1433;Database=synctile;Uid=shaunxu@ac6271ya9e;Pwd=eszqu94XZY;Encrypt=yes;Connection Timeout=30;"; 3: // sql server 4: var connectionString = "Driver={SQL Server Native Client 11.0};Server={.};Database={Caspar};Trusted_Connection={Yes};"; 5: 6: var azure = require("azure"); 7: var storageAccountName = "synctile"; 8: var storageAccountKey = "/cOy9L7xysXOgPYU9FjDvjrRAhaMX/5tnOpcjqloPNDJYucbgTy7MOrAW7CbUg6PjaDdmyl+6pkwUnKETsPVNw=="; 9: var tableName = "resource"; 10: // windows azure storage 11: //var client = azure.createTableService(storageAccountName, storageAccountKey); 12: // local storage emulator 13: var client = azure.createTableService(azure.ServiceClient.DEVSTORE_STORAGE_ACCOUNT, azure.ServiceClient.DEVSTORE_STORAGE_ACCESS_KEY, azure.ServiceClient.DEVSTORE_TABLE_HOST); Now let’s run the application and navigate to “localhost:12345/was/init” as I hosted it on port 12345. We can find it transformed the data from my local database to local table service. Everything looks fine. But there is a bug in my code. If we have a look on the Node.js command window we will find that it sent response before all records had been inserted, which is not what I expected. The reason is that, as I mentioned before, Node.js perform all IO operations in non-blocking model. When we inserted the records we executed the table service insert method in parallel, and the operation of sending response was also executed in parallel, even though I wrote it at the end of my logic. The correct logic should be, when all entities had been copied to table service with no error, then I will send response to the browser, otherwise I should send error message to the browser. To do so I need to import another module named “async”, which helps us to coordinate our asynchronous code. Install the module and import it at the beginning of the code. Then we can use its “forEach” method for the asynchronous code of inserting table entities. The first argument of “forEach” is the array that will be performed. The second argument is the operation for each items in the array. And the third argument will be invoked then all items had been performed or any errors occurred. Here we can send our response to browser. 1: app.get("/was/init", function (req, res) { 2: // load all records from windows azure sql database 3: sql.open(connectionString, function (err, conn) { 4: if (err) { 5: console.log(err); 6: res.send(500, "Cannot open connection."); 7: } 8: else { 9: conn.queryRaw("SELECT * FROM [Resource]", function (err, results) { 10: if (err) { 11: console.log(err); 12: res.send(500, "Cannot retrieve records."); 13: } 14: else { 15: if (results.rows.length > 0) { 16: // begin to transform the records into table service 17: // recreate the table named 'resource' 18: client.deleteTable(tableName, function (error) { 19: client.createTableIfNotExists(tableName, function (error) { 20: if (error) { 21: error["target"] = "createTableIfNotExists"; 22: res.send(500, error); 23: } 24: else { 25: async.forEach(results.rows, 26: // transform the records 27: function (row, callback) { 28: var entity = { 29: "PartitionKey": row[1], 30: "RowKey": row[0], 31: "Value": row[2] 32: }; 33: client.insertEntity(tableName, entity, function (error) { 34: if (error) { 35: callback(error); 36: } 37: else { 38: console.log("entity inserted."); 39: callback(null); 40: } 41: }); 42: }, 43: // send reponse 44: function (error) { 45: if (error) { 46: error["target"] = "insertEntity"; 47: res.send(500, error); 48: } 49: else { 50: console.log("all done"); 51: res.send(200, "All done!"); 52: } 53: } 54: ); 55: } 56: }); 57: }); 58: } 59: } 60: }); 61: } 62: }); 63: }); Run it locally and now we can find the response was sent after all entities had been inserted. Query entities against table service is simple as well. Just use the “queryEntity” method from the table service client and providing the partition key and row key. We can also provide a complex query criteria as well, for example the code here. In the code below I queried an entity by the partition key and row key, and return the proper localization value in response. 1: app.get("/was/:key/:culture", function (req, res) { 2: var key = req.params.key; 3: var culture = req.params.culture; 4: client.queryEntity(tableName, culture, key, function (error, entity) { 5: if (error) { 6: res.send(500, error); 7: } 8: else { 9: res.json(entity); 10: } 11: }); 12: }); And then tested it on local emulator. Finally if we want to publish this application to the cloud we should change the database connection string and storage account. For more information about how to consume blob and queue service, as well as the service bus please refer to the MSDN page. Consume Service Runtime As I mentioned above, before we published our application to the cloud we need to change the connection string and account information in our code. But if you had played with WACS you should have known that the service runtime provides the ability to retrieve configuration settings, endpoints and local resource information at runtime. Which means we can have these values defined in CSCFG and CSDEF files and then the runtime should be able to retrieve the proper values. For example we can add some role settings though the property window of the role, specify the connection string and storage account for cloud and local. And the can also use the endpoint which defined in role environment to our Node.js application. In Node.js SDK we can get an object from “azure.RoleEnvironment”, which provides the functionalities to retrieve the configuration settings and endpoints, etc.. In the code below I defined the connection string variants and then use the SDK to retrieve and initialize the table client. 1: var connectionString = ""; 2: var storageAccountName = ""; 3: var storageAccountKey = ""; 4: var tableName = ""; 5: var client; 6: 7: azure.RoleEnvironment.getConfigurationSettings(function (error, settings) { 8: if (error) { 9: console.log("ERROR: getConfigurationSettings"); 10: console.log(JSON.stringify(error)); 11: } 12: else { 13: console.log(JSON.stringify(settings)); 14: connectionString = settings["SqlConnectionString"]; 15: storageAccountName = settings["StorageAccountName"]; 16: storageAccountKey = settings["StorageAccountKey"]; 17: tableName = settings["TableName"]; 18: 19: console.log("connectionString = %s", connectionString); 20: console.log("storageAccountName = %s", storageAccountName); 21: console.log("storageAccountKey = %s", storageAccountKey); 22: console.log("tableName = %s", tableName); 23: 24: client = azure.createTableService(storageAccountName, storageAccountKey); 25: } 26: }); In this way we don’t need to amend the code for the configurations between local and cloud environment since the service runtime will take care of it. At the end of the code we will listen the application on the port retrieved from SDK as well. 1: azure.RoleEnvironment.getCurrentRoleInstance(function (error, instance) { 2: if (error) { 3: console.log("ERROR: getCurrentRoleInstance"); 4: console.log(JSON.stringify(error)); 5: } 6: else { 7: console.log(JSON.stringify(instance)); 8: if (instance["endpoints"] && instance["endpoints"]["nodejs"]) { 9: var endpoint = instance["endpoints"]["nodejs"]; 10: app.listen(endpoint["port"]); 11: } 12: else { 13: app.listen(8080); 14: } 15: } 16: }); But if we tested the application right now we will find that it cannot retrieve any values from service runtime. This is because by default, the entry point of this role was defined to the worker role class. In windows azure environment the service runtime will open a named pipeline to the entry point instance, so that it can connect to the runtime and retrieve values. But in this case, since the entry point was worker role and the Node.js was opened inside the role, the named pipeline was established between our worker role class and service runtime, so our Node.js application cannot use it. To fix this problem we need to open the CSDEF file under the azure project, add a new element named Runtime. Then add an element named EntryPoint which specify the Node.js command line. So that the Node.js application will have the connection to service runtime, then it’s able to read the configurations. Start the Node.js at local emulator we can find it retrieved the connections, storage account for local. And if we publish our application to azure then it works with WASD and storage service through the configurations for cloud. Summary In this post I demonstrated how to use Windows Azure SDK for Node.js to interact with storage service, especially the table service. I also demonstrated on how to use WACS service runtime, how to retrieve the configuration settings and the endpoint information. And in order to make the service runtime available to my Node.js application I need to create an entry point element in CSDEF file and set “node.exe” as the entry point. I used five posts to introduce and demonstrate on how to run a Node.js application on Windows platform, how to use Windows Azure Web Site and Windows Azure Cloud Service worker role to host our Node.js application. I also described how to work with other services provided by Windows Azure platform through Windows Azure SDK for Node.js. Node.js is a very new and young network application platform. But since it’s very simple and easy to learn and deploy, as well as, it utilizes single thread non-blocking IO model, Node.js became more and more popular on web application and web service development especially for those IO sensitive projects. And as Node.js is very good at scaling-out, it’s more useful on cloud computing platform. Use Node.js on Windows platform is new, too. The modules for SQL database and Windows Azure SDK are still under development and enhancement. It doesn’t support SQL parameter in “node-sqlserver”. It does support using storage connection string to create the storage client in “azure”. But Microsoft is working on make them easier to use, working on add more features and functionalities. PS, you can download the source code here. You can download the source code of my “Copy all always” tool here. Hope this helps, Shaun All documents and related graphics, codes are provided "AS IS" without warranty of any kind. Copyright © Shaun Ziyan Xu. This work is licensed under the Creative Commons License.

Read the article

ASP.NET MVC2 Implementing Custom RoleManager problem

- by ile

To create a custom membership provider I followed these instructions: http://stackoverflow.com/questions/2771094/asp-net-mvc2-custom-membership and these: http://mattwrock.com/post/2009/10/14/Implementing-custom-Membership-Provider-and-Role-Provider-for-Authinticating-ASPNET-MVC-Applications.aspx So far, I've managed to implement custom membership provider and that part works fine. RoleManager still needs some modifications... Project structure: SAMembershipProvider.cs: public class SAMembershipProvider : MembershipProvider { #region - Properties - private int NewPasswordLength { get; set; } private string ConnectionString { get; set; } public bool enablePasswordReset { get; set; } public bool enablePasswordRetrieval { get; set; } public bool requiresQuestionAndAnswer { get; set; } public bool requiresUniqueEmail { get; set; } public int maxInvalidPasswordAttempts { get; set; } public int passwordAttemptWindow { get; set; } public MembershipPasswordFormat passwordFormat { get; set; } public int minRequiredNonAlphanumericCharacters { get; set; } public int minRequiredPasswordLength { get; set; } public string passwordStrengthRegularExpression { get; set; } public override string ApplicationName { get; set; } public override bool EnablePasswordRetrieval { get { return enablePasswordRetrieval; } } public override bool EnablePasswordReset { get { return enablePasswordReset; } } public override bool RequiresQuestionAndAnswer { get { return requiresQuestionAndAnswer; } } public override int MaxInvalidPasswordAttempts { get { return maxInvalidPasswordAttempts; } } public override int PasswordAttemptWindow { get { return passwordAttemptWindow; } } public override bool RequiresUniqueEmail { get { return requiresUniqueEmail; } } public override MembershipPasswordFormat PasswordFormat { get { return passwordFormat; } } public override int MinRequiredPasswordLength { get { return minRequiredPasswordLength; } } public override int MinRequiredNonAlphanumericCharacters { get { return minRequiredNonAlphanumericCharacters; } } public override string PasswordStrengthRegularExpression { get { return passwordStrengthRegularExpression; } } #endregion #region - Methods - public override void Initialize(string name, NameValueCollection config) { throw new NotImplementedException(); } public override bool ChangePassword(string username, string oldPassword, string newPassword) { throw new NotImplementedException(); } public override bool ChangePasswordQuestionAndAnswer(string username, string password, string newPasswordQuestion, string newPasswordAnswer) { throw new NotImplementedException(); } public override MembershipUser CreateUser(string username, string password, string email, string passwordQuestion, string passwordAnswer, bool isApproved, object providerUserKey, out MembershipCreateStatus status) { throw new NotImplementedException(); } public override bool DeleteUser(string username, bool deleteAllRelatedData) { throw new NotImplementedException(); } public override MembershipUserCollection FindUsersByEmail(string emailToMatch, int pageIndex, int pageSize, out int totalRecords) { throw new NotImplementedException(); } public override MembershipUserCollection FindUsersByName(string usernameToMatch, int pageIndex, int pageSize, out int totalRecords) { throw new NotImplementedException(); } public override MembershipUserCollection GetAllUsers(int pageIndex, int pageSize, out int totalRecords) { throw new NotImplementedException(); } public override int GetNumberOfUsersOnline() { throw new NotImplementedException(); } public override string GetPassword(string username, string answer) { throw new NotImplementedException(); } public override MembershipUser GetUser(object providerUserKey, bool userIsOnline) { throw new NotImplementedException(); } public override MembershipUser GetUser(string username, bool userIsOnline) { throw new NotImplementedException(); } public override string GetUserNameByEmail(string email) { throw new NotImplementedException(); } protected override void OnValidatingPassword(ValidatePasswordEventArgs e) { base.OnValidatingPassword(e); } public override string ResetPassword(string username, string answer) { throw new NotImplementedException(); } public override bool UnlockUser(string userName) { throw new NotImplementedException(); } public override void UpdateUser(MembershipUser user) { throw new NotImplementedException(); } public override bool ValidateUser(string username, string password) { AccountRepository accountRepository = new AccountRepository(); var user = accountRepository.GetUser(username); if (string.IsNullOrEmpty(password.Trim())) return false; if (user == null) return false; //string hash = EncryptPassword(password); var email = user.Email; var pass = user.Password; if (user == null) return false; if (pass == password) { //User = user; return true; } return false; } #endregion protected string EncryptPassword(string password) { //we use codepage 1252 because that is what sql server uses byte[] pwdBytes = Encoding.GetEncoding(1252).GetBytes(password); byte[] hashBytes = System.Security.Cryptography.MD5.Create().ComputeHash(pwdBytes); return Encoding.GetEncoding(1252).GetString(hashBytes); } } SARoleProvider.cs public class SARoleProvider : RoleProvider { AccountRepository accountRepository = new AccountRepository(); public override bool IsUserInRole(string username, string roleName) { return true; } public override string ApplicationName { get { throw new NotImplementedException(); } set { throw new NotImplementedException(); } } public override void AddUsersToRoles(string[] usernames, string[] roleNames) { throw new NotImplementedException(); } public override void RemoveUsersFromRoles(string[] usernames, string[] roleNames) { throw new NotImplementedException(); } public override void CreateRole(string roleName) { throw new NotImplementedException(); } public override bool DeleteRole(string roleName, bool throwOnPopulatedRole) { throw new NotImplementedException(); } public override bool RoleExists(string roleName) { throw new NotImplementedException(); } public override string[] GetRolesForUser(string username) { int rolesCount = 0; IQueryable<RoleViewModel> rolesNames; try { // get roles for this user from DB... rolesNames = accountRepository.GetRolesForUser(username); rolesCount = rolesNames.Count(); } catch (Exception ex) { throw ex; } string[] roles = new string[rolesCount]; int counter = 0; foreach (var item in rolesNames) { roles[counter] = item.RoleName.ToString(); counter++; } return roles; } public override string[] GetUsersInRole(string roleName) { throw new NotImplementedException(); } public override string[] FindUsersInRole(string roleName, string usernameToMatch) { throw new NotImplementedException(); } public override string[] GetAllRoles() { throw new NotImplementedException(); } } AccountRepository.cs public class RoleViewModel { public string RoleName { get; set; } } public class AccountRepository { private DB db = new DB(); public User GetUser(string email) { return db.Users.SingleOrDefault(d => d.Email == email); } public IQueryable<RoleViewModel> GetRolesForUser(string email) { var result = ( from role in db.Roles join user in db.Users on role.RoleID equals user.RoleID where user.Email == email select new RoleViewModel { RoleName = role.Name }); return result; } } webconfig <membership defaultProvider="SAMembershipProvider" userIsOnlineTimeWindow="15"> <providers> <clear/> <add name="SAMembershipProvider" type="SA_Contacts.Membership.SAMembershipProvider, SA_Contacts" connectionStringName ="ShinyAntConnectionString" /> </providers> </membership> <roleManager defaultProvider="SARoleProvider" enabled="true" cacheRolesInCookie="true"> <providers> <clear/> <add name="SARoleProvider" type="SA_Contacts.Membership.SARoleProvider" connectionStringName ="ShinyAntConnectionString" /> </providers> </roleManager> AccountController.cs: public class AccountController : Controller { SAMembershipProvider provider = new SAMembershipProvider(); AccountRepository accountRepository = new AccountRepository(); public AccountController() { } public ActionResult LogOn() { return View(); } [AcceptVerbs(HttpVerbs.Post)] public ActionResult LogOn(string userName, string password, string returnUrl) { if (!ValidateLogOn(userName, password)) { return View(); } var user = accountRepository.GetUser(userName); var userFullName = user.FirstName + " " + user.LastName; FormsAuthentication.SetAuthCookie(userFullName, false); if (!String.IsNullOrEmpty(returnUrl) && returnUrl != "/") { return Redirect(returnUrl); } else { return RedirectToAction("Index", "Home"); } } public ActionResult LogOff() { FormsAuthentication.SignOut(); return RedirectToAction("Index", "Home"); } private bool ValidateLogOn(string userName, string password) { if (String.IsNullOrEmpty(userName)) { ModelState.AddModelError("username", "You must specify a username."); } if (String.IsNullOrEmpty(password)) { ModelState.AddModelError("password", "You must specify a password."); } if (!provider.ValidateUser(userName, password)) { ModelState.AddModelError("_FORM", "The username or password provided is incorrect."); } return ModelState.IsValid; } } In some testing controller I have following: [Authorize] public class ContactsController : Controller { SAMembershipProvider saMembershipProvider = new SAMembershipProvider(); SARoleProvider saRoleProvider = new SARoleProvider(); // // GET: /Contact/ public ActionResult Index() { string[] roleNames = Roles.GetRolesForUser("[email protected]"); // Outputs admin ViewData["r1"] = roleNames[0].ToString(); // Outputs True // I'm not even sure if this method is the same as the one below ViewData["r2"] = Roles.IsUserInRole("[email protected]", roleNames[0].ToString()); // Outputs True ViewData["r3"] = saRoleProvider.IsUserInRole("[email protected]", "admin"); return View(); } If I use attribute [Authorize] then everything works ok, but if I use [Authorize(Roles="admin")] then user is always rejected, like he is not in role. Any idea of what could be wrong here? Thanks in advance, Ile

Read the article

Data Modeling: Logical Modeling Exercise

- by swisscheese

In trying to learn the art of data storage I have been trying to take in as much solid information as possible. PerformanceDBA posted some really helpful tutorials/examples in the following posts among others: is my data normalized? and Relational table naming convention. I already asked a subset question of this model here. So to make sure I understood the concepts he presented and I have seen elsewhere I wanted to take things a step or two further and see if I am grasping the concepts. Hence the purpose of this post, which hopefully others can also learn from. Everything I present is conceptual to me and for learning rather than applying it in some production system. It would be cool to get some input from PerformanceDBA also since I used his models to get started, but I appreciate all input given from anyone. As I am new to databases and especially modeling I will be the first to admit that I may not always ask the right questions, explain my thoughts clearly, or use the right verbage due to lack of expertise on the subject. So please keep that in mind and feel free to steer me in the right direction if I head off track. If there is enough interest in this I would like to take this from the logical to physical phases to show the evolution of the process and share it here on Stack. I will keep this thread for the Logical Diagram though and start new one for the additional steps. For my understanding I will be building a MySQL DB in the end to run some tests and see if what I came up with actually works. Here is the list of things that I want to capture in this conceptual model. Edit for V1.2 The purpose of this is to list Bands, their members, and the Events that they will be appearing at, as well as offer music and other merchandise for sale Members will be able to match up with friends Members can write reviews on the Bands, their music, and their events. There can only be one review per member on a given item, although they can edit their reviews and history will be maintained. BandMembers will have the chance to write a single Comment on Reviews about the Band they are associated with. Collectively as a Band only one Comment is allowed per Review. Members can then rate all Reviews and Comments but only once per given instance Members can select their favorite Bands, music, Merchandise, and Events Bands, Songs, and Events will be categorized into the type of Genre that they are and then further subcategorized into a SubGenre if necessary. It is ok for a Band or Event to fall into more then one Genre/SubGenre combination. Event date, time, and location will be posted for a given band and members can show that they will be attending the Event. An Event can be comprised of more than one Band, and multiple Events can take place at a single location on the same day Every party will be tied to at least one address and address history shall be maintained. Each party could also be tied to more then one address at a time (i.e. billing, shipping, physical) There will be stored profiles for Bands, BandMembers, and general members. So there it is, maybe a bit involved but could be a great learning tool for many hopefully as the process evolves and input is given by the community. Any input? EDIT v1.1 In response to PerformanceDBA U.3) That means no merchandise other than Band merchandise in the database. Correct ? That was my original thought but you got me thinking. Maybe the site would want to sell its own merchandise or even other merchandise from the bands. Not sure a mod to make for that. Would it require an entire rework of the Catalog section or just the identifying relationship that exists with the Band? Attempted a mod to sell both complete albums or song. Either way they would both be in electronic format only available for download. That is why I listed an Album as being comprised of Songs rather then 2 separate entities. U.5) I understand what you bring up about the circular relation with Favorite. I would like to get to this “It is either one Entity with some form of differentiation (FavoriteType) which identifies its treatment” but how to is not clear to me. What am I missing here? u.6) “Business Rules This is probably the only area you are weak in.” Thanks for the honest response. I will readdress these but I hope to clear up some confusion in my head first with the responses I have posted back to you. Q.1) Yes I would like to have Accepted, Rejected, and Blocked. I am not sure what you are referring to as to how this would change the logical model? Q.2) A person does not have to be a User. They can exist only as a BandMember. Is that what you are asking? Minor Issue Zero, One, or More…Oops I admit I forgot to give this attention when building the model. I am submitting this version as is and will address in a future version. I need to read up more on Constraint Checking to make sure I am understanding things. M.4) Depends if you envision OrderPurchase in the future. Can you expand as to what you mean here? EDIT V1.2 In response to PerformanceDBA input... Lessons learned. I was mixing the concept of Identifying / Non-Identifying and Cardinality (i.e. Genre / SubGenre), and doing so inconsistently to make things worse. Associative Tables are not required in Logical Diagrams as their many-to-many relationships can be depicted and then expanded in the Physical Model. I was overlooking the Cardinality in a lot of the relationships The importance of reading through relationships using effective Verb Phrases to reassure I am modeling what I want to accomplish. U.2) In the concept of this model it is only required to track a Venue as a location for an Event. No further data needs to be collected. With that being said Events will take place on a given EventDate and will be hosted at a Venue. Venues will host multiple events and possibly multiple events on a given date. In my new model my thinking was that EventDate is already tied to Event . Therefore, Venue will not need a relationship with EventDate. The 5th and 6th bullets you have listed under U.2) leave me questioning my thinking though. Am I missing something here? U.3) Is it time to move the link between Item and Band up to Item and Party instead? With the current design I don't see a possibility to sell merchandise not tied to the band as you have brought up. U.5) I left as per your input rather than making it a discrete Supertype/Subtype Relationship as I don’t see a benefit of having that type of roll up. Additional Revisions AR.1) After going through the exercise for FavoriteItem, I feel that Item to Review requires a many-to-many relationship so that is indicated. Necessary? Ok here we go for v1.3 I took a few days on this version, going back and forth with my design. Once the logical process is complete, as I want to see if I am on the right track, I will go through in depth what I had learned and the troubles I faced as a beginner going through this process. The big point for this version was it took throwing in some Keys to help see what I was missing in the past. Going through the process of doing a matrix proved to be of great help also. Regardless of anything, if it wasn't for the input given by PerformanceDBA I would still be a lost soul wondering in the dark. Who knows my current design might reaffirm that I still am, but I have learned a lot so I am know I at least have a flashlight in my hand. At this point in time I admit that I am still confused about identifying and non-identifying relationships. In my model I had to use non-identifying relationships with non nulls just to join the relationships I wanted to model. In reading a lot on the subject there seems to be a lot of disagreement and indecisiveness on the subject so I did what I thought represented the right things in my model. When to force (identifying) and when to be free (non-identifying)? Anyone have inputs? EDIT V1.4 Ok took the V1.3 inputs and cleaned things up for this V1.4 Currently working on a V1.5 to include attributes.

Read the article

WiX major upgrade refuses to replace existing file!

- by Joshua

Hello! I have inherited this project with a WiX installer, and am required to make this version usefully upgrade the previous one! My problem comes in replacing the database files with new versions. No, the problem is not that they are locked, I can replace them manually, and in fact now ONE of them is replaced, while the other is not. Please, please tell me what I'm doing wrong here. I've tried several other solutions (including registry keys as KeyPath instead of CompanionFile) but nothing is quite working. Here is (most of) the code of the .WXS file: <Product Id='$(var.ProductCode)' UpgradeCode='$(var.UpgradeCode)' Name="Pathways" Version='$(var.ProductVersion)' Manufacturer='$(var.Manufacturer)' Language='1033'> <Package Id="*" Description="Pathways Directory Software" InstallerVersion="301" Compressed="yes" /> <WixVariable Id="WixUILicenseRtf" Value="License.rtf" /> <Media Id="1" Cabinet="Pathways.cab" EmbedCab="yes" /> <Upgrade Id="$(var.UpgradeCode)"> <UpgradeVersion OnlyDetect="no" Maximum="$(var.ProductVersion)" IncludeMaximum="no" Language="1033" Property="OLDAPPFOUND" /> <UpgradeVersion Minimum="$(var.ProductVersion)" IncludeMinimum="yes" OnlyDetect="no" Language="1033" Property="NEWAPPFOUND" /> </Upgrade> <Property Id="ALLUSERS">2</Property>  <Directory Id="TARGETDIR" Name="SourceDir">  <Directory Id="ProgramFilesFolder"> <Directory Id="INSTALLDIR" Name="Pathways"/> </Directory>  <Directory Id="CommonAppDataFolder" Name="CommonAppData"> <Directory Id="CommonAppDataPathways" Name="Pathways" /> </Directory>  <Directory Id="ProgramMenuFolder"> <Directory Id="ProgramsMenuPathwaysFolder" Name="Pathways" /> </Directory>  <Directory Id="DesktopFolder" /> </Directory> <Icon Id="PathwaysIcon" SourceFile="\\Fileserver\Release\Pathways\Latest\Release\Pathways.exe" />  <DirectoryRef Id="INSTALLDIR"> <Component Id="Application" Guid="EEE4EB55-A515-4872-A4A5-06D6AB4A06A6"> <File Id="pathwaysExe" Name="Pathways.exe" DiskId="1" Source="\\Fileserver\Release\Pathways\Latest\Release\Pathways.exe" Vital="yes" KeyPath="yes" Assembly=".net" AssemblyApplication="pathwaysExe" AssemblyManifest="pathwaysExe">  </File> <File Id="pathwaysChm" Name="Pathways.chm" DiskId="1" Source="\\fileserver\Release\Pathways\Dependencies\Pathways.chm" /> <File Id="publicKeyXml" ShortName="RSAPUBLI.XML" Name="RSAPublicKey.xml" DiskId="1" Source="\\fileserver\Release\Pathways\Dependencies\RSAPublicKey.xml" Vital="yes" /> <File Id="staticListsXml" ShortName="STATICLI.XML" Name="StaticLists.xml" DiskId="1" Source="\\fileserver\Release\Pathways\Dependencies\StaticLists.xml" Vital="yes" /> <File Id="axInteropMapPointDll" ShortName="AXMPOINT.DLL" Name="AxInterop.MapPoint.dll" DiskId="1" Source="\\fileserver\Release\Pathways\Dependencies\AxInterop.MapPoint.dll" Vital="yes" /> <File Id="interopMapPointDll" ShortName="INMPOINT.DLL" Name="Interop.MapPoint.dll" DiskId="1" Source="\\fileserver\Release\Pathways\Dependencies\Interop.MapPoint.dll" Vital="yes" /> <File Id="mapPointDll" ShortName="MAPPOINT.DLL" Name="MapPoint.dll" DiskId="1" Source="\\fileserver\Release\Pathways\Dependencies\Interop.MapPoint.dll" Vital="yes" /> <File Id="devExpressData63Dll" ShortName="DAAT63.DLL" Name="DevExpress.Data.v6.3.dll" DiskId="1" Source="\\fileserver\Release\Pathways\Dependencies\DevExpress.Data.v6.3.dll" Vital="yes" /> <File Id="devExpressUtils63Dll" ShortName="UTILS63.DLL" Name="DevExpress.Utils.v6.3.dll" DiskId="1" Source="\\fileserver\Release\Pathways\Dependencies\DevExpress.Utils.v6.3.dll" Vital="yes" /> <File Id="devExpressXtraBars63Dll" ShortName="BARS63.DLL" Name="DevExpress.XtraBars.v6.3.dll" DiskId="1" Source="\\fileserver\Release\Pathways\Dependencies\DevExpress.XtraBars.v6.3.dll" Vital="yes" /> <File Id="devExpressXtraNavBar63Dll" ShortName="NAVBAR63.DLL" Name="DevExpress.XtraNavBar.v6.3.dll" DiskId="1" Source="\\fileserver\Release\Pathways\Dependencies\DevExpress.XtraNavBar.v6.3.dll" Vital="yes" /> <File Id="devExpressXtraCharts63Dll" ShortName="CHARTS63.DLL" Name="DevExpress.XtraCharts.v6.3.dll" DiskId="1" Source="\\fileserver\Release\Pathways\Dependencies\DevExpress.XtraCharts.v6.3.dll" Vital="yes" /> <File Id="devExpressXtraEditors63Dll" ShortName="EDITOR63.DLL" Name="DevExpress.XtraEditors.v6.3.dll" DiskId="1" Source="\\fileserver\Release\Pathways\Dependencies\DevExpress.XtraEditors.v6.3.dll" Vital="yes" /> <File Id="devExpressXtraPrinting63Dll" ShortName="PRINT63.DLL" Name="DevExpress.XtraPrinting.v6.3.dll" DiskId="1" Source="\\fileserver\Release\Pathways\Dependencies\DevExpress.XtraPrinting.v6.3.dll" Vital="yes" /> <File Id="devExpressXtraReports63Dll" ShortName="REPORT63.DLL" Name="DevExpress.XtraReports.v6.3.dll" DiskId="1" Source="\\fileserver\Release\Pathways\Dependencies\DevExpress.XtraReports.v6.3.dll" Vital="yes" /> <File Id="devExpressXtraRichTextEdit63Dll" ShortName="RICHTE63.DLL" Name="DevExpress.XtraRichTextEdit.v6.3.dll" DiskId="1" Source="\\fileserver\Release\Pathways\Dependencies\DevExpress.XtraRichTextEdit.v6.3.dll" Vital="yes" /> <RegistryValue Id="PathwaysInstallDir" Root="HKLM" Key="Software\Tribal Data Resources\Pathways" Name="InstallDir" Action="write" Type="string" Value="[INSTALLDIR]" /> </Component> </DirectoryRef>  <DirectoryRef Id="CommonAppDataPathways"> <Component Id="CommonAppDataPathwaysFolderComponent" Guid="087C6F14-E87E-4B57-A7FA-C03FC8488E0D"> <CreateFolder> <Permission User="Everyone" GenericAll="yes" /> </CreateFolder> <RemoveFolder Id="CommonAppDataPathways" On="uninstall" />  </Component> <Component Id="Settings" Guid="A3513208-4F12-4496-B609-197812B4A953" NeverOverwrite="yes"> <File Id="settingsXml" KeyPath="yes" ShortName="SETTINGS.XML" Name="Settings.xml" DiskId="1" Source="\\fileserver\Release\Pathways\Dependencies\Settings\settings.xml" Vital="yes" /> </Component> <Component Id="Database" Guid="1D8756EF-FD6C-49BC-8400-299492E8C65D" >  <File Id="pathwaysMdf" Name="Pathways.mdf" DiskId="1" Source="\\fileserver\Shared\Databases\Pathways\SystemDBs\Pathways.mdf" CompanionFile="pathwaysExe" Vital="yes"/> <File Id="pathwaysLdf" Name="Pathways_log.ldf" DiskId="1" Source="\\fileserver\Shared\Databases\Pathways\SystemDBs\Pathways.ldf" CompanionFile="pathwaysExe" Vital="yes"/> </Component>  </DirectoryRef>  <DirectoryRef Id="DesktopFolder"> <Component Id="DesktopShortcutComponent" Guid="1BF412BA-9C6B-460D-80ED-8388AC66703F"> <Shortcut Id="DesktopShortcut" Target="[INSTALLDIR]Pathways.exe" Name="Pathways" Description="Pathways Tribal Directory" Icon="PathwaysIcon" Show="normal" WorkingDirectory="INSTALLDIR" /> <RegistryValue Root="HKCU" Key="Software\TDR\Pathways" Name="installed" Type="integer" Value="1" KeyPath="yes"/> </Component> </DirectoryRef> <DirectoryRef Id ="ProgramsMenuPathwaysFolder"> <Component Id="ProgramsMenuShortcutComponent" Guid="83A18245-4C22-4CDC-94E0-B480F80A407D"> <Shortcut Id="ProgramsMenuShortcut" Target="[INSTALLDIR]Pathways.exe" Name="Pathways" Icon="PathwaysIcon" Show="normal" WorkingDirectory="INSTALLDIR" /> <RemoveFolder Id="ProgramsMenuPathwaysFolder" On="uninstall"/> <RegistryValue Root="HKCU" Key="Software\TDR\Pathways" Name="installed" Type="integer" Value="1" KeyPath="yes"/> </Component> </DirectoryRef> <Feature Id="App" Title="Pathways Application" Level="1" Description="Pathways software" Display="expand" ConfigurableDirectory="INSTALLDIR" Absent="disallow" AllowAdvertise="no" InstallDefault="local"> <ComponentRef Id="Application" /> <ComponentRef Id="CommonAppDataPathwaysFolderComponent" /> <ComponentRef Id="Settings"/> <ComponentRef Id="ProgramsMenuShortcutComponent" /> <Feature Id="Shortcuts" Title="Desktop Shortcut" Level="1" Absent="allow" AllowAdvertise="no" InstallDefault="local"> <ComponentRef Id="DesktopShortcutComponent" /> </Feature> </Feature> <Feature Id="Data" Title="Database" Level="1" Absent="allow" AllowAdvertise="no" InstallDefault="local"> <ComponentRef Id="Database" /> </Feature> <UIRef Id ="WixUI_FeatureTree"/> <UIRef Id="WixUI_ErrorProgressText"/> <UI> <Error Id="2000">There is a later version of this program installed.</Error> </UI> <CustomAction Id="NewerVersionDetected" Error="2000" /> <InstallExecuteSequence> <RemoveExistingProducts After="InstallFinalize"/> </InstallExecuteSequence> </Product> Running this installer attempting the upgrade from the previous version ALMOST WORKS. The file that is giving me trouble is the one called "PathwaysMdf". Even though it's Component code is EXACTLY the same as the PathwaysLdf file, that file is replaced, while the MDF is NOT. You can see, commented out, some of the other things I've attempted, some from suggestions on stackoverflow. The entire log file from running the upgrade is located at: http://pastebin.com/ppjhq6Wi THANK YOU! Joshua

Read the article

Returning a list from a function in Python

- by Jasper

Hi, I'm creating a game for my sister, and I want a function to return a list variable, so I can pass it to another variable. The relevant code is as follows: def startNewGame(): while 1: #Introduction: print print """Hello, You will now be guided through the setup process. There are 7 steps to this. You can cancel setup at any time by typing 'cancelSetup' Thankyou""" #Step 1 (Name): print print """Step 1 of 7: Type in a name for your PotatoHead: """ inputPHName = raw_input('|Enter Name:|') if inputPHName == 'cancelSetup': sys.exit() #Step 2 (Gender): print print """Step 2 of 7: Choose the gender of your PotatoHead: input either 'm' or 'f' """ inputPHGender = raw_input('|Enter Gender:|') if inputPHGender == 'cancelSetup': sys.exit() #Step 3 (Colour): print print """Step 3 of 7: Choose the colour your PotatoHead will be: Only Red, Blue, Green and Yellow are currently supported """ inputPHColour = raw_input('|Enter Colour:|') if inputPHColour == 'cancelSetup': sys.exit() #Step 4 (Favourite Thing): print print """Step 4 of 7: Type your PotatoHead's favourite thing: """ inputPHFavThing = raw_input('|Enter Favourite Thing:|') if inputPHFavThing == 'cancelSetup': sys.exit() # Step 5 (First Toy): print print """Step 5 of 7: Choose a first toy for your PotatoHead: """ inputPHFirstToy = raw_input('|Enter First Toy:|') if inputPHFirstToy == 'cancelSetup': sys.exit() #Step 6 (Check stats): while 1: print print """Step 6 of 7: Check the following details to make sure that they are correct: """ print print """Name:\t\t\t""" + inputPHName + """ Gender:\t\t\t""" + inputPHGender + """ Colour:\t\t\t""" + inputPHColour + """ Favourite Thing:\t""" + inputPHFavThing + """ First Toy:\t\t""" + inputPHFirstToy + """ """ print print "Enter 'y' or 'n'" inputMCheckStats = raw_input('|Is this information correct?|') if inputMCheckStats == 'cancelSetup': sys.exit() elif inputMCheckStats == 'y': break elif inputMCheckStats == 'n': print "Re-enter info: ..." print break else: "The value you entered was incorrect, please re-enter your choice" if inputMCheckStats == 'y': break #Step 7 (Define variables for the creation of the PotatoHead): MFCreatePH = [] print print """Step 7 of 7: Your PotatoHead will now be created... Creating variables... """ MFCreatePH = [inputPHName, inputPHGender, inputPHColour, inputPHFavThing, inputPHFirstToy] time.sleep(1) print "inputPHName" print time.sleep(1) print "inputPHFirstToy" print return MFCreatePH print "Your PotatoHead varibles have been successfully created!" Then it is passed to another function that was imported from another module from potatohead import * ... welcomeMessage() MCreatePH = startGame() myPotatoHead = PotatoHead(MCreatePH) the code for the PotatoHead object is in the potatohead.py module which was imported above, and is as follows: class PotatoHead: #Initialise the PotatoHead object: def __init__(self, data): self.data = data #Takes the data from the start new game function - see main.py #Defines the PotatoHead starting attributes: self.name = data[0] self.gender = data[1] self.colour = data[2] self.favouriteThing = data[3] self.firstToy = data[4] self.age = '0.0' self.education = [self.eduScience, self.eduEnglish, self.eduMaths] = '0.0', '0.0', '0.0' self.fitness = '0.0' self.happiness = '10.0' self.health = '10.0' self.hunger = '0.0' self.tiredness = 'Not in this version' self.toys = [] self.toys.append(self.firstToy) self.time = '0' #Sets data lists for saving, loading and general use: self.phData = (self.name, self.gender, self.colour, self.favouriteThing, self.firstToy) self.phAdvData = (self.name, self.gender, self.colour, self.favouriteThing, self.firstToy, self.age, self.education, self.fitness, self.happiness, self.health, self.hunger, self.tiredness, self.toys) However, when I run the program this error appears: Traceback (most recent call last): File "/Users/Jasper/Documents/Programming/Potato Head Game/Current/main.py", line 158, in <module> myPotatoHead = PotatoHead(MCreatePH) File "/Users/Jasper/Documents/Programming/Potato Head Game/Current/potatohead.py", line 15, in __init__ self.name = data[0] TypeError: 'NoneType' object is unsubscriptable What am i doing wrong? -----EDIT----- The program finishes as so: Step 7 of 7: Your PotatoHead will now be created... Creating variables... inputPHName inputPHFirstToy Then it goes to the Tracback -----EDIT2----- This is the EXACT code I'm running in its entirety: #+--------------------------------------+# #| main.py |# #| A main module for the Potato Head |# #| Game to pull the other modules |# #| together and control through user |# #| input |# #| Author: |# #| Date Created / Modified: |# #| 3/2/10 | 20/2/10 |# #+--------------------------------------+# Tested: No #Import the required modules: import time import random import sys from potatohead import * from toy import * #Start the Game: def welcomeMessage(): print "----- START NEW GAME -----------------------" print "==Print Welcome Message==" print "loading... \t loading... \t loading..." time.sleep(1) print "loading..." time.sleep(1) print "LOADED..." print; print; print; print """Hello, Welcome to the Potato Head Game. In this game you can create a Potato Head, and look after it, like a Virtual Pet. This game is constantly being updated and expanded. Please look out for updates. """ #Choose whether to start a new game or load a previously saved game: def startGame(): while 1: print "--------------------" print """ Choose an option: New_Game or Load_Game """ startGameInput = raw_input('>>> >') if startGameInput == 'New_Game': startNewGame() break elif startGameInput == 'Load_Game': print "This function is not yet supported" print "Try Again" print else: print "You must have mistyped the command: Type either 'New_Game' or 'Load_Game'" print #Set the new game up: def startNewGame(): while 1: #Introduction: print print """Hello, You will now be guided through the setup process. There are 7 steps to this. You can cancel setup at any time by typing 'cancelSetup' Thankyou""" #Step 1 (Name): print print """Step 1 of 7: Type in a name for your PotatoHead: """ inputPHName = raw_input('|Enter Name:|') if inputPHName == 'cancelSetup': sys.exit() #Step 2 (Gender): print print """Step 2 of 7: Choose the gender of your PotatoHead: input either 'm' or 'f' """ inputPHGender = raw_input('|Enter Gender:|') if inputPHGender == 'cancelSetup': sys.exit() #Step 3 (Colour): print print """Step 3 of 7: Choose the colour your PotatoHead will be: Only Red, Blue, Green and Yellow are currently supported """ inputPHColour = raw_input('|Enter Colour:|') if inputPHColour == 'cancelSetup': sys.exit() #Step 4 (Favourite Thing): print print """Step 4 of 7: Type your PotatoHead's favourite thing: """ inputPHFavThing = raw_input('|Enter Favourite Thing:|') if inputPHFavThing == 'cancelSetup': sys.exit() # Step 5 (First Toy): print print """Step 5 of 7: Choose a first toy for your PotatoHead: """ inputPHFirstToy = raw_input('|Enter First Toy:|') if inputPHFirstToy == 'cancelSetup': sys.exit() #Step 6 (Check stats): while 1: print print """Step 6 of 7: Check the following details to make sure that they are correct: """ print print """Name:\t\t\t""" + inputPHName + """ Gender:\t\t\t""" + inputPHGender + """ Colour:\t\t\t""" + inputPHColour + """ Favourite Thing:\t""" + inputPHFavThing + """ First Toy:\t\t""" + inputPHFirstToy + """ """ print print "Enter 'y' or 'n'" inputMCheckStats = raw_input('|Is this information correct?|') if inputMCheckStats == 'cancelSetup': sys.exit() elif inputMCheckStats == 'y': break elif inputMCheckStats == 'n': print "Re-enter info: ..." print break else: "The value you entered was incorrect, please re-enter your choice" if inputMCheckStats == 'y': break #Step 7 (Define variables for the creation of the PotatoHead): MFCreatePH = [] print print """Step 7 of 7: Your PotatoHead will now be created... Creating variables... """ MFCreatePH = [inputPHName, inputPHGender, inputPHColour, inputPHFavThing, inputPHFirstToy] time.sleep(1) print "inputPHName" print time.sleep(1) print "inputPHFirstToy" print return MFCreatePH print "Your PotatoHead varibles have been successfully created!" #Run Program: welcomeMessage() MCreatePH = startGame() myPotatoHead = PotatoHead(MCreatePH) The potatohead.py module is as follows: #+--------------------------------------+# #| potatohead.py |# #| A module for the Potato Head Game |# #| Author: |# #| Date Created / Modified: |# #| 24/1/10 | 24/1/10 |# #+--------------------------------------+# Tested: Yes (24/1/10) #Create the PotatoHead class: class PotatoHead: #Initialise the PotatoHead object: def __init__(self, data): self.data = data #Takes the data from the start new game function - see main.py #Defines the PotatoHead starting attributes: self.name = data[0] self.gender = data[1] self.colour = data[2] self.favouriteThing = data[3] self.firstToy = data[4] self.age = '0.0' self.education = [self.eduScience, self.eduEnglish, self.eduMaths] = '0.0', '0.0', '0.0' self.fitness = '0.0' self.happiness = '10.0' self.health = '10.0' self.hunger = '0.0' self.tiredness = 'Not in this version' self.toys = [] self.toys.append(self.firstToy) self.time = '0' #Sets data lists for saving, loading and general use: self.phData = (self.name, self.gender, self.colour, self.favouriteThing, self.firstToy) self.phAdvData = (self.name, self.gender, self.colour, self.favouriteThing, self.firstToy, self.age, self.education, self.fitness, self.happiness, self.health, self.hunger, self.tiredness, self.toys) #Define the phStats variable, enabling easy display of PotatoHead attributes: def phDefStats(self): self.phStats = """Your Potato Head's Stats are as follows: ---------------------------------------- Name: \t\t""" + self.name + """ Gender: \t\t""" + self.gender + """ Colour: \t\t""" + self.colour + """ Favourite Thing: \t""" + self.favouriteThing + """ First Toy: \t""" + self.firstToy + """ Age: \t\t""" + self.age + """ Education: \t""" + str(float(self.eduScience) + float(self.eduEnglish) + float(self.eduMaths)) + """ -> Science: \t""" + self.eduScience + """ -> English: \t""" + self.eduEnglish + """ -> Maths: \t""" + self.eduMaths + """ Fitness: \t""" + self.fitness + """ Happiness: \t""" + self.happiness + """ Health: \t""" + self.health + """ Hunger: \t""" + self.hunger + """ Tiredness: \t""" + self.tiredness + """ Toys: \t\t""" + str(self.toys) + """ Time: \t\t""" + self.time + """ """ #Change the PotatoHead's favourite thing: def phChangeFavouriteThing(self, newFavouriteThing): self.favouriteThing = newFavouriteThing phChangeFavouriteThingMsg = "Your Potato Head's favourite thing is " + self.favouriteThing + "." #"Feed" the Potato Head i.e. Reduce the 'self.hunger' attribute's value: def phFeed(self): if float(self.hunger) >=3.0: self.hunger = str(float(self.hunger) - 3.0) elif float(self.hunger) < 3.0: self.hunger = '0.0' self.time = str(int(self.time) + 1) #Pass time #"Exercise" the Potato Head if between the ages of 5 and 25: def phExercise(self): if float(self.age) < 5.1 or float(self.age) > 25.1: print "This Potato Head is either too young or too old for this activity!" else: if float(self.fitness) <= 8.0: self.fitness = str(float(self.fitness) + 2.0) elif float(self.fitness) > 8.0: self.fitness = '10.0' self.time = str(int(self.time) + 1) #Pass time #"Teach" the Potato Head: def phTeach(self, subject): if subject == 'Science': if float(self.eduScience) <= 9.0: self.eduScience = str(float(self.eduScience) + 1.0) elif float(self.eduScience) > 9.0 and float(self.eduScience) < 10.0: self.eduScience = '10.0' elif float(self.eduScience) == 10.0: print "Your Potato Head has gained the highest level of qualifications in this subject! It cannot learn any more!" elif subject == 'English': if float(self.eduEnglish) <= 9.0: self.eduEnglish = str(float(self.eduEnglish) + 1.0) elif float(self.eduEnglish) > 9.0 and float(self.eduEnglish) < 10.0: self.eduEnglish = '10.0' elif float(self.eduEnglish) == 10.0: print "Your Potato Head has gained the highest level of qualifications in this subject! It cannot learn any more!" elif subject == 'Maths': if float(self.eduMaths) <= 9.0: self.eduMaths = str(float(self.eduMaths) + 1.0) elif float(self.eduMaths) > 9.0 and float(self.eduMaths) < 10.0: self.eduMaths = '10.0' elif float(self.eduMaths) == 10.0: print "Your Potato Head has gained the highest level of qualifications in this subject! It cannot learn any more!" else: print "That subject is not an option..." print "Please choose either Science, English or Maths" self.time = str(int(self.time) + 1) #Pass time #Increase Health: def phGoToDoctor(self): self.health = '10.0' self.time = str(int(self.time) + 1) #Pass time #Sleep: Age, change stats: #(Time Passes) def phSleep(self): self.time = '0' #Resets time for next 'day' (can do more things next day) #Increase hunger: if float(self.hunger) <= 5.0: self.hunger = str(float(self.hunger) + 5.0) elif float(self.hunger) > 5.0: self.hunger = '10.0' #Lower Fitness: if float(self.fitness) >= 0.5: self.fitness = str(float(self.fitness) - 0.5) elif float(self.fitness) < 0.5: self.fitness = '0.0' #Lower Health: if float(self.health) >= 0.5: self.health = str(float(self.health) - 0.5) elif float(self.health) < 0.5: self.health = '0.0' #Lower Happiness: if float(self.happiness) >= 2.0: self.happiness = str(float(self.happiness) - 2.0) elif float(self.happiness) < 2.0: self.happiness = '0.0' #Increase the Potato Head's age: self.age = str(float(self.age) + 0.1) The game is still under development - There may be parts of modules that aren't complete, but I don't think they're causing the problem

Search Results

Search found 10804 results on 433 pages for 'attribute keys'.

Page 431/433 | < Previous Page | 427 428 429 430 431 432 433 | Next Page >

- by eerok512

- by tugbucket

- by Adam Nygate

- by Sriram

- by Jacob

- by Jared

- by Iamjon

- by StarDust

- by user1437820

- by chacham15

- by Tiny

- by I82Much

- by James

- by Chris Wilson

- by bob_cobb

- by anon-user0

- by Ralf Westphal

- by Rick Strahl

- by Jani Rautiainen

- by Tarun Arora

- by Shaun

- by ile

- by swisscheese

- by Joshua

- by Jasper

< Previous Page | 427 428 429 430 431 432 433 | Next Page >