Search Results

Search found 2313 results on 93 pages for 'continuous delivery'.

Page 93/93 | < Previous Page | 89 90 91 92 93 

  • Why Java servlet can't get Paypal IPN messages everytime ?

    - by Frank
    I have a Java servlet running on my notebook with Windows Vista, I set up a static IP, did port forwarding and registered for a free DDNS service, now my servlet is running, I gave the url to Paypal to send me IPN messages, I went on to it's sandbox site got to the test tools page, tried to send test messages by clicking the "Send IPN" button, most of the time it would fail, the error is : "IPN delivery failed. Unable to connect to the specified URL. Please verify the URL and try again." But maybe 1 in 10 times, it might be successful and my servlet would get the message, and I looked at the messages I got, they are in correct format. So I called Paypal asking why, he said I shouldn't run the servlet on my notebook, in stead I should run it on the web server, but I told him my ISP doesn't support Java on their server, and since I did all the above steps, shouldn't it be the same to run the servlet on my notebook ? He said his test showed he couldn't get to my servlet, but I asked why maybe 1 in 10 times it could get through ? If there is something wrong with running it on my notebook, then 100% times it should fail, am I correct on this point ? But anyway he said that's all he could do, and I should troubleshoot it myself. The servlet looks like this : import java.io.*; import java.net.*; import javax.servlet.*; import javax.servlet.http.*; import java.util.*; public class PayPal_Servlet extends HttpServlet { static boolean Debug=true; static String PayPal_Url="https://www.paypal.com/cgi-bin/webscr",Sandbox_Url="https://www.sandbox.paypal.com/cgi-bin/webscr", Dir_License_Messages="C:/Dir_License_Messages/"; static TransparencyExample Transparency_Example; static PayPal_Message_To_License_File_Worker PayPal_message_to_license_file_worker; // Initializes the servlet. public void init(ServletConfig config) throws ServletException { super.init(config); if (!new File(Dir_License_Messages).exists()) new File(Dir_License_Messages).mkdirs(); System.gc(); } /** Processes requests for both HTTP <code>GET</code> and <code>POST</code> methods. * @param request servlet request * @param response servlet response */ protected void processRequest(HttpServletRequest request,HttpServletResponse response) throws ServletException,IOException { // Read post from PayPal system and add 'cmd' Enumeration en=request.getParameterNames(); String str="cmd=_notify-validate"; while (en.hasMoreElements()) { String paramName=(String)en.nextElement(); String paramValue=request.getParameter(paramName); str=str+"&"+paramName+"="+URLEncoder.encode(paramValue); } // Post back to PayPal system to validate // NOTE: change http: to https: in the following URL to verify using SSL (for increased security). // using HTTPS requires either Java 1.4 or greater, or Java Secure Socket Extension (JSSE) and configured for older versions. URL u=new URL(Debug?Sandbox_Url:PayPal_Url); URLConnection uc=u.openConnection(); uc.setDoOutput(true); uc.setRequestProperty("Content-Type","application/x-www-form-urlencoded"); PrintWriter pw=new PrintWriter(uc.getOutputStream()); pw.println(str); pw.close(); BufferedReader in=new BufferedReader(new InputStreamReader(uc.getInputStream())); String res=in.readLine(); in.close(); // Assign posted variables to local variables String itemName=request.getParameter("item_name"); String itemNumber=request.getParameter("item_number"); String paymentStatus=request.getParameter("payment_status"); String paymentAmount=request.getParameter("mc_gross"); String paymentCurrency=request.getParameter("mc_currency"); String txnId=request.getParameter("txn_id"); String receiverEmail=request.getParameter("receiver_email"); String payerEmail=request.getParameter("payer_email"); if (res.equals("VERIFIED")) // Check notification validation { // check that paymentStatus=Completed // check that txnId has not been previously processed // check that receiverEmail is your Primary PayPal email // check that paymentAmount/paymentCurrency are correct // process payment } else if (res.equals("INVALID")) // Log for investigation { } else // Log for error { } // =========================================================================== if (txnId!=null) { Write_File_Safe_Fast(Dir_License_Messages+txnId+".txt",new StringBuffer(str.replace("&","\n")),false); } // =========================================================================== String Message_File_List[]=Tool_Lib.Get_File_List_From_Dir(Dir_License_Messages); response.setContentType("text/html"); PrintWriter out=response.getWriter(); String title="Reading All Request Parameters",Name="",Value; out.println("<Html><Head><Title>"+title+"</Title></Head>\n<Body Bgcolor=\"#FDF5E6\">\n<H1 Align=Center>"+title+"</H1>\n"+ "<Table Border=1 Align=Center>\n"+"<Tr Bgcolor=\"#FFAD00\"><Th>Parameter Name</Th><Th>Parameter Value(s) Messages = "+Message_File_List.length+"</Th></Tr>"); Enumeration paramNames=request.getParameterNames(); while(paramNames.hasMoreElements()) { String paramName=(String)paramNames.nextElement(); out.print("<Tr><Td>"+paramName+"</Td><Td>"); String[] paramValues=request.getParameterValues(paramName); if (paramValues.length == 1) { String paramValue=paramValues[0]; if (paramValue.length() == 0) out.print("<I>No Value</I>"); else { out.println(paramValue+"</Td></Tr>"); // Out("paramName = "+paramName+" paramValue = "+paramValue); // if (paramName.startsWith("Name")) Name=paramValue; // else if (paramName.startsWith("Value")) Write_File_Safe_Fast("C:/Dir_Data/"+Name,new StringBuffer(paramValue),false); } } else { out.println("<Ul>"); for (int i=0;i<paramValues.length;i++) out.println("<Li>"+paramValues[i]); out.println("</Ul></Td</Tr>"); } } out.println("</Table>\n</Body></Html>"); } /** Handles the HTTP <code>GET</code> method. * @param request servlet request * @param response servlet response */ protected void doGet(HttpServletRequest request,HttpServletResponse response) throws ServletException,IOException { processRequest(request,response); } /** Handles the HTTP <code>POST</code> method. * @param request servlet request * @param response servlet response */ protected void doPost(HttpServletRequest request,HttpServletResponse response) throws ServletException,IOException { processRequest(request,response); } // Returns a short description of the servlet. public String getServletInfo() { return "Short description"; } // Destroys the servlet. public void destroy() { System.gc(); } public static void Write_File_Safe_Fast(String File_Path,StringBuffer Str_Buf,boolean Append) { FileOutputStream fos=null; BufferedOutputStream bos=null; try { fos=new FileOutputStream(File_Path,Append); bos=new BufferedOutputStream(fos); for (int j=0;j<Str_Buf.length();j++) bos.write(Str_Buf.charAt(j)); } catch (Exception e) { e.printStackTrace(); } finally { try { if (bos!=null) { bos.close(); bos=null; } if (fos!=null) { fos.close(); fos=null; } } catch (Exception ex) { ex.printStackTrace(); } } System.gc(); } } I use Netbean6.7 to develop the servlet, and the code was from Paypal's JSP sample code, what can I do to debug the problem ?

    Read the article

  • CSS layout problem on Firefox with filling space between end of left column and footer

    - by Jean
    Basically, the left column is supposed to extend to the footer with the continuous red color. However, in Firefox on pages with lots of text, the column does not extend to the footer and leaves a large white gap--see site: http://library.luhs.org/JHSII/about.html I've tried readjusting the heights, creating the sticky footer, and other things I've read about on this site. So I admit that I'm stumped, and what's really odd is that the layout seems to work in IE as there is no white space! I didn't create the site, but I recently inherited it and trying to work through the mess Any help is much appreciated, here's the CSS #html,body{ margin:0; padding:0; border:0; height:100%; } #body{ background:#ffffff; min-width:965px; text-align:center; width: 600px; font: Geneva, Arial, Helvetica, sans-serif; } #.style7{ clear:both; height:1px; overflow:hidden; line-height:1%; font-size:0px; margin-bottom:-1px; } #fullheightcontainer{ margin-left:auto; margin-right:auto; text-align:left; position:relative; width:965px; height:100%; } #wrapper{ min-height:100%; height:100%; background:#660000; background-color: #660000; background-repeat: repeat; } #wrapp\65 r{ height:auto; } # html wrapper{ height:100%; } #outer{ z-index:1; position:relative; margin-left:150px; width:815px; background:#FFFFFF; height:100%; background-color: #FFFFFF; } #left{ width:151px; float:left; display:inline; position:relative; margin-left:-150px; } padding: 20px; border: 0; margin: 0 0 0 240px *>html #left{width:150px;} #container-left{ width:150px; color: #CCCCCC; } * html #left{margin-right:-3px;} #center{ width:800px; float:right; display:inline; margin-left:-1px; } #clearheadercenter{ height:125px; overflow:hidden; } #clearfootercenter{ height:50px; overflow:hidden; } #footer{ z-index:1; position:relative; clear: both; width:965px; height:50px; overflow:hidden; margin-top:-50px; background-color: #660000; } #subfooter1{ background:#FFFFCC; text-align:left; margin-left:150px; height:50px; } #header{ z-index:1; position:absolute; top:0px; width:815px; margin-left:150px; height:100px; overflow:hidden; background-color: #660000; } #subheader1{ background:#FFFFCC; text-align:center; height:70px; } #gfx_bg_middle{ top:0px; position:absolute; height:100%; overflow:hidden; width:815px; margin-left:150px; background:#FFFFFF; } # html #gfx_bg_middle{ display:none; } #floatingnav { margin: 5px 10px 5px 5px; padding: 0px 5px 5px; float: right; font: .75em/1.35em Geneva, Arial, Helvetica, sans-serif; height: 600px; width: 300px; } #floatingnav a { color: #630; } #floatingnav ul { margin-top: -5; } #.floatright { float: right; margin: 0 0 10px 10px; border: 1px solid #666; padding: 2px; } #outer{ word-wrap:break-word; } #table.s1 { border-width: medium; border-spacing: 2px; border-style: none; border-color: rgb(85, 0, 0); border-collapse: collapse; background-color: white; } #table.s1 th { border-width: medium; padding: 2px; border-style: groove; border-color: red; background-color: white; -moz-border-radius: 0px 0px 0px 0px; } #table.s1 td { border-width: medium; padding: 2px; border-style: groove; border-color: #660000; background-color: #FFFFFF; -moz-border-radius: 0px 0px 0px 0px; } #a:link { color: #000066; } #a:visited { color: #000066; } #p.sample { font-family: serif; font-style: normal; font-variant: normal; font-weight: normal; font-size: medium; line-height: 100%; word-spacing: normal; letter-spacing: normal; text-decoration: none; text-transform: none; text-align: left; text-indent: 0ex; }

    Read the article

  • How do I get .NET to garbage collect aggressively?

    - by mmr
    I have an application that is used in image processing, and I find myself typically allocating arrays in the 4000x4000 ushort size, as well as the occasional float and the like. Currently, the .NET framework tends to crash in this app apparently randomly, almost always with an out of memory error. 32mb is not a huge declaration, but if .NET is fragmenting memory, then it's very possible that such large continuous allocations aren't behaving as expected. Is there a way to tell the garbage collector to be more aggressive, or to defrag memory (if that's the problem)? I realize that there's the GC.Collect and GC.WaitForPendingFinalizers calls, and I've sprinkled them pretty liberally through my code, but I'm still getting the errors. It may be because I'm calling dll routines that use native code a lot, but I'm not sure. I've gone over that C++ code, and make sure that any memory I declare I delete, but still I get these C# crashes, so I'm pretty sure it's not there. I wonder if the C++ calls could be interfering with the GC, making it leave behind memory because it once interacted with a native call-- is that possible? If so, can I turn that functionality off? EDIT: Here is some very specific code that will cause the crash. According to this SO question, I do not need to be disposing of the BitmapSource objects here. Here is the naive version, no GC.Collects in it. It generally crashes on iteration 4 to 10 of the undo procedure. This code replaces the constructor in a blank WPF project, since I'm using WPF. I do the wackiness with the bitmapsource because of the limitations I explained in my answer to @dthorpe below as well as the requirements listed in this SO question. public partial class Window1 : Window { public Window1() { InitializeComponent(); //Attempts to create an OOM crash //to do so, mimic minute croppings of an 'image' (ushort array), and then undoing the crops int theRows = 4000, currRows; int theColumns = 4000, currCols; int theMaxChange = 30; int i; List<ushort[]> theList = new List<ushort[]>();//the list of images in the undo/redo stack byte[] displayBuffer = null;//the buffer used as a bitmap source BitmapSource theSource = null; for (i = 0; i < theMaxChange; i++) { currRows = theRows - i; currCols = theColumns - i; theList.Add(new ushort[(theRows - i) * (theColumns - i)]); displayBuffer = new byte[theList[i].Length]; theSource = BitmapSource.Create(currCols, currRows, 96, 96, PixelFormats.Gray8, null, displayBuffer, (currCols * PixelFormats.Gray8.BitsPerPixel + 7) / 8); System.Console.WriteLine("Got to change " + i.ToString()); System.Threading.Thread.Sleep(100); } //should get here. If not, then theMaxChange is too large. //Now, go back up the undo stack. for (i = theMaxChange - 1; i >= 0; i--) { displayBuffer = new byte[theList[i].Length]; theSource = BitmapSource.Create((theColumns - i), (theRows - i), 96, 96, PixelFormats.Gray8, null, displayBuffer, ((theColumns - i) * PixelFormats.Gray8.BitsPerPixel + 7) / 8); System.Console.WriteLine("Got to undo change " + i.ToString()); System.Threading.Thread.Sleep(100); } } } Now, if I'm explicit in calling the garbage collector, I have to wrap the entire code in an outer loop to cause the OOM crash. For me, this tends to happen around x = 50 or so: public partial class Window1 : Window { public Window1() { InitializeComponent(); //Attempts to create an OOM crash //to do so, mimic minute croppings of an 'image' (ushort array), and then undoing the crops for (int x = 0; x < 1000; x++){ int theRows = 4000, currRows; int theColumns = 4000, currCols; int theMaxChange = 30; int i; List<ushort[]> theList = new List<ushort[]>();//the list of images in the undo/redo stack byte[] displayBuffer = null;//the buffer used as a bitmap source BitmapSource theSource = null; for (i = 0; i < theMaxChange; i++) { currRows = theRows - i; currCols = theColumns - i; theList.Add(new ushort[(theRows - i) * (theColumns - i)]); displayBuffer = new byte[theList[i].Length]; theSource = BitmapSource.Create(currCols, currRows, 96, 96, PixelFormats.Gray8, null, displayBuffer, (currCols * PixelFormats.Gray8.BitsPerPixel + 7) / 8); } //should get here. If not, then theMaxChange is too large. //Now, go back up the undo stack. for (i = theMaxChange - 1; i >= 0; i--) { displayBuffer = new byte[theList[i].Length]; theSource = BitmapSource.Create((theColumns - i), (theRows - i), 96, 96, PixelFormats.Gray8, null, displayBuffer, ((theColumns - i) * PixelFormats.Gray8.BitsPerPixel + 7) / 8); GC.WaitForPendingFinalizers();//force gc to collect, because we're in scenario 2, lots of large random changes GC.Collect(); } System.Console.WriteLine("Got to changelist " + x.ToString()); System.Threading.Thread.Sleep(100); } } } If I'm mishandling memory in either scenario, if there's something I should spot with a profiler, let me know. That's a pretty simple routine there. Unfortunately, it looks like @Kevin's answer is right-- this is a bug in .NET and how .NET handles objects larger than 85k. This situation strikes me as exceedingly strange; could Powerpoint be rewritten in .NET with this kind of limitation, or any of the other Office suite applications? 85k does not seem to me to be a whole lot of space, and I'd also think that any program that uses so-called 'large' allocations frequently would become unstable within a matter of days to weeks when using .NET. EDIT: It looks like Kevin is right, this is a limitation of .NET's GC. For those who don't want to follow the entire thread, .NET has four GC heaps: gen0, gen1, gen2, and LOH (Large Object Heap). Everything that's 85k or smaller goes on one of the first three heaps, depending on creation time (moved from gen0 to gen1 to gen2, etc). Objects larger than 85k get placed on the LOH. The LOH is never compacted, so eventually, allocations of the type I'm doing will eventually cause an OOM error as objects get scattered about that memory space. We've found that moving to .NET 4.0 does help the problem somewhat, delaying the exception, but not preventing it. To be honest, this feels a bit like the 640k barrier-- 85k ought to be enough for any user application (to paraphrase this video of a discussion of the GC in .NET). For the record, Java does not exhibit this behavior with its GC.

    Read the article

  • FreeBSD 8.1 unstable network connection

    - by frankcheong
    I have three FreeBSD 8.1 running on three different hardware and therefore consist of different network adapter as well (bce, bge and igb). I found that the network connection is kind of unstable which I have tried to scp some 10MB file and found that I cannot always get the files completed successfully. I have further checked with my network admin and he claim that the problem is being caused by the network driver which cannot support the load whereby he tried to ping using huge packet size (around 15k) and my server will drop packet consistently at a regular interval. I found that this statement may not be valid since the three server is using three different network drive and it would be quite impossible that the same problem is being caused by three different network adapter and thus different network driver. Since then I have tried to tune up the performance by playing around with the /etc/sysctl.conf figures with no luck. kern.ipc.somaxconn=1024 kern.ipc.shmall=3276800 kern.ipc.shmmax=1638400000 # Security net.inet.ip.redirect=0 net.inet.ip.sourceroute=0 net.inet.ip.accept_sourceroute=0 net.inet.icmp.maskrepl=0 net.inet.icmp.log_redirect=0 net.inet.icmp.drop_redirect=1 net.inet.tcp.drop_synfin=1 # Security net.inet.udp.blackhole=1 net.inet.tcp.blackhole=2 # Required by pf net.inet.ip.forwarding=1 #Network Performance Tuning kern.ipc.maxsockbuf=16777216 net.inet.tcp.rfc1323=1 net.inet.tcp.sendbuf_max=16777216 net.inet.tcp.recvbuf_max=16777216 # Setting specifically for 1 or even 10Gbps network net.local.stream.sendspace=262144 net.local.stream.recvspace=262144 net.inet.tcp.local_slowstart_flightsize=10 net.inet.tcp.nolocaltimewait=1 net.inet.tcp.mssdflt=1460 net.inet.tcp.sendbuf_auto=1 net.inet.tcp.sendbuf_inc=16384 net.inet.tcp.recvbuf_auto=1 net.inet.tcp.recvbuf_inc=524288 net.inet.tcp.sendspace=262144 net.inet.tcp.recvspace=262144 net.inet.udp.recvspace=262144 kern.ipc.maxsockbuf=16777216 kern.ipc.nmbclusters=32768 net.inet.tcp.delayed_ack=1 net.inet.tcp.delacktime=100 net.inet.tcp.slowstart_flightsize=179 net.inet.tcp.inflight.enable=1 net.inet.tcp.inflight.min=6144 # Reduce the cache size of slow start connection net.inet.tcp.hostcache.expire=1 Our network admin also claim that they see quite a lot of network up and down from their cisco switch log while I cannot find any up down message inside the dmesg. Have further checked the netstat -s but dont have concrete idea. tcp: 133695291 packets sent 39408539 data packets (3358837321 bytes) 61868 data packets (89472844 bytes) retransmitted 24 data packets unnecessarily retransmitted 0 resends initiated by MTU discovery 50756141 ack-only packets (2148 delayed) 0 URG only packets 0 window probe packets 4372385 window update packets 39781869 control packets 134898031 packets received 72339403 acks (for 3357601899 bytes) 190712 duplicate acks 0 acks for unsent data 59339201 packets (3647021974 bytes) received in-sequence 114 completely duplicate packets (135202 bytes) 27 old duplicate packets 0 packets with some dup. data (0 bytes duped) 42090 out-of-order packets (60817889 bytes) 0 packets (0 bytes) of data after window 0 window probes 3953896 window update packets 64181 packets received after close 0 discarded for bad checksums 0 discarded for bad header offset fields 0 discarded because packet too short 45192 discarded due to memory problems 19945391 connection requests 1323420 connection accepts 0 bad connection attempts 0 listen queue overflows 0 ignored RSTs in the windows 21133581 connections established (including accepts) 21268724 connections closed (including 32737 drops) 207874 connections updated cached RTT on close 207874 connections updated cached RTT variance on close 132439 connections updated cached ssthresh on close 42392 embryonic connections dropped 72339338 segments updated rtt (of 69477829 attempts) 390871 retransmit timeouts 0 connections dropped by rexmit timeout 0 persist timeouts 0 connections dropped by persist timeout 0 Connections (fin_wait_2) dropped because of timeout 13990 keepalive timeouts 2 keepalive probes sent 13988 connections dropped by keepalive 173044 correct ACK header predictions 36947371 correct data packet header predictions 1323420 syncache entries added 0 retransmitted 0 dupsyn 0 dropped 1323420 completed 0 bucket overflow 0 cache overflow 0 reset 0 stale 0 aborted 0 badack 0 unreach 0 zone failures 1323420 cookies sent 0 cookies received 1864 SACK recovery episodes 18005 segment rexmits in SACK recovery episodes 26066896 byte rexmits in SACK recovery episodes 147327 SACK options (SACK blocks) received 87473 SACK options (SACK blocks) sent 0 SACK scoreboard overflow 0 packets with ECN CE bit set 0 packets with ECN ECT(0) bit set 0 packets with ECN ECT(1) bit set 0 successful ECN handshakes 0 times ECN reduced the congestion window udp: 5141258 datagrams received 0 with incomplete header 0 with bad data length field 0 with bad checksum 1 with no checksum 0 dropped due to no socket 129616 broadcast/multicast datagrams undelivered 0 dropped due to full socket buffers 0 not for hashed pcb 5011642 delivered 5016050 datagrams output 0 times multicast source filter matched sctp: 0 input packets 0 datagrams 0 packets that had data 0 input SACK chunks 0 input DATA chunks 0 duplicate DATA chunks 0 input HB chunks 0 HB-ACK chunks 0 input ECNE chunks 0 input AUTH chunks 0 chunks missing AUTH 0 invalid HMAC ids received 0 invalid secret ids received 0 auth failed 0 fast path receives all one chunk 0 fast path multi-part data 0 output packets 0 output SACKs 0 output DATA chunks 0 retransmitted DATA chunks 0 fast retransmitted DATA chunks 0 FR's that happened more than once to same chunk 0 intput HB chunks 0 output ECNE chunks 0 output AUTH chunks 0 ip_output error counter Packet drop statistics: 0 from middle box 0 from end host 0 with data 0 non-data, non-endhost 0 non-endhost, bandwidth rep only 0 not enough for chunk header 0 not enough data to confirm 0 where process_chunk_drop said break 0 failed to find TSN 0 attempt reverse TSN lookup 0 e-host confirms zero-rwnd 0 midbox confirms no space 0 data did not match TSN 0 TSN's marked for Fast Retran Timeouts: 0 iterator timers fired 0 T3 data time outs 0 window probe (T3) timers fired 0 INIT timers fired 0 sack timers fired 0 shutdown timers fired 0 heartbeat timers fired 0 a cookie timeout fired 0 an endpoint changed its cookiesecret 0 PMTU timers fired 0 shutdown ack timers fired 0 shutdown guard timers fired 0 stream reset timers fired 0 early FR timers fired 0 an asconf timer fired 0 auto close timer fired 0 asoc free timers expired 0 inp free timers expired 0 packet shorter than header 0 checksum error 0 no endpoint for port 0 bad v-tag 0 bad SID 0 no memory 0 number of multiple FR in a RTT window 0 RFC813 allowed sending 0 RFC813 does not allow sending 0 times max burst prohibited sending 0 look ahead tells us no memory in interface 0 numbers of window probes sent 0 times an output error to clamp down on next user send 0 times sctp_senderrors were caused from a user 0 number of in data drops due to chunk limit reached 0 number of in data drops due to rwnd limit reached 0 times a ECN reduced the cwnd 0 used express lookup via vtag 0 collision in express lookup 0 times the sender ran dry of user data on primary 0 same for above 0 sacks the slow way 0 window update only sacks sent 0 sends with sinfo_flags !=0 0 unordered sends 0 sends with EOF flag set 0 sends with ABORT flag set 0 times protocol drain called 0 times we did a protocol drain 0 times recv was called with peek 0 cached chunks used 0 cached stream oq's used 0 unread messages abandonded by close 0 send burst avoidance, already max burst inflight to net 0 send cwnd full avoidance, already max burst inflight to net 0 number of map array over-runs via fwd-tsn's ip: 137814085 total packets received 0 bad header checksums 0 with size smaller than minimum 0 with data size < data length 0 with ip length > max ip packet size 0 with header length < data size 0 with data length < header length 0 with bad options 0 with incorrect version number 1200 fragments received 0 fragments dropped (dup or out of space) 0 fragments dropped after timeout 300 packets reassembled ok 137813009 packets for this host 530 packets for unknown/unsupported protocol 0 packets forwarded (0 packets fast forwarded) 61 packets not forwardable 0 packets received for unknown multicast group 0 redirects sent 137234598 packets sent from this host 0 packets sent with fabricated ip header 685307 output packets dropped due to no bufs, etc. 52 output packets discarded due to no route 300 output datagrams fragmented 1200 fragments created 0 datagrams that can't be fragmented 0 tunneling packets that can't find gif 0 datagrams with bad address in header icmp: 0 calls to icmp_error 0 errors not generated in response to an icmp message Output histogram: echo reply: 305 0 messages with bad code fields 0 messages less than the minimum length 0 messages with bad checksum 0 messages with bad length 0 multicast echo requests ignored 0 multicast timestamp requests ignored Input histogram: destination unreachable: 530 echo: 305 305 message responses generated 0 invalid return addresses 0 no return routes ICMP address mask responses are disabled igmp: 0 messages received 0 messages received with too few bytes 0 messages received with wrong TTL 0 messages received with bad checksum 0 V1/V2 membership queries received 0 V3 membership queries received 0 membership queries received with invalid field(s) 0 general queries received 0 group queries received 0 group-source queries received 0 group-source queries dropped 0 membership reports received 0 membership reports received with invalid field(s) 0 membership reports received for groups to which we belong 0 V3 reports received without Router Alert 0 membership reports sent arp: 376748 ARP requests sent 3207 ARP replies sent 245245 ARP requests received 80845 ARP replies received 326090 ARP packets received 267712 total packets dropped due to no ARP entry 108876 ARP entrys timed out 0 Duplicate IPs seen ip6: 2226633 total packets received 0 with size smaller than minimum 0 with data size < data length 0 with bad options 0 with incorrect version number 0 fragments received 0 fragments dropped (dup or out of space) 0 fragments dropped after timeout 0 fragments that exceeded limit 0 packets reassembled ok 2226633 packets for this host 0 packets forwarded 0 packets not forwardable 0 redirects sent 2226633 packets sent from this host 0 packets sent with fabricated ip header 0 output packets dropped due to no bufs, etc. 8 output packets discarded due to no route 0 output datagrams fragmented 0 fragments created 0 datagrams that can't be fragmented 0 packets that violated scope rules 0 multicast packets which we don't join Input histogram: UDP: 2226633 Mbuf statistics: 962679 one mbuf 1263954 one ext mbuf 0 two or more ext mbuf 0 packets whose headers are not continuous 0 tunneling packets that can't find gif 0 packets discarded because of too many headers 0 failures of source address selection Source addresses selection rule applied: icmp6: 0 calls to icmp6_error 0 errors not generated in response to an icmp6 message 0 errors not generated because of rate limitation 0 messages with bad code fields 0 messages < minimum length 0 bad checksums 0 messages with bad length Histogram of error messages to be generated: 0 no route 0 administratively prohibited 0 beyond scope 0 address unreachable 0 port unreachable 0 packet too big 0 time exceed transit 0 time exceed reassembly 0 erroneous header field 0 unrecognized next header 0 unrecognized option 0 redirect 0 unknown 0 message responses generated 0 messages with too many ND options 0 messages with bad ND options 0 bad neighbor solicitation messages 0 bad neighbor advertisement messages 0 bad router solicitation messages 0 bad router advertisement messages 0 bad redirect messages 0 path MTU changes rip6: 0 messages received 0 checksum calculations on inbound 0 messages with bad checksum 0 messages dropped due to no socket 0 multicast messages dropped due to no socket 0 messages dropped due to full socket buffers 0 delivered 0 datagrams output netstat -m 516/5124/5640 mbufs in use (current/cache/total) 512/1634/2146/32768 mbuf clusters in use (current/cache/total/max) 512/1536 mbuf+clusters out of packet secondary zone in use (current/cache) 0/1303/1303/12800 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/6400 9k jumbo clusters in use (current/cache/total/max) 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) 1153K/9761K/10914K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/8/6656 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines Anyone got an idea what might be the possible cause?

    Read the article

  • Red Gate Coder interviews: Alex Davies

    - by Michael Williamson
    Alex Davies has been a software engineer at Red Gate since graduating from university, and is currently busy working on .NET Demon. We talked about tackling parallel programming with his actors framework, a scientific approach to debugging, and how JavaScript is going to affect the programming languages we use in years to come. So, if we start at the start, how did you get started in programming? When I was seven or eight, I was given a BBC Micro for Christmas. I had asked for a Game Boy, but my dad thought it would be better to give me a proper computer. For a year or so, I only played games on it, but then I found the user guide for writing programs in it. I gradually started doing more stuff on it and found it fun. I liked creating. As I went into senior school I continued to write stuff on there, trying to write games that weren’t very good. I got a real computer when I was fourteen and found ways to write BASIC on it. Visual Basic to start with, and then something more interesting than that. How did you learn to program? Was there someone helping you out? Absolutely not! I learnt out of a book, or by experimenting. I remember the first time I found a loop, I was like “Oh my God! I don’t have to write out the same line over and over and over again any more. It’s amazing!” When did you think this might be something that you actually wanted to do as a career? For a long time, I thought it wasn’t something that you would do as a career, because it was too much fun to be a career. I thought I’d do chemistry at university and some kind of career based on chemical engineering. And then I went to a careers fair at school when I was seventeen or eighteen, and it just didn’t interest me whatsoever. I thought “I could be a programmer, and there’s loads of money there, and I’m good at it, and it’s fun”, but also that I shouldn’t spoil my hobby. Now I don’t really program in my spare time any more, which is a bit of a shame, but I program all the rest of the time, so I can live with it. Do you think you learnt much about programming at university? Yes, definitely! I went into university knowing how to make computers do anything I wanted them to do. However, I didn’t have the language to talk about algorithms, so the algorithms course in my first year was massively important. Learning other language paradigms like functional programming was really good for breadth of understanding. Functional programming influences normal programming through design rather than actually using it all the time. I draw inspiration from it to write imperative programs which I think is actually becoming really fashionable now, but I’ve been doing it for ages. I did it first! There were also some courses on really odd programming languages, a bit of Prolog, a little bit of C. Having a little bit of each of those is something that I would have never done on my own, so it was important. And then there are knowledge-based courses which are about not programming itself but things that have been programmed like TCP. Those are really important for examples for how to approach things. Did you do any internships while you were at university? Yeah, I spent both of my summers at the same company. I thought I could code well before I went there. Looking back at the crap that I produced, it was only surpassed in its crappiness by all of the other code already in that company. I’m so much better at writing nice code now than I used to be back then. Was there just not a culture of looking after your code? There was, they just didn’t hire people for their abilities in that area. They hired people for raw IQ. The first indicator of it going wrong was that they didn’t have any computer scientists, which is a bit odd in a programming company. But even beyond that they didn’t have people who learnt architecture from anyone else. Most of them had started straight out of university, so never really had experience or mentors to learn from. There wasn’t the experience to draw from to teach each other. In the second half of my second internship, I was being given tasks like looking at new technologies and teaching people stuff. Interns shouldn’t be teaching people how to do their jobs! All interns are going to have little nuggets of things that you don’t know about, but they shouldn’t consistently be the ones who know the most. It’s not a good environment to learn. I was going to ask how you found working with people who were more experienced than you… When I reached Red Gate, I found some people who were more experienced programmers than me, and that was difficult. I’ve been coding since I was tiny. At university there were people who were cleverer than me, but there weren’t very many who were more experienced programmers than me. During my internship, I didn’t find anyone who I classed as being a noticeably more experienced programmer than me. So, it was a shock to the system to have valid criticisms rather than just formatting criticisms. However, Red Gate’s not so big on the actual code review, at least it wasn’t when I started. We did an entire product release and then somebody looked over all of the UI of that product which I’d written and say what they didn’t like. By that point, it was way too late and I’d disagree with them. Do you think the lack of code reviews was a bad thing? I think if there’s going to be any oversight of new people, then it should be continuous rather than chunky. For me I don’t mind too much, I could go out and get oversight if I wanted it, and in those situations I felt comfortable without it. If I was managing the new person, then maybe I’d be keener on oversight and then the right way to do it is continuously and in very, very small chunks. Have you had any significant projects you’ve worked on outside of a job? When I was a teenager I wrote all sorts of stuff. I used to write games, I derived how to do isomorphic projections myself once. I didn’t know what the word was so I couldn’t Google for it, so I worked it out myself. It was horrifically complicated. But it sort of tailed off when I started at university, and is now basically zero. If I do side-projects now, they tend to be work-related side projects like my actors framework, NAct, which I started in a down tools week. Could you explain a little more about NAct? It is a little C# framework for writing parallel code more easily. Parallel programming is difficult when you need to write to shared data. Sometimes parallel programming is easy because you don’t need to write to shared data. When you do need to access shared data, you could just have your threads pile in and do their work, but then you would screw up the data because the threads would trample on each other’s toes. You could lock, but locks are really dangerous if you’re using more than one of them. You get interactions like deadlocks, and that’s just nasty. Actors instead allows you to say this piece of data belongs to this thread of execution, and nobody else can read it. If you want to read it, then ask that thread of execution for a piece of it by sending a message, and it will send the data back by a message. And that avoids deadlocks as long as you follow some obvious rules about not making your actors sit around waiting for other actors to do something. There are lots of ways to write actors, NAct allows you to do it as if it was method calls on other objects, which means you get all the strong type-safety that C# programmers like. Do you think that this is suitable for the majority of parallel programming, or do you think it’s only suitable for specific cases? It’s suitable for most difficult parallel programming. If you’ve just got a hundred web requests which are all independent of each other, then I wouldn’t bother because it’s easier to just spin them up in separate threads and they can proceed independently of each other. But where you’ve got difficult parallel programming, where you’ve got multiple threads accessing multiple bits of data in multiple ways at different times, then actors is at least as good as all other ways, and is, I reckon, easier to think about. When you’re using actors, you presumably still have to write your code in a different way from you would otherwise using single-threaded code. You can’t use actors with any methods that have return types, because you’re not allowed to call into another actor and wait for it. If you want to get a piece of data out of another actor, then you’ve got to use tasks so that you can use “async” and “await” to await asynchronously for it. But other than that, you can still stick things in classes so it’s not too different really. Rather than having thousands of objects with mutable state, you can use component-orientated design, where there are only a few mutable classes which each have a small number of instances. Then there can be thousands of immutable objects. If you tend to do that anyway, then actors isn’t much of a jump. If I’ve already built my system without any parallelism, how hard is it to add actors to exploit all eight cores on my desktop? Usually pretty easy. If you can identify even one boundary where things look like messages and you have components where some objects live on one side and these other objects live on the other side, then you can have a granddaddy object on one side be an actor and it will parallelise as it goes across that boundary. Not too difficult. If we do get 1000-core desktop PCs, do you think actors will scale up? It’s hard. There are always in the order of twenty to fifty actors in my whole program because I tend to write each component as actors, and I tend to have one instance of each component. So this won’t scale to a thousand cores. What you can do is write data structures out of actors. I use dictionaries all over the place, and if you need a dictionary that is going to be accessed concurrently, then you could build one of those out of actors in no time. You can use queuing to marshal requests between different slices of the dictionary which are living on different threads. So it’s like a distributed hash table but all of the chunks of it are on the same machine. That means that each of these thousand processors has cached one small piece of the dictionary. I reckon it wouldn’t be too big a leap to start doing proper parallelism. Do you think it helps if actors get baked into the language, similarly to Erlang? Erlang is excellent in that it has thread-local garbage collection. C# doesn’t, so there’s a limit to how well C# actors can possibly scale because there’s a single garbage collected heap shared between all of them. When you do a global garbage collection, you’ve got to stop all of the actors, which is seriously expensive, whereas in Erlang garbage collections happen per-actor, so they’re insanely cheap. However, Erlang deviated from all the sensible language design that people have used recently and has just come up with crazy stuff. You can definitely retrofit thread-local garbage collection to .NET, and then it’s quite well-suited to support actors, even if it’s not baked into the language. Speaking of language design, do you have a favourite programming language? I’ll choose a language which I’ve never written before. I like the idea of Scala. It sounds like C#, only with some of the niggles gone. I enjoy writing static types. It means you don’t have to writing tests so much. When you say it doesn’t have some of the niggles? C# doesn’t allow the use of a property as a method group. It doesn’t have Scala case classes, or sum types, where you can do a switch statement and the compiler checks that you’ve checked all the cases, which is really useful in functional-style programming. Pattern-matching, in other words. That’s actually the major niggle. C# is pretty good, and I’m quite happy with C#. And what about going even further with the type system to remove the need for tests to something like Haskell? Or is that a step too far? I’m quite a pragmatist, I don’t think I could deal with trying to write big systems in languages with too few other users, especially when learning how to structure things. I just don’t know anyone who can teach me, and the Internet won’t teach me. That’s the main reason I wouldn’t use it. If I turned up at a company that writes big systems in Haskell, I would have no objection to that, but I wouldn’t instigate it. What about things in C#? For instance, there’s contracts in C#, so you can try to statically verify a bit more about your code. Do you think that’s useful, or just not worthwhile? I’ve not really tried it. My hunch is that it needs to be built into the language and be quite mathematical for it to work in real life, and that doesn’t seem to have ended up true for C# contracts. I don’t think anyone who’s tried them thinks they’re any good. I might be wrong. On a slightly different note, how do you like to debug code? I think I’m quite an odd debugger. I use guesswork extremely rarely, especially if something seems quite difficult to debug. I’ve been bitten spending hours and hours on guesswork and not being scientific about debugging in the past, so now I’m scientific to a fault. What I want is to see the bug happening in the debugger, to step through the bug happening. To watch the program going from a valid state to an invalid state. When there’s a bug and I can’t work out why it’s happening, I try to find some piece of evidence which places the bug in one section of the code. From that experiment, I binary chop on the possible causes of the bug. I suppose that means binary chopping on places in the code, or binary chopping on a stage through a processing cycle. Basically, I’m very stupid about how I debug. I won’t make any guesses, I won’t use any intuition, I will only identify the experiment that’s going to binary chop most effectively and repeat rather than trying to guess anything. I suppose it’s quite top-down. Is most of the time then spent in the debugger? Absolutely, if at all possible I will never debug using print statements or logs. I don’t really hold much stock in outputting logs. If there’s any bug which can be reproduced locally, I’d rather do it in the debugger than outputting logs. And with SmartAssembly error reporting, there’s not a lot that can’t be either observed in an error report and just fixed, or reproduced locally. And in those other situations, maybe I’ll use logs. But I hate using logs. You stare at the log, trying to guess what’s going on, and that’s exactly what I don’t like doing. You have to just look at it and see does this look right or wrong. We’ve covered how you get to grip with bugs. How do you get to grips with an entire codebase? I watch it in the debugger. I find little bugs and then try to fix them, and mostly do it by watching them in the debugger and gradually getting an understanding of how the code works using my process of binary chopping. I have to do a lot of reading and watching code to choose where my slicing-in-half experiment is going to be. The last time I did it was SmartAssembly. The old code was a complete mess, but at least it did things top to bottom. There wasn’t too much of some of the big abstractions where flow of control goes all over the place, into a base class and back again. Code’s really hard to understand when that happens. So I like to choose a little bug and try to fix it, and choose a bigger bug and try to fix it. Definitely learn by doing. I want to always have an aim so that I get a little achievement after every few hours of debugging. Once I’ve learnt the codebase I might be able to fix all the bugs in an hour, but I’d rather be using them as an aim while I’m learning the codebase. If I was a maintainer of a codebase, what should I do to make it as easy as possible for you to understand? Keep distinct concepts in different places. And name your stuff so that it’s obvious which concepts live there. You shouldn’t have some variable that gets set miles up the top of somewhere, and then is read miles down to choose some later behaviour. I’m talking from a very much SmartAssembly point of view because the old SmartAssembly codebase had tons and tons of these things, where it would read some property of the code and then deal with it later. Just thousands of variables in scope. Loads of things to think about. If you can keep concepts separate, then it aids me in my process of fixing bugs one at a time, because each bug is going to more or less be understandable in the one place where it is. And what about tests? Do you think they help at all? I’ve never had the opportunity to learn a codebase which has had tests, I don’t know what it’s like! What about when you’re actually developing? How useful do you find tests in finding bugs or regressions? Finding regressions, absolutely. Running bits of code that would be quite hard to run otherwise, definitely. It doesn’t happen very often that a test finds a bug in the first place. I don’t really buy nebulous promises like tests being a good way to think about the spec of the code. My thinking goes something like “This code works at the moment, great, ship it! Ah, there’s a way that this code doesn’t work. Okay, write a test, demonstrate that it doesn’t work, fix it, use the test to demonstrate that it’s now fixed, and keep the test for future regressions.” The most valuable tests are for bugs that have actually happened at some point, because bugs that have actually happened at some point, despite the fact that you think you’ve fixed them, are way more likely to appear again than new bugs are. Does that mean that when you write your code the first time, there are no tests? Often. The chance of there being a bug in a new feature is relatively unaffected by whether I’ve written a test for that new feature because I’m not good enough at writing tests to think of bugs that I would have written into the code. So not writing regression tests for all of your code hasn’t affected you too badly? There are different kinds of features. Some of them just always work, and are just not flaky, they just continue working whatever you throw at them. Maybe because the type-checker is particularly effective around them. Writing tests for those features which just tend to always work is a waste of time. And because it’s a waste of time I’ll tend to wait until a feature has demonstrated its flakiness by having bugs in it before I start trying to test it. You can get a feel for whether it’s going to be flaky code as you’re writing it. I try to write it to make it not flaky, but there are some things that are just inherently flaky. And very occasionally, I’ll think “this is going to be flaky” as I’m writing, and then maybe do a test, but not most of the time. How do you think your programming style has changed over time? I’ve got clearer about what the right way of doing things is. I used to flip-flop a lot between different ideas. Five years ago I came up with some really good ideas and some really terrible ideas. All of them seemed great when I thought of them, but they were quite diverse ideas, whereas now I have a smaller set of reliable ideas that are actually good for structuring code. So my code is probably more similar to itself than it used to be back in the day, when I was trying stuff out. I’ve got more disciplined about encapsulation, I think. There are operational things like I use actors more now than I used to, and that forces me to use immutability more than I used to. The first code that I wrote in Red Gate was the memory profiler UI, and that was an actor, I just didn’t know the name of it at the time. I don’t really use object-orientation. By object-orientation, I mean having n objects of the same type which are mutable. I want a constant number of objects that are mutable, and they should be different types. I stick stuff in dictionaries and then have one thing that owns the dictionary and puts stuff in and out of it. That’s definitely a pattern that I’ve seen recently. I think maybe I’m doing functional programming. Possibly. It’s plausible. If you had to summarise the essence of programming in a pithy sentence, how would you do it? Programming is the form of art that, without losing any of the beauty of architecture or fine art, allows you to produce things that people love and you make money from. So you think it’s an art rather than a science? It’s a little bit of engineering, a smidgeon of maths, but it’s not science. Like architecture, programming is on that boundary between art and engineering. If you want to do it really nicely, it’s mostly art. You can get away with doing architecture and programming entirely by having a good engineering mind, but you’re not going to produce anything nice. You’re not going to have joy doing it if you’re an engineering mind. Architects who are just engineering minds are not going to enjoy their job. I suppose engineering is the foundation on which you build the art. Exactly. How do you think programming is going to change over the next ten years? There will be an unfortunate shift towards dynamically-typed languages, because of JavaScript. JavaScript has an unfair advantage. JavaScript’s unfair advantage will cause more people to be exposed to dynamically-typed languages, which means other dynamically-typed languages crop up and the best features go into dynamically-typed languages. Then people conflate the good features with the fact that it’s dynamically-typed, and more investment goes into dynamically-typed languages. They end up better, so people use them. What about the idea of compiling other languages, possibly statically-typed, to JavaScript? It’s a reasonable idea. I would like to do it, but I don’t think enough people in the world are going to do it to make it pick up. The hordes of beginners are the lifeblood of a language community. They are what makes there be good tools and what makes there be vibrant community websites. And any particular thing which is the same as JavaScript only with extra stuff added to it, although it might be technically great, is not going to have the hordes of beginners. JavaScript is always to be quickest and easiest way for a beginner to start programming in the browser. And dynamically-typed languages are great for beginners. Compilers are pretty scary and beginners don’t write big code. And having your errors come up in the same place, whether they’re statically checkable errors or not, is quite nice for a beginner. If someone asked me to teach them some programming, I’d teach them JavaScript. If dynamically-typed languages are great for beginners, when do you think the benefits of static typing start to kick in? The value of having a statically typed program is in the tools that rely on the static types to produce a smooth IDE experience rather than actually telling me my compile errors. And only once you’re experienced enough a programmer that having a really smooth IDE experience makes a blind bit of difference, does static typing make a blind bit of difference. So it’s not really about size of codebase. If I go and write up a tiny program, I’m still going to get value out of writing it in C# using ReSharper because I’m experienced with C# and ReSharper enough to be able to write code five times faster if I have that help. Any other visions of the future? Nobody’s going to use actors. Because everyone’s going to be running on single-core VMs connected over network-ready protocols like JSON over HTTP. So, parallelism within one operating system is going to die. But until then, you should use actors. More Red Gater Coder interviews

    Read the article

  • What's New in ASP.NET 4

    - by Navaneeth
    The .NET Framework version 4 includes enhancements for ASP.NET 4 in targeted areas. Visual Studio 2010 and Microsoft Visual Web Developer Express also include enhancements and new features for improved Web development. This document provides an overview of many of the new features that are included in the upcoming release. This topic contains the following sections: ASP.NET Core Services ASP.NET Web Forms ASP.NET MVC Dynamic Data ASP.NET Chart Control Visual Web Developer Enhancements Web Application Deployment with Visual Studio 2010 Enhancements to ASP.NET Multi-Targeting ASP.NET Core Services ASP.NET 4 introduces many features that improve core ASP.NET services such as output caching and session state storage. Extensible Output Caching Since the time that ASP.NET 1.0 was released, output caching has enabled developers to store the generated output of pages, controls, and HTTP responses in memory. On subsequent Web requests, ASP.NET can serve content more quickly by retrieving the generated output from memory instead of regenerating the output from scratch. However, this approach has a limitation — generated content always has to be stored in memory. On servers that experience heavy traffic, the memory requirements for output caching can compete with memory requirements for other parts of a Web application. ASP.NET 4 adds extensibility to output caching that enables you to configure one or more custom output-cache providers. Output-cache providers can use any storage mechanism to persist HTML content. These storage options can include local or remote disks, cloud storage, and distributed cache engines. Output-cache provider extensibility in ASP.NET 4 lets you design more aggressive and more intelligent output-caching strategies for Web sites. For example, you can create an output-cache provider that caches the "Top 10" pages of a site in memory, while caching pages that get lower traffic on disk. Alternatively, you can cache every vary-by combination for a rendered page, but use a distributed cache so that the memory consumption is offloaded from front-end Web servers. You create a custom output-cache provider as a class that derives from the OutputCacheProvider type. You can then configure the provider in the Web.config file by using the new providers subsection of the outputCache element For more information and for examples that show how to configure the output cache, see outputCache Element for caching (ASP.NET Settings Schema). For more information about the classes that support caching, see the documentation for the OutputCache and OutputCacheProvider classes. By default, in ASP.NET 4, all HTTP responses, rendered pages, and controls use the in-memory output cache. The defaultProvider attribute for ASP.NET is AspNetInternalProvider. You can change the default output-cache provider used for a Web application by specifying a different provider name for defaultProvider attribute. In addition, you can select different output-cache providers for individual control and for individual requests and programmatically specify which provider to use. For more information, see the HttpApplication.GetOutputCacheProviderName(HttpContext) method. The easiest way to choose a different output-cache provider for different Web user controls is to do so declaratively by using the new providerName attribute in a page or control directive, as shown in the following example: <%@ OutputCache Duration="60" VaryByParam="None" providerName="DiskCache" %> Preloading Web Applications Some Web applications must load large amounts of data or must perform expensive initialization processing before serving the first request. In earlier versions of ASP.NET, for these situations you had to devise custom approaches to "wake up" an ASP.NET application and then run initialization code during the Application_Load method in the Global.asax file. To address this scenario, a new application preload manager (autostart feature) is available when ASP.NET 4 runs on IIS 7.5 on Windows Server 2008 R2. The preload feature provides a controlled approach for starting up an application pool, initializing an ASP.NET application, and then accepting HTTP requests. It lets you perform expensive application initialization prior to processing the first HTTP request. For example, you can use the application preload manager to initialize an application and then signal a load-balancer that the application was initialized and ready to accept HTTP traffic. To use the application preload manager, an IIS administrator sets an application pool in IIS 7.5 to be automatically started by using the following configuration in the applicationHost.config file: <applicationPools> <add name="MyApplicationPool" startMode="AlwaysRunning" /> </applicationPools> Because a single application pool can contain multiple applications, you specify individual applications to be automatically started by using the following configuration in the applicationHost.config file: <sites> <site name="MySite" id="1"> <application path="/" serviceAutoStartEnabled="true" serviceAutoStartProvider="PrewarmMyCache" > <!-- Additional content --> </application> </site> </sites> <!-- Additional content --> <serviceAutoStartProviders> <add name="PrewarmMyCache" type="MyNamespace.CustomInitialization, MyLibrary" /> </serviceAutoStartProviders> When an IIS 7.5 server is cold-started or when an individual application pool is recycled, IIS 7.5 uses the information in the applicationHost.config file to determine which Web applications have to be automatically started. For each application that is marked for preload, IIS7.5 sends a request to ASP.NET 4 to start the application in a state during which the application temporarily does not accept HTTP requests. When it is in this state, ASP.NET instantiates the type defined by the serviceAutoStartProvider attribute (as shown in the previous example) and calls into its public entry point. You create a managed preload type that has the required entry point by implementing the IProcessHostPreloadClient interface, as shown in the following example: public class CustomInitialization : System.Web.Hosting.IProcessHostPreloadClient { public void Preload(string[] parameters) { // Perform initialization. } } After your initialization code runs in the Preload method and after the method returns, the ASP.NET application is ready to process requests. Permanently Redirecting a Page Content in Web applications is often moved over the lifetime of the application. This can lead to links to be out of date, such as the links that are returned by search engines. In ASP.NET, developers have traditionally handled requests to old URLs by using the Redirect method to forward a request to the new URL. However, the Redirect method issues an HTTP 302 (Found) response (which is used for a temporary redirect). This results in an extra HTTP round trip. ASP.NET 4 adds a RedirectPermanent helper method that makes it easy to issue HTTP 301 (Moved Permanently) responses, as in the following example: RedirectPermanent("/newpath/foroldcontent.aspx"); Search engines and other user agents that recognize permanent redirects will store the new URL that is associated with the content, which eliminates the unnecessary round trip made by the browser for temporary redirects. Session State Compression By default, ASP.NET provides two options for storing session state across a Web farm. The first option is a session state provider that invokes an out-of-process session state server. The second option is a session state provider that stores data in a Microsoft SQL Server database. Because both options store state information outside a Web application's worker process, session state has to be serialized before it is sent to remote storage. If a large amount of data is saved in session state, the size of the serialized data can become very large. ASP.NET 4 introduces a new compression option for both kinds of out-of-process session state providers. By using this option, applications that have spare CPU cycles on Web servers can achieve substantial reductions in the size of serialized session state data. You can set this option using the new compressionEnabled attribute of the sessionState element in the configuration file. When the compressionEnabled configuration option is set to true, ASP.NET compresses (and decompresses) serialized session state by using the .NET Framework GZipStreamclass. The following example shows how to set this attribute. <sessionState mode="SqlServer" sqlConnectionString="data source=dbserver;Initial Catalog=aspnetstate" allowCustomSqlDatabase="true" compressionEnabled="true" /> ASP.NET Web Forms Web Forms has been a core feature in ASP.NET since the release of ASP.NET 1.0. Many enhancements have been in this area for ASP.NET 4, such as the following: The ability to set meta tags. More control over view state. Support for recently introduced browsers and devices. Easier ways to work with browser capabilities. Support for using ASP.NET routing with Web Forms. More control over generated IDs. The ability to persist selected rows in data controls. More control over rendered HTML in the FormView and ListView controls. Filtering support for data source controls. Enhanced support for Web standards and accessibility Setting Meta Tags with the Page.MetaKeywords and Page.MetaDescription Properties Two properties have been added to the Page class: MetaKeywords and MetaDescription. These two properties represent corresponding meta tags in the HTML rendered for a page, as shown in the following example: <head id="Head1" runat="server"> <title>Untitled Page</title> <meta name="keywords" content="keyword1, keyword2' /> <meta name="description" content="Description of my page" /> </head> These two properties work like the Title property does, and they can be set in the @ Page directive. For more information, see Page.MetaKeywords and Page.MetaDescription. Enabling View State for Individual Controls A new property has been added to the Control class: ViewStateMode. You can use this property to disable view state for all controls on a page except those for which you explicitly enable view state. View state data is included in a page's HTML and increases the amount of time it takes to send a page to the client and post it back. Storing more view state than is necessary can cause significant decrease in performance. In earlier versions of ASP.NET, you could reduce the impact of view state on a page's performance by disabling view state for specific controls. But sometimes it is easier to enable view state for a few controls that need it instead of disabling it for many that do not need it. For more information, see Control.ViewStateMode. Support for Recently Introduced Browsers and Devices ASP.NET includes a feature that is named browser capabilities that lets you determine the capabilities of the browser that a user is using. Browser capabilities are represented by the HttpBrowserCapabilities object which is stored in the HttpRequest.Browser property. Information about a particular browser's capabilities is defined by a browser definition file. In ASP.NET 4, these browser definition files have been updated to contain information about recently introduced browsers and devices such as Google Chrome, Research in Motion BlackBerry smart phones, and Apple iPhone. Existing browser definition files have also been updated. For more information, see How to: Upgrade an ASP.NET Web Application to ASP.NET 4 and ASP.NET Web Server Controls and Browser Capabilities. The browser definition files that are included with ASP.NET 4 are shown in the following list: •blackberry.browser •chrome.browser •Default.browser •firefox.browser •gateway.browser •generic.browser •ie.browser •iemobile.browser •iphone.browser •opera.browser •safari.browser A New Way to Define Browser Capabilities ASP.NET 4 includes a new feature referred to as browser capabilities providers. As the name suggests, this lets you build a provider that in turn lets you write custom code to determine browser capabilities. In ASP.NET version 3.5 Service Pack 1, you define browser capabilities in an XML file. This file resides in a machine-level folder or an application-level folder. Most developers do not need to customize these files, but for those who do, the provider approach can be easier than dealing with complex XML syntax. The provider approach makes it possible to simplify the process by implementing a common browser definition syntax, or a database that contains up-to-date browser definitions, or even a Web service for such a database. For more information about the new browser capabilities provider, see the What's New for ASP.NET 4 White Paper. Routing in ASP.NET 4 ASP.NET 4 adds built-in support for routing with Web Forms. Routing is a feature that was introduced with ASP.NET 3.5 SP1 and lets you configure an application to use URLs that are meaningful to users and to search engines because they do not have to specify physical file names. This can make your site more user-friendly and your site content more discoverable by search engines. For example, the URL for a page that displays product categories in your application might look like the following example: http://website/products.aspx?categoryid=12 By using routing, you can use the following URL to render the same information: http://website/products/software The second URL lets the user know what to expect and can result in significantly improved rankings in search engine results. the new features include the following: The PageRouteHandler class is a simple HTTP handler that you use when you define routes. You no longer have to write a custom route handler. The HttpRequest.RequestContext and Page.RouteData properties make it easier to access information that is passed in URL parameters. The RouteUrl expression provides a simple way to create a routed URL in markup. The RouteValue expression provides a simple way to extract URL parameter values in markup. The RouteParameter class makes it easier to pass URL parameter values to a query for a data source control (similar to FormParameter). You no longer have to change the Web.config file to enable routing. For more information about routing, see the following topics: ASP.NET Routing Walkthrough: Using ASP.NET Routing in a Web Forms Application How to: Define Routes for Web Forms Applications How to: Construct URLs from Routes How to: Access URL Parameters in a Routed Page Setting Client IDs The new ClientIDMode property makes it easier to write client script that references HTML elements rendered for server controls. Increasing use of Microsoft Ajax makes the need to do this more common. For example, you may have a data control that renders a long list of products with prices and you want to use client script to make a Web service call and update individual prices in the list as they change without refreshing the entire page. Typically you get a reference to an HTML element in client script by using the document.GetElementById method. You pass to this method the value of the id attribute of the HTML element you want to reference. In the case of elements that are rendered for ASP.NET server controls earlier versions of ASP.NET could make this difficult or impossible. You were not always able to predict what id values ASP.NET would generate, or ASP.NET could generate very long id values. The problem was especially difficult for data controls that would generate multiple rows for a single instance of the control in your markup. ASP.NET 4 adds two new algorithms for generating id attributes. These algorithms can generate id attributes that are easier to work with in client script because they are more predictable and that are easier to work with because they are simpler. For more information about how to use the new algorithms, see the following topics: ASP.NET Web Server Control Identification Walkthrough: Making Data-Bound Controls Easier to Access from JavaScript Walkthrough: Making Controls Located in Web User Controls Easier to Access from JavaScript How to: Access Controls from JavaScript by ID Persisting Row Selection in Data Controls The GridView and ListView controls enable users to select a row. In previous versions of ASP.NET, row selection was based on the row index on the page. For example, if you select the third item on page 1 and then move to page 2, the third item on page 2 is selected. In most cases, is more desirable not to select any rows on page 2. ASP.NET 4 supports Persisted Selection, a new feature that was initially supported only in Dynamic Data projects in the .NET Framework 3.5 SP1. When this feature is enabled, the selected item is based on the row data key. This means that if you select the third row on page 1 and move to page 2, nothing is selected on page 2. When you move back to page 1, the third row is still selected. This is a much more natural behavior than the behavior in earlier versions of ASP.NET. Persisted selection is now supported for the GridView and ListView controls in all projects. You can enable this feature in the GridView control, for example, by setting the EnablePersistedSelection property, as shown in the following example: <asp:GridView id="GridView2" runat="server" PersistedSelection="true"> </asp:GridView> FormView Control Enhancements The FormView control is enhanced to make it easier to style the content of the control with CSS. In previous versions of ASP.NET, the FormView control rendered it contents using an item template. This made styling more difficult in the markup because unexpected table row and table cell tags were rendered by the control. The FormView control supports RenderOuterTable, a property in ASP.NET 4. When this property is set to false, as show in the following example, the table tags are not rendered. This makes it easier to apply CSS style to the contents of the control. <asp:FormView ID="FormView1" runat="server" RenderTable="false"> For more information, see FormView Web Server Control Overview. ListView Control Enhancements The ListView control, which was introduced in ASP.NET 3.5, has all the functionality of the GridView control while giving you complete control over the output. This control has been made easier to use in ASP.NET 4. The earlier version of the control required that you specify a layout template that contained a server control with a known ID. The following markup shows a typical example of how to use the ListView control in ASP.NET 3.5. <asp:ListView ID="ListView1" runat="server"> <LayoutTemplate> <asp:PlaceHolder ID="ItemPlaceHolder" runat="server"></asp:PlaceHolder> </LayoutTemplate> <ItemTemplate> <% Eval("LastName")%> </ItemTemplate> </asp:ListView> In ASP.NET 4, the ListView control does not require a layout template. The markup shown in the previous example can be replaced with the following markup: <asp:ListView ID="ListView1" runat="server"> <ItemTemplate> <% Eval("LastName")%> </ItemTemplate> </asp:ListView> For more information, see ListView Web Server Control Overview. Filtering Data with the QueryExtender Control A very common task for developers who create data-driven Web pages is to filter data. This traditionally has been performed by building Where clauses in data source controls. This approach can be complicated, and in some cases the Where syntax does not let you take advantage of the full functionality of the underlying database. To make filtering easier, a new QueryExtender control has been added in ASP.NET 4. This control can be added to EntityDataSource or LinqDataSource controls in order to filter the data returned by these controls. Because the QueryExtender control relies on LINQ, but you do not to need to know how to write LINQ queries to use the query extender. The QueryExtender control supports a variety of filter options. The following lists QueryExtender filter options. Term Definition SearchExpression Searches a field or fields for string values and compares them to a specified string value. RangeExpression Searches a field or fields for values in a range specified by a pair of values. PropertyExpression Compares a specified value to a property value in a field. If the expression evaluates to true, the data that is being examined is returned. OrderByExpression Sorts data by a specified column and sort direction. CustomExpression Calls a function that defines custom filter in the page. For more information, see QueryExtenderQueryExtender Web Server Control Overview. Enhanced Support for Web Standards and Accessibility Earlier versions of ASP.NET controls sometimes render markup that does not conform to HTML, XHTML, or accessibility standards. ASP.NET 4 eliminates most of these exceptions. For details about how the HTML that is rendered by each control meets accessibility standards, see ASP.NET Controls and Accessibility. CSS for Controls that Can be Disabled In ASP.NET 3.5, when a control is disabled (see WebControl.Enabled), a disabled attribute is added to the rendered HTML element. For example, the following markup creates a Label control that is disabled: <asp:Label id="Label1" runat="server"   Text="Test" Enabled="false" /> In ASP.NET 3.5, the previous control settings generate the following HTML: <span id="Label1" disabled="disabled">Test</span> In HTML 4.01, the disabled attribute is not considered valid on span elements. It is valid only on input elements because it specifies that they cannot be accessed. On display-only elements such as span elements, browsers typically support rendering for a disabled appearance, but a Web page that relies on this non-standard behavior is not robust according to accessibility standards. For display-only elements, you should use CSS to indicate a disabled visual appearance. Therefore, by default ASP.NET 4 generates the following HTML for the control settings shown previously: <span id="Label1" class="aspNetDisabled">Test</span> You can change the value of the class attribute that is rendered by default when a control is disabled by setting the DisabledCssClass property. CSS for Validation Controls In ASP.NET 3.5, validation controls render a default color of red as an inline style. For example, the following markup creates a RequiredFieldValidator control: <asp:RequiredFieldValidator ID="RequiredFieldValidator1" runat="server"   ErrorMessage="Required Field" ControlToValidate="RadioButtonList1" /> ASP.NET 3.5 renders the following HTML for the validator control: <span id="RequiredFieldValidator1"   style="color:Red;visibility:hidden;">RequiredFieldValidator</span> By default, ASP.NET 4 does not render an inline style to set the color to red. An inline style is used only to hide or show the validator, as shown in the following example: <span id="RequiredFieldValidator1"   style"visibility:hidden;">RequiredFieldValidator</span> Therefore, ASP.NET 4 does not automatically show error messages in red. For information about how to use CSS to specify a visual style for a validation control, see Validating User Input in ASP.NET Web Pages. CSS for the Hidden Fields Div Element ASP.NET uses hidden fields to store state information such as view state and control state. These hidden fields are contained by a div element. In ASP.NET 3.5, this div element does not have a class attribute or an id attribute. Therefore, CSS rules that affect all div elements could unintentionally cause this div to be visible. To avoid this problem, ASP.NET 4 renders the div element for hidden fields with a CSS class that you can use to differentiate the hidden fields div from others. The new classvalue is shown in the following example: <div class="aspNetHidden"> CSS for the Table, Image, and ImageButton Controls By default, in ASP.NET 3.5, some controls set the border attribute of rendered HTML to zero (0). The following example shows HTML that is generated by the Table control in ASP.NET 3.5: <table id="Table2" border="0"> The Image control and the ImageButton control also do this. Because this is not necessary and provides visual formatting information that should be provided by using CSS, the attribute is not generated in ASP.NET 4. CSS for the UpdatePanel and UpdateProgress Controls In ASP.NET 3.5, the UpdatePanel and UpdateProgress controls do not support expando attributes. This makes it impossible to set a CSS class on the HTMLelements that they render. In ASP.NET 4 these controls have been changed to accept expando attributes, as shown in the following example: <asp:UpdatePanel runat="server" class="myStyle"> </asp:UpdatePanel> The following HTML is rendered for this markup: <div id="ctl00_MainContent_UpdatePanel1" class="expandoclass"> </div> Eliminating Unnecessary Outer Tables In ASP.NET 3.5, the HTML that is rendered for the following controls is wrapped in a table element whose purpose is to apply inline styles to the entire control: FormView Login PasswordRecovery ChangePassword If you use templates to customize the appearance of these controls, you can specify CSS styles in the markup that you provide in the templates. In that case, no extra outer table is required. In ASP.NET 4, you can prevent the table from being rendered by setting the new RenderOuterTable property to false. Layout Templates for Wizard Controls In ASP.NET 3.5, the Wizard and CreateUserWizard controls generate an HTML table element that is used for visual formatting. In ASP.NET 4 you can use a LayoutTemplate element to specify the layout. If you do this, the HTML table element is not generated. In the template, you create placeholder controls to indicate where items should be dynamically inserted into the control. (This is similar to how the template model for the ListView control works.) For more information, see the Wizard.LayoutTemplate property. New HTML Formatting Options for the CheckBoxList and RadioButtonList Controls ASP.NET 3.5 uses HTML table elements to format the output for the CheckBoxList and RadioButtonList controls. To provide an alternative that does not use tables for visual formatting, ASP.NET 4 adds two new options to the RepeatLayout enumeration: UnorderedList. This option causes the HTML output to be formatted by using ul and li elements instead of a table. OrderedList. This option causes the HTML output to be formatted by using ol and li elements instead of a table. For examples of HTML that is rendered for the new options, see the RepeatLayout enumeration. Header and Footer Elements for the Table Control In ASP.NET 3.5, the Table control can be configured to render thead and tfoot elements by setting the TableSection property of the TableHeaderRow class and the TableFooterRow class. In ASP.NET 4 these properties are set to the appropriate values by default. CSS and ARIA Support for the Menu Control In ASP.NET 3.5, the Menu control uses HTML table elements for visual formatting, and in some configurations it is not keyboard-accessible. ASP.NET 4 addresses these problems and improves accessibility in the following ways: The generated HTML is structured as an unordered list (ul and li elements). CSS is used for visual formatting. The menu behaves in accordance with ARIA standards for keyboard access. You can use arrow keys to navigate menu items. (For information about ARIA, see Accessibility in Visual Studio and ASP.NET.) ARIA role and property attributes are added to the generated HTML. (Attributes are added by using JavaScript instead of included in the HTML, to avoid generating HTML that would cause markup validation errors.) Styles for the Menu control are rendered in a style block at the top of the page, instead of inline with the rendered HTML elements. If you want to use a separate CSS file so that you can modify the menu styles, you can set the Menu control's new IncludeStyleBlock property to false, in which case the style block is not generated. Valid XHTML for the HtmlForm Control In ASP.NET 3.5, the HtmlForm control (which is created implicitly by the <form runat="server"> tag) renders an HTML form element that has both name and id attributes. The name attribute is deprecated in XHTML 1.1. Therefore, this control does not render the name attribute in ASP.NET 4. Maintaining Backward Compatibility in Control Rendering An existing ASP.NET Web site might have code in it that assumes that controls are rendering HTML the way they do in ASP.NET 3.5. To avoid causing backward compatibility problems when you upgrade the site to ASP.NET 4, you can have ASP.NET continue to generate HTML the way it does in ASP.NET 3.5 after you upgrade the site. To do so, you can set the controlRenderingCompatibilityVersion attribute of the pages element to "3.5" in the Web.config file of an ASP.NET 4 Web site, as shown in the following example: <system.web>   <pages controlRenderingCompatibilityVersion="3.5"/> </system.web> If this setting is omitted, the default value is the same as the version of ASP.NET that the Web site targets. (For information about multi-targeting in ASP.NET, see .NET Framework Multi-Targeting for ASP.NET Web Projects.) ASP.NET MVC ASP.NET MVC helps Web developers build compelling standards-based Web sites that are easy to maintain because it decreases the dependency among application layers by using the Model-View-Controller (MVC) pattern. MVC provides complete control over the page markup. It also improves testability by inherently supporting Test Driven Development (TDD). Web sites created using ASP.NET MVC have a modular architecture. This allows members of a team to work independently on the various modules and can be used to improve collaboration. For example, developers can work on the model and controller layers (data and logic), while the designer work on the view (presentation). For tutorials, walkthroughs, conceptual content, code samples, and a complete API reference, see ASP.NET MVC 2. Dynamic Data Dynamic Data was introduced in the .NET Framework 3.5 SP1 release in mid-2008. This feature provides many enhancements for creating data-driven applications, such as the following: A RAD experience for quickly building a data-driven Web site. Automatic validation that is based on constraints defined in the data model. The ability to easily change the markup that is generated for fields in the GridView and DetailsView controls by using field templates that are part of your Dynamic Data project. For ASP.NET 4, Dynamic Data has been enhanced to give developers even more power for quickly building data-driven Web sites. For more information, see ASP.NET Dynamic Data Content Map. Enabling Dynamic Data for Individual Data-Bound Controls in Existing Web Applications You can use Dynamic Data features in existing ASP.NET Web applications that do not use scaffolding by enabling Dynamic Data for individual data-bound controls. Dynamic Data provides the presentation and data layer support for rendering these controls. When you enable Dynamic Data for data-bound controls, you get the following benefits: Setting default values for data fields. Dynamic Data enables you to provide default values at run time for fields in a data control. Interacting with the database without creating and registering a data model. Automatically validating the data that is entered by the user without writing any code. For more information, see Walkthrough: Enabling Dynamic Data in ASP.NET Data-Bound Controls. New Field Templates for URLs and E-mail Addresses ASP.NET 4 introduces two new built-in field templates, EmailAddress.ascx and Url.ascx. These templates are used for fields that are marked as EmailAddress or Url using the DataTypeAttribute attribute. For EmailAddress objects, the field is displayed as a hyperlink that is created by using the mailto: protocol. When users click the link, it opens the user's e-mail client and creates a skeleton message. Objects typed as Url are displayed as ordinary hyperlinks. The following example shows how to mark fields. [DataType(DataType.EmailAddress)] public object HomeEmail { get; set; } [DataType(DataType.Url)] public object Website { get; set; } Creating Links with the DynamicHyperLink Control Dynamic Data uses the new routing feature that was added in the .NET Framework 3.5 SP1 to control the URLs that users see when they access the Web site. The new DynamicHyperLink control makes it easy to build links to pages in a Dynamic Data site. For information, see How to: Create Table Action Links in Dynamic Data Support for Inheritance in the Data Model Both the ADO.NET Entity Framework and LINQ to SQL support inheritance in their data models. An example of this might be a database that has an InsurancePolicy table. It might also contain CarPolicy and HousePolicy tables that have the same fields as InsurancePolicy and then add more fields. Dynamic Data has been modified to understand inherited objects in the data model and to support scaffolding for the inherited tables. For more information, see Walkthrough: Mapping Table-per-Hierarchy Inheritance in Dynamic Data. Support for Many-to-Many Relationships (Entity Framework Only) The Entity Framework has rich support for many-to-many relationships between tables, which is implemented by exposing the relationship as a collection on an Entity object. New field templates (ManyToMany.ascx and ManyToMany_Edit.ascx) have been added to provide support for displaying and editing data that is involved in many-to-many relationships. For more information, see Working with Many-to-Many Data Relationships in Dynamic Data. New Attributes to Control Display and Support Enumerations The DisplayAttribute has been added to give you additional control over how fields are displayed. The DisplayNameAttribute attribute in earlier versions of Dynamic Data enabled you to change the name that is used as a caption for a field. The new DisplayAttribute class lets you specify more options for displaying a field, such as the order in which a field is displayed and whether a field will be used as a filter. The attribute also provides independent control of the name that is used for the labels in a GridView control, the name that is used in a DetailsView control, the help text for the field, and the watermark used for the field (if the field accepts text input). The EnumDataTypeAttribute class has been added to let you map fields to enumerations. When you apply this attribute to a field, you specify an enumeration type. Dynamic Data uses the new Enumeration.ascx field template to create UI for displaying and editing enumeration values. The template maps the values from the database to the names in the enumeration. Enhanced Support for Filters Dynamic Data 1.0 had built-in filters for Boolean columns and foreign-key columns. The filters did not let you specify the order in which they were displayed. The new DisplayAttribute attribute addresses this by giving you control over whether a column appears as a filter and in what order it will be displayed. An additional enhancement is that filtering support has been rewritten to use the new QueryExtender feature of Web Forms. This lets you create filters without requiring knowledge of the data source control that the filters will be used with. Along with these extensions, filters have also been turned into template controls, which lets you add new ones. Finally, the DisplayAttribute class mentioned earlier allows the default filter to be overridden, in the same way that UIHint allows the default field template for a column to be overridden. For more information, see Walkthrough: Filtering Rows in Tables That Have a Parent-Child Relationship and QueryableFilterRepeater. ASP.NET Chart Control The ASP.NET chart server control enables you to create ASP.NET pages applications that have simple, intuitive charts for complex statistical or financial analysis. The chart control supports the following features: Data series, chart areas, axes, legends, labels, titles, and more. Data binding. Data manipulation, such as copying, splitting, merging, alignment, grouping, sorting, searching, and filtering. Statistical formulas and financial formulas. Advanced chart appearance, such as 3-D, anti-aliasing, lighting, and perspective. Events and customizations. Interactivity and Microsoft Ajax. Support for the Ajax Content Delivery Network (CDN), which provides an optimized way for you to add Microsoft Ajax Library and jQuery scripts to your Web applications. For more information, see Chart Web Server Control Overview. Visual Web Developer Enhancements The following sections provide information about enhancements and new features in Visual Studio 2010 and Visual Web Developer Express. The Web page designer in Visual Studio 2010 has been enhanced for better CSS compatibility, includes additional support for HTML and ASP.NET markup snippets, and features a redesigned version of IntelliSense for JScript. Improved CSS Compatibility The Visual Web Developer designer in Visual Studio 2010 has been updated to improve CSS 2.1 standards compliance. The designer better preserves HTML source code and is more robust than in previous versions of Visual Studio. HTML and JScript Snippets In the HTML editor, IntelliSense auto-completes tag names. The IntelliSense Snippets feature auto-completes whole tags and more. In Visual Studio 2010, IntelliSense snippets are supported for JScript, alongside C# and Visual Basic, which were supported in earlier versions of Visual Studio. Visual Studio 2010 includes over 200 snippets that help you auto-complete common ASP.NET and HTML tags, including required attributes (such as runat="server") and common attributes specific to a tag (such as ID, DataSourceID, ControlToValidate, and Text). You can download additional snippets, or you can write your own snippets that encapsulate the blocks of markup that you or your team use for common tasks. For more information on HTML snippets, see Walkthrough: Using HTML Snippets. JScript IntelliSense Enhancements In Visual 2010, JScript IntelliSense has been redesigned to provide an even richer editing experience. IntelliSense now recognizes objects that have been dynamically generated by methods such as registerNamespace and by similar techniques used by other JavaScript frameworks. Performance has been improved to analyze large libraries of script and to display IntelliSense with little or no processing delay. Compatibility has been significantly increased to support almost all third-party libraries and to support diverse coding styles. Documentation comments are now parsed as you type and are immediately leveraged by IntelliSense. Web Application Deployment with Visual Studio 2010 For Web application projects, Visual Studio now provides tools that work with the IIS Web Deployment Tool (Web Deploy) to automate many processes that had to be done manually in earlier versions of ASP.NET. For example, the following tasks can now be automated: Creating an IIS application on the destination computer and configuring IIS settings. Copying files to the destination computer. Changing Web.config settings that must be different in the destination environment. Propagating changes to data or data structures in SQL Server databases that are used by the Web application. For more information about Web application deployment, see ASP.NET Deployment Content Map. Enhancements to ASP.NET Multi-Targeting ASP.NET 4 adds new features to the multi-targeting feature to make it easier to work with projects that target earlier versions of the .NET Framework. Multi-targeting was introduced in ASP.NET 3.5 to enable you to use the latest version of Visual Studio without having to upgrade existing Web sites or Web services to the latest version of the .NET Framework. In Visual Studio 2008, when you work with a project targeted for an earlier version of the .NET Framework, most features of the development environment adapt to the targeted version. However, IntelliSense displays language features that are available in the current version, and property windows display properties available in the current version. In Visual Studio 2010, only language features and properties available in the targeted version of the .NET Framework are shown. For more information about multi-targeting, see the following topics: .NET Framework Multi-Targeting for ASP.NET Web Projects ASP.NET Side-by-Side Execution Overview How to: Host Web Applications That Use Different Versions of the .NET Framework on the Same Server How to: Deploy Web Site Projects Targeted for Earlier Versions of the .NET Framework

    Read the article

  • How John Got 15x Improvement Without Really Trying

    - by rchrd
    The following article was published on a Sun Microsystems website a number of years ago by John Feo. It is still useful and worth preserving. So I'm republishing it here.  How I Got 15x Improvement Without Really Trying John Feo, Sun Microsystems Taking ten "personal" program codes used in scientific and engineering research, the author was able to get from 2 to 15 times performance improvement easily by applying some simple general optimization techniques. Introduction Scientific research based on computer simulation depends on the simulation for advancement. The research can advance only as fast as the computational codes can execute. The codes' efficiency determines both the rate and quality of results. In the same amount of time, a faster program can generate more results and can carry out a more detailed simulation of physical phenomena than a slower program. Highly optimized programs help science advance quickly and insure that monies supporting scientific research are used as effectively as possible. Scientific computer codes divide into three broad categories: ISV, community, and personal. ISV codes are large, mature production codes developed and sold commercially. The codes improve slowly over time both in methods and capabilities, and they are well tuned for most vendor platforms. Since the codes are mature and complex, there are few opportunities to improve their performance solely through code optimization. Improvements of 10% to 15% are typical. Examples of ISV codes are DYNA3D, Gaussian, and Nastran. Community codes are non-commercial production codes used by a particular research field. Generally, they are developed and distributed by a single academic or research institution with assistance from the community. Most users just run the codes, but some develop new methods and extensions that feed back into the general release. The codes are available on most vendor platforms. Since these codes are younger than ISV codes, there are more opportunities to optimize the source code. Improvements of 50% are not unusual. Examples of community codes are AMBER, CHARM, BLAST, and FASTA. Personal codes are those written by single users or small research groups for their own use. These codes are not distributed, but may be passed from professor-to-student or student-to-student over several years. They form the primordial ocean of applications from which community and ISV codes emerge. Government research grants pay for the development of most personal codes. This paper reports on the nature and performance of this class of codes. Over the last year, I have looked at over two dozen personal codes from more than a dozen research institutions. The codes cover a variety of scientific fields, including astronomy, atmospheric sciences, bioinformatics, biology, chemistry, geology, and physics. The sources range from a few hundred lines to more than ten thousand lines, and are written in Fortran, Fortran 90, C, and C++. For the most part, the codes are modular, documented, and written in a clear, straightforward manner. They do not use complex language features, advanced data structures, programming tricks, or libraries. I had little trouble understanding what the codes did or how data structures were used. Most came with a makefile. Surprisingly, only one of the applications is parallel. All developers have access to parallel machines, so availability is not an issue. Several tried to parallelize their applications, but stopped after encountering difficulties. Lack of education and a perception that parallelism is difficult prevented most from trying. I parallelized several of the codes using OpenMP, and did not judge any of the codes as difficult to parallelize. Even more surprising than the lack of parallelism is the inefficiency of the codes. I was able to get large improvements in performance in a matter of a few days applying simple optimization techniques. Table 1 lists ten representative codes [names and affiliation are omitted to preserve anonymity]. Improvements on one processor range from 2x to 15.5x with a simple average of 4.75x. I did not use sophisticated performance tools or drill deep into the program's execution character as one would do when tuning ISV or community codes. Using only a profiler and source line timers, I identified inefficient sections of code and improved their performance by inspection. The changes were at a high level. I am sure there is another factor of 2 or 3 in each code, and more if the codes are parallelized. The study’s results show that personal scientific codes are running many times slower than they should and that the problem is pervasive. Computational scientists are not sloppy programmers; however, few are trained in the art of computer programming or code optimization. I found that most have a working knowledge of some programming language and standard software engineering practices; but they do not know, or think about, how to make their programs run faster. They simply do not know the standard techniques used to make codes run faster. In fact, they do not even perceive that such techniques exist. The case studies described in this paper show that applying simple, well known techniques can significantly increase the performance of personal codes. It is important that the scientific community and the Government agencies that support scientific research find ways to better educate academic scientific programmers. The inefficiency of their codes is so bad that it is retarding both the quality and progress of scientific research. # cacheperformance redundantoperations loopstructures performanceimprovement 1 x x 15.5 2 x 2.8 3 x x 2.5 4 x 2.1 5 x x 2.0 6 x 5.0 7 x 5.8 8 x 6.3 9 2.2 10 x x 3.3 Table 1 — Area of improvement and performance gains of 10 codes The remainder of the paper is organized as follows: sections 2, 3, and 4 discuss the three most common sources of inefficiencies in the codes studied. These are cache performance, redundant operations, and loop structures. Each section includes several examples. The last section summaries the work and suggests a possible solution to the issues raised. Optimizing cache performance Commodity microprocessor systems use caches to increase memory bandwidth and reduce memory latencies. Typical latencies from processor to L1, L2, local, and remote memory are 3, 10, 50, and 200 cycles, respectively. Moreover, bandwidth falls off dramatically as memory distances increase. Programs that do not use cache effectively run many times slower than programs that do. When optimizing for cache, the biggest performance gains are achieved by accessing data in cache order and reusing data to amortize the overhead of cache misses. Secondary considerations are prefetching, associativity, and replacement; however, the understanding and analysis required to optimize for the latter are probably beyond the capabilities of the non-expert. Much can be gained simply by accessing data in the correct order and maximizing data reuse. 6 out of the 10 codes studied here benefited from such high level optimizations. Array Accesses The most important cache optimization is the most basic: accessing Fortran array elements in column order and C array elements in row order. Four of the ten codes—1, 2, 4, and 10—got it wrong. Compilers will restructure nested loops to optimize cache performance, but may not do so if the loop structure is too complex, or the loop body includes conditionals, complex addressing, or function calls. In code 1, the compiler failed to invert a key loop because of complex addressing do I = 0, 1010, delta_x IM = I - delta_x IP = I + delta_x do J = 5, 995, delta_x JM = J - delta_x JP = J + delta_x T1 = CA1(IP, J) + CA1(I, JP) T2 = CA1(IM, J) + CA1(I, JM) S1 = T1 + T2 - 4 * CA1(I, J) CA(I, J) = CA1(I, J) + D * S1 end do end do In code 2, the culprit is conditionals do I = 1, N do J = 1, N If (IFLAG(I,J) .EQ. 0) then T1 = Value(I, J-1) T2 = Value(I-1, J) T3 = Value(I, J) T4 = Value(I+1, J) T5 = Value(I, J+1) Value(I,J) = 0.25 * (T1 + T2 + T5 + T4) Delta = ABS(T3 - Value(I,J)) If (Delta .GT. MaxDelta) MaxDelta = Delta endif enddo enddo I fixed both programs by inverting the loops by hand. Code 10 has three-dimensional arrays and triply nested loops. The structure of the most computationally intensive loops is too complex to invert automatically or by hand. The only practical solution is to transpose the arrays so that the dimension accessed by the innermost loop is in cache order. The arrays can be transposed at construction or prior to entering a computationally intensive section of code. The former requires all array references to be modified, while the latter is cost effective only if the cost of the transpose is amortized over many accesses. I used the second approach to optimize code 10. Code 5 has four-dimensional arrays and loops are nested four deep. For all of the reasons cited above the compiler is not able to restructure three key loops. Assume C arrays and let the four dimensions of the arrays be i, j, k, and l. In the original code, the index structure of the three loops is L1: for i L2: for i L3: for i for l for l for j for k for j for k for j for k for l So only L3 accesses array elements in cache order. L1 is a very complex loop—much too complex to invert. I brought the loop into cache alignment by transposing the second and fourth dimensions of the arrays. Since the code uses a macro to compute all array indexes, I effected the transpose at construction and changed the macro appropriately. The dimensions of the new arrays are now: i, l, k, and j. L3 is a simple loop and easily inverted. L2 has a loop-carried scalar dependence in k. By promoting the scalar name that carries the dependence to an array, I was able to invert the third and fourth subloops aligning the loop with cache. Code 5 is by far the most difficult of the four codes to optimize for array accesses; but the knowledge required to fix the problems is no more than that required for the other codes. I would judge this code at the limits of, but not beyond, the capabilities of appropriately trained computational scientists. Array Strides When a cache miss occurs, a line (64 bytes) rather than just one word is loaded into the cache. If data is accessed stride 1, than the cost of the miss is amortized over 8 words. Any stride other than one reduces the cost savings. Two of the ten codes studied suffered from non-unit strides. The codes represent two important classes of "strided" codes. Code 1 employs a multi-grid algorithm to reduce time to convergence. The grids are every tenth, fifth, second, and unit element. Since time to convergence is inversely proportional to the distance between elements, coarse grids converge quickly providing good starting values for finer grids. The better starting values further reduce the time to convergence. The downside is that grids of every nth element, n > 1, introduce non-unit strides into the computation. In the original code, much of the savings of the multi-grid algorithm were lost due to this problem. I eliminated the problem by compressing (copying) coarse grids into continuous memory, and rewriting the computation as a function of the compressed grid. On convergence, I copied the final values of the compressed grid back to the original grid. The savings gained from unit stride access of the compressed grid more than paid for the cost of copying. Using compressed grids, the loop from code 1 included in the previous section becomes do j = 1, GZ do i = 1, GZ T1 = CA(i+0, j-1) + CA(i-1, j+0) T4 = CA1(i+1, j+0) + CA1(i+0, j+1) S1 = T1 + T4 - 4 * CA1(i+0, j+0) CA(i+0, j+0) = CA1(i+0, j+0) + DD * S1 enddo enddo where CA and CA1 are compressed arrays of size GZ. Code 7 traverses a list of objects selecting objects for later processing. The labels of the selected objects are stored in an array. The selection step has unit stride, but the processing steps have irregular stride. A fix is to save the parameters of the selected objects in temporary arrays as they are selected, and pass the temporary arrays to the processing functions. The fix is practical if the same parameters are used in selection as in processing, or if processing comprises a series of distinct steps which use overlapping subsets of the parameters. Both conditions are true for code 7, so I achieved significant improvement by copying parameters to temporary arrays during selection. Data reuse In the previous sections, we optimized for spatial locality. It is also important to optimize for temporal locality. Once read, a datum should be used as much as possible before it is forced from cache. Loop fusion and loop unrolling are two techniques that increase temporal locality. Unfortunately, both techniques increase register pressure—as loop bodies become larger, the number of registers required to hold temporary values grows. Once register spilling occurs, any gains evaporate quickly. For multiprocessors with small register sets or small caches, the sweet spot can be very small. In the ten codes presented here, I found no opportunities for loop fusion and only two opportunities for loop unrolling (codes 1 and 3). In code 1, unrolling the outer and inner loop one iteration increases the number of result values computed by the loop body from 1 to 4, do J = 1, GZ-2, 2 do I = 1, GZ-2, 2 T1 = CA1(i+0, j-1) + CA1(i-1, j+0) T2 = CA1(i+1, j-1) + CA1(i+0, j+0) T3 = CA1(i+0, j+0) + CA1(i-1, j+1) T4 = CA1(i+1, j+0) + CA1(i+0, j+1) T5 = CA1(i+2, j+0) + CA1(i+1, j+1) T6 = CA1(i+1, j+1) + CA1(i+0, j+2) T7 = CA1(i+2, j+1) + CA1(i+1, j+2) S1 = T1 + T4 - 4 * CA1(i+0, j+0) S2 = T2 + T5 - 4 * CA1(i+1, j+0) S3 = T3 + T6 - 4 * CA1(i+0, j+1) S4 = T4 + T7 - 4 * CA1(i+1, j+1) CA(i+0, j+0) = CA1(i+0, j+0) + DD * S1 CA(i+1, j+0) = CA1(i+1, j+0) + DD * S2 CA(i+0, j+1) = CA1(i+0, j+1) + DD * S3 CA(i+1, j+1) = CA1(i+1, j+1) + DD * S4 enddo enddo The loop body executes 12 reads, whereas as the rolled loop shown in the previous section executes 20 reads to compute the same four values. In code 3, two loops are unrolled 8 times and one loop is unrolled 4 times. Here is the before for (k = 0; k < NK[u]; k++) { sum = 0.0; for (y = 0; y < NY; y++) { sum += W[y][u][k] * delta[y]; } backprop[i++]=sum; } and after code for (k = 0; k < KK - 8; k+=8) { sum0 = 0.0; sum1 = 0.0; sum2 = 0.0; sum3 = 0.0; sum4 = 0.0; sum5 = 0.0; sum6 = 0.0; sum7 = 0.0; for (y = 0; y < NY; y++) { sum0 += W[y][0][k+0] * delta[y]; sum1 += W[y][0][k+1] * delta[y]; sum2 += W[y][0][k+2] * delta[y]; sum3 += W[y][0][k+3] * delta[y]; sum4 += W[y][0][k+4] * delta[y]; sum5 += W[y][0][k+5] * delta[y]; sum6 += W[y][0][k+6] * delta[y]; sum7 += W[y][0][k+7] * delta[y]; } backprop[k+0] = sum0; backprop[k+1] = sum1; backprop[k+2] = sum2; backprop[k+3] = sum3; backprop[k+4] = sum4; backprop[k+5] = sum5; backprop[k+6] = sum6; backprop[k+7] = sum7; } for one of the loops unrolled 8 times. Optimizing for temporal locality is the most difficult optimization considered in this paper. The concepts are not difficult, but the sweet spot is small. Identifying where the program can benefit from loop unrolling or loop fusion is not trivial. Moreover, it takes some effort to get it right. Still, educating scientific programmers about temporal locality and teaching them how to optimize for it will pay dividends. Reducing instruction count Execution time is a function of instruction count. Reduce the count and you usually reduce the time. The best solution is to use a more efficient algorithm; that is, an algorithm whose order of complexity is smaller, that converges quicker, or is more accurate. Optimizing source code without changing the algorithm yields smaller, but still significant, gains. This paper considers only the latter because the intent is to study how much better codes can run if written by programmers schooled in basic code optimization techniques. The ten codes studied benefited from three types of "instruction reducing" optimizations. The two most prevalent were hoisting invariant memory and data operations out of inner loops. The third was eliminating unnecessary data copying. The nature of these inefficiencies is language dependent. Memory operations The semantics of C make it difficult for the compiler to determine all the invariant memory operations in a loop. The problem is particularly acute for loops in functions since the compiler may not know the values of the function's parameters at every call site when compiling the function. Most compilers support pragmas to help resolve ambiguities; however, these pragmas are not comprehensive and there is no standard syntax. To guarantee that invariant memory operations are not executed repetitively, the user has little choice but to hoist the operations by hand. The problem is not as severe in Fortran programs because in the absence of equivalence statements, it is a violation of the language's semantics for two names to share memory. Codes 3 and 5 are C programs. In both cases, the compiler did not hoist all invariant memory operations from inner loops. Consider the following loop from code 3 for (y = 0; y < NY; y++) { i = 0; for (u = 0; u < NU; u++) { for (k = 0; k < NK[u]; k++) { dW[y][u][k] += delta[y] * I1[i++]; } } } Since dW[y][u] can point to the same memory space as delta for one or more values of y and u, assignment to dW[y][u][k] may change the value of delta[y]. In reality, dW and delta do not overlap in memory, so I rewrote the loop as for (y = 0; y < NY; y++) { i = 0; Dy = delta[y]; for (u = 0; u < NU; u++) { for (k = 0; k < NK[u]; k++) { dW[y][u][k] += Dy * I1[i++]; } } } Failure to hoist invariant memory operations may be due to complex address calculations. If the compiler can not determine that the address calculation is invariant, then it can hoist neither the calculation nor the associated memory operations. As noted above, code 5 uses a macro to address four-dimensional arrays #define MAT4D(a,q,i,j,k) (double *)((a)->data + (q)*(a)->strides[0] + (i)*(a)->strides[3] + (j)*(a)->strides[2] + (k)*(a)->strides[1]) The macro is too complex for the compiler to understand and so, it does not identify any subexpressions as loop invariant. The simplest way to eliminate the address calculation from the innermost loop (over i) is to define a0 = MAT4D(a,q,0,j,k) before the loop and then replace all instances of *MAT4D(a,q,i,j,k) in the loop with a0[i] A similar problem appears in code 6, a Fortran program. The key loop in this program is do n1 = 1, nh nx1 = (n1 - 1) / nz + 1 nz1 = n1 - nz * (nx1 - 1) do n2 = 1, nh nx2 = (n2 - 1) / nz + 1 nz2 = n2 - nz * (nx2 - 1) ndx = nx2 - nx1 ndy = nz2 - nz1 gxx = grn(1,ndx,ndy) gyy = grn(2,ndx,ndy) gxy = grn(3,ndx,ndy) balance(n1,1) = balance(n1,1) + (force(n2,1) * gxx + force(n2,2) * gxy) * h1 balance(n1,2) = balance(n1,2) + (force(n2,1) * gxy + force(n2,2) * gyy)*h1 end do end do The programmer has written this loop well—there are no loop invariant operations with respect to n1 and n2. However, the loop resides within an iterative loop over time and the index calculations are independent with respect to time. Trading space for time, I precomputed the index values prior to the entering the time loop and stored the values in two arrays. I then replaced the index calculations with reads of the arrays. Data operations Ways to reduce data operations can appear in many forms. Implementing a more efficient algorithm produces the biggest gains. The closest I came to an algorithm change was in code 4. This code computes the inner product of K-vectors A(i) and B(j), 0 = i < N, 0 = j < M, for most values of i and j. Since the program computes most of the NM possible inner products, it is more efficient to compute all the inner products in one triply-nested loop rather than one at a time when needed. The savings accrue from reading A(i) once for all B(j) vectors and from loop unrolling. for (i = 0; i < N; i+=8) { for (j = 0; j < M; j++) { sum0 = 0.0; sum1 = 0.0; sum2 = 0.0; sum3 = 0.0; sum4 = 0.0; sum5 = 0.0; sum6 = 0.0; sum7 = 0.0; for (k = 0; k < K; k++) { sum0 += A[i+0][k] * B[j][k]; sum1 += A[i+1][k] * B[j][k]; sum2 += A[i+2][k] * B[j][k]; sum3 += A[i+3][k] * B[j][k]; sum4 += A[i+4][k] * B[j][k]; sum5 += A[i+5][k] * B[j][k]; sum6 += A[i+6][k] * B[j][k]; sum7 += A[i+7][k] * B[j][k]; } C[i+0][j] = sum0; C[i+1][j] = sum1; C[i+2][j] = sum2; C[i+3][j] = sum3; C[i+4][j] = sum4; C[i+5][j] = sum5; C[i+6][j] = sum6; C[i+7][j] = sum7; }} This change requires knowledge of a typical run; i.e., that most inner products are computed. The reasons for the change, however, derive from basic optimization concepts. It is the type of change easily made at development time by a knowledgeable programmer. In code 5, we have the data version of the index optimization in code 6. Here a very expensive computation is a function of the loop indices and so cannot be hoisted out of the loop; however, the computation is invariant with respect to an outer iterative loop over time. We can compute its value for each iteration of the computation loop prior to entering the time loop and save the values in an array. The increase in memory required to store the values is small in comparison to the large savings in time. The main loop in Code 8 is doubly nested. The inner loop includes a series of guarded computations; some are a function of the inner loop index but not the outer loop index while others are a function of the outer loop index but not the inner loop index for (j = 0; j < N; j++) { for (i = 0; i < M; i++) { r = i * hrmax; R = A[j]; temp = (PRM[3] == 0.0) ? 1.0 : pow(r, PRM[3]); high = temp * kcoeff * B[j] * PRM[2] * PRM[4]; low = high * PRM[6] * PRM[6] / (1.0 + pow(PRM[4] * PRM[6], 2.0)); kap = (R > PRM[6]) ? high * R * R / (1.0 + pow(PRM[4]*r, 2.0) : low * pow(R/PRM[6], PRM[5]); < rest of loop omitted > }} Note that the value of temp is invariant to j. Thus, we can hoist the computation for temp out of the loop and save its values in an array. for (i = 0; i < M; i++) { r = i * hrmax; TEMP[i] = pow(r, PRM[3]); } [N.B. – the case for PRM[3] = 0 is omitted and will be reintroduced later.] We now hoist out of the inner loop the computations invariant to i. Since the conditional guarding the value of kap is invariant to i, it behooves us to hoist the computation out of the inner loop, thereby executing the guard once rather than M times. The final version of the code is for (j = 0; j < N; j++) { R = rig[j] / 1000.; tmp1 = kcoeff * par[2] * beta[j] * par[4]; tmp2 = 1.0 + (par[4] * par[4] * par[6] * par[6]); tmp3 = 1.0 + (par[4] * par[4] * R * R); tmp4 = par[6] * par[6] / tmp2; tmp5 = R * R / tmp3; tmp6 = pow(R / par[6], par[5]); if ((par[3] == 0.0) && (R > par[6])) { for (i = 1; i <= imax1; i++) KAP[i] = tmp1 * tmp5; } else if ((par[3] == 0.0) && (R <= par[6])) { for (i = 1; i <= imax1; i++) KAP[i] = tmp1 * tmp4 * tmp6; } else if ((par[3] != 0.0) && (R > par[6])) { for (i = 1; i <= imax1; i++) KAP[i] = tmp1 * TEMP[i] * tmp5; } else if ((par[3] != 0.0) && (R <= par[6])) { for (i = 1; i <= imax1; i++) KAP[i] = tmp1 * TEMP[i] * tmp4 * tmp6; } for (i = 0; i < M; i++) { kap = KAP[i]; r = i * hrmax; < rest of loop omitted > } } Maybe not the prettiest piece of code, but certainly much more efficient than the original loop, Copy operations Several programs unnecessarily copy data from one data structure to another. This problem occurs in both Fortran and C programs, although it manifests itself differently in the two languages. Code 1 declares two arrays—one for old values and one for new values. At the end of each iteration, the array of new values is copied to the array of old values to reset the data structures for the next iteration. This problem occurs in Fortran programs not included in this study and in both Fortran 77 and Fortran 90 code. Introducing pointers to the arrays and swapping pointer values is an obvious way to eliminate the copying; but pointers is not a feature that many Fortran programmers know well or are comfortable using. An easy solution not involving pointers is to extend the dimension of the value array by 1 and use the last dimension to differentiate between arrays at different times. For example, if the data space is N x N, declare the array (N, N, 2). Then store the problem’s initial values in (_, _, 2) and define the scalar names new = 2 and old = 1. At the start of each iteration, swap old and new to reset the arrays. The old–new copy problem did not appear in any C program. In programs that had new and old values, the code swapped pointers to reset data structures. Where unnecessary coping did occur is in structure assignment and parameter passing. Structures in C are handled much like scalars. Assignment causes the data space of the right-hand name to be copied to the data space of the left-hand name. Similarly, when a structure is passed to a function, the data space of the actual parameter is copied to the data space of the formal parameter. If the structure is large and the assignment or function call is in an inner loop, then copying costs can grow quite large. While none of the ten programs considered here manifested this problem, it did occur in programs not included in the study. A simple fix is always to refer to structures via pointers. Optimizing loop structures Since scientific programs spend almost all their time in loops, efficient loops are the key to good performance. Conditionals, function calls, little instruction level parallelism, and large numbers of temporary values make it difficult for the compiler to generate tightly packed, highly efficient code. Conditionals and function calls introduce jumps that disrupt code flow. Users should eliminate or isolate conditionls to their own loops as much as possible. Often logical expressions can be substituted for if-then-else statements. For example, code 2 includes the following snippet MaxDelta = 0.0 do J = 1, N do I = 1, M < code omitted > Delta = abs(OldValue ? NewValue) if (Delta > MaxDelta) MaxDelta = Delta enddo enddo if (MaxDelta .gt. 0.001) goto 200 Since the only use of MaxDelta is to control the jump to 200 and all that matters is whether or not it is greater than 0.001, I made MaxDelta a boolean and rewrote the snippet as MaxDelta = .false. do J = 1, N do I = 1, M < code omitted > Delta = abs(OldValue ? NewValue) MaxDelta = MaxDelta .or. (Delta .gt. 0.001) enddo enddo if (MaxDelta) goto 200 thereby, eliminating the conditional expression from the inner loop. A microprocessor can execute many instructions per instruction cycle. Typically, it can execute one or more memory, floating point, integer, and jump operations. To be executed simultaneously, the operations must be independent. Thick loops tend to have more instruction level parallelism than thin loops. Moreover, they reduce memory traffice by maximizing data reuse. Loop unrolling and loop fusion are two techniques to increase the size of loop bodies. Several of the codes studied benefitted from loop unrolling, but none benefitted from loop fusion. This observation is not too surpising since it is the general tendency of programmers to write thick loops. As loops become thicker, the number of temporary values grows, increasing register pressure. If registers spill, then memory traffic increases and code flow is disrupted. A thick loop with many temporary values may execute slower than an equivalent series of thin loops. The biggest gain will be achieved if the thick loop can be split into a series of independent loops eliminating the need to write and read temporary arrays. I found such an occasion in code 10 where I split the loop do i = 1, n do j = 1, m A24(j,i)= S24(j,i) * T24(j,i) + S25(j,i) * U25(j,i) B24(j,i)= S24(j,i) * T25(j,i) + S25(j,i) * U24(j,i) A25(j,i)= S24(j,i) * C24(j,i) + S25(j,i) * V24(j,i) B25(j,i)= S24(j,i) * U25(j,i) + S25(j,i) * V25(j,i) C24(j,i)= S26(j,i) * T26(j,i) + S27(j,i) * U26(j,i) D24(j,i)= S26(j,i) * T27(j,i) + S27(j,i) * V26(j,i) C25(j,i)= S27(j,i) * S28(j,i) + S26(j,i) * U28(j,i) D25(j,i)= S27(j,i) * T28(j,i) + S26(j,i) * V28(j,i) end do end do into two disjoint loops do i = 1, n do j = 1, m A24(j,i)= S24(j,i) * T24(j,i) + S25(j,i) * U25(j,i) B24(j,i)= S24(j,i) * T25(j,i) + S25(j,i) * U24(j,i) A25(j,i)= S24(j,i) * C24(j,i) + S25(j,i) * V24(j,i) B25(j,i)= S24(j,i) * U25(j,i) + S25(j,i) * V25(j,i) end do end do do i = 1, n do j = 1, m C24(j,i)= S26(j,i) * T26(j,i) + S27(j,i) * U26(j,i) D24(j,i)= S26(j,i) * T27(j,i) + S27(j,i) * V26(j,i) C25(j,i)= S27(j,i) * S28(j,i) + S26(j,i) * U28(j,i) D25(j,i)= S27(j,i) * T28(j,i) + S26(j,i) * V28(j,i) end do end do Conclusions Over the course of the last year, I have had the opportunity to work with over two dozen academic scientific programmers at leading research universities. Their research interests span a broad range of scientific fields. Except for two programs that relied almost exclusively on library routines (matrix multiply and fast Fourier transform), I was able to improve significantly the single processor performance of all codes. Improvements range from 2x to 15.5x with a simple average of 4.75x. Changes to the source code were at a very high level. I did not use sophisticated techniques or programming tools to discover inefficiencies or effect the changes. Only one code was parallel despite the availability of parallel systems to all developers. Clearly, we have a problem—personal scientific research codes are highly inefficient and not running parallel. The developers are unaware of simple optimization techniques to make programs run faster. They lack education in the art of code optimization and parallel programming. I do not believe we can fix the problem by publishing additional books or training manuals. To date, the developers in questions have not studied the books or manual available, and are unlikely to do so in the future. Short courses are a possible solution, but I believe they are too concentrated to be much use. The general concepts can be taught in a three or four day course, but that is not enough time for students to practice what they learn and acquire the experience to apply and extend the concepts to their codes. Practice is the key to becoming proficient at optimization. I recommend that graduate students be required to take a semester length course in optimization and parallel programming. We would never give someone access to state-of-the-art scientific equipment costing hundreds of thousands of dollars without first requiring them to demonstrate that they know how to use the equipment. Yet the criterion for time on state-of-the-art supercomputers is at most an interesting project. Requestors are never asked to demonstrate that they know how to use the system, or can use the system effectively. A semester course would teach them the required skills. Government agencies that fund academic scientific research pay for most of the computer systems supporting scientific research as well as the development of most personal scientific codes. These agencies should require graduate schools to offer a course in optimization and parallel programming as a requirement for funding. About the Author John Feo received his Ph.D. in Computer Science from The University of Texas at Austin in 1986. After graduate school, Dr. Feo worked at Lawrence Livermore National Laboratory where he was the Group Leader of the Computer Research Group and principal investigator of the Sisal Language Project. In 1997, Dr. Feo joined Tera Computer Company where he was project manager for the MTA, and oversaw the programming and evaluation of the MTA at the San Diego Supercomputer Center. In 2000, Dr. Feo joined Sun Microsystems as an HPC application specialist. He works with university research groups to optimize and parallelize scientific codes. Dr. Feo has published over two dozen research articles in the areas of parallel parallel programming, parallel programming languages, and application performance.

    Read the article

  • Optimizing transition/movement smoothness for a 2D flash game.

    - by Tom
    Update 6: Fenomenas suggested me to re-create everything as simple as possible. I had my doubts that this would make any difference as the algorithm remains the same, and performance did not seem to be the issue. Anyway, it was the only suggestion I got so here it is: 30 FPS: http://www.feedpostal.com/test/simple/30/SimpleMovement.html 40 FPS: http://www.feedpostal.com/test/simple/40/SimpleMovement.html 60 FPS: http://www.feedpostal.com/test/simple/60/SimpleMovement.html 100 FPS: http://www.feedpostal.com/test/simple/100/SimpleMovement.html The code: package { import flash.display.Sprite; import flash.events.Event; import flash.events.KeyboardEvent; import flash.utils.getTimer; [SWF(width="800", height="600", frameRate="40", backgroundColor="#000000")] public class SimpleMovement extends Sprite { private static const TURNING_SPEED:uint = 180; private static const MOVEMENT_SPEED:uint = 400; private static const RADIAN_DIVIDE:Number = Math.PI/180; private var playerObject:Sprite; private var shipContainer:Sprite; private var moving:Boolean = false; private var turningMode:uint = 0; private var movementTimestamp:Number = getTimer(); private var turningTimestamp:Number = movementTimestamp; public function SimpleMovement() { //step 1: create player object playerObject = new Sprite(); playerObject.graphics.lineStyle(1, 0x000000); playerObject.graphics.beginFill(0x6D7B8D); playerObject.graphics.drawRect(0, 0, 25, 50); //make it rotate around the center playerObject.x = 0 - playerObject.width / 2; playerObject.y = 0 - playerObject.height / 2; shipContainer = new Sprite(); shipContainer.addChild(playerObject); shipContainer.x = 100; shipContainer.y = 100; shipContainer.rotation = 180; addChild(shipContainer); //step 2: install keyboard hook when stage is ready addEventListener(Event.ADDED_TO_STAGE, stageReady, false, 0, true); //step 3: install rendering update poll addEventListener(Event.ENTER_FRAME, updatePoller, false, 0, true); } private function updatePoller(event:Event):void { var newTime:Number = getTimer(); //turning if (turningMode != 0) { var turningDeltaTime:Number = newTime - turningTimestamp; turningTimestamp = newTime; var rotation:Number = TURNING_SPEED * turningDeltaTime / 1000; if (turningMode == 1) shipContainer.rotation -= rotation; else shipContainer.rotation += rotation; } //movement if (moving) { var movementDeltaTime:Number = newTime - movementTimestamp; movementTimestamp = newTime; var distance:Number = MOVEMENT_SPEED * movementDeltaTime / 1000; var rAngle:Number = shipContainer.rotation * RADIAN_DIVIDE; //convert degrees to radian shipContainer.x += distance * Math.sin(rAngle); shipContainer.y -= distance * Math.cos(rAngle); } } private function stageReady(event:Event):void { //install keyboard hook stage.addEventListener(KeyboardEvent.KEY_DOWN, keyDown, false, 0, true); stage.addEventListener(KeyboardEvent.KEY_UP, keyUp, false, 0, true); } private final function keyDown(event:KeyboardEvent):void { if ((event.keyCode == 87) && (!moving)) //87 = W { movementTimestamp = getTimer(); moving = true; } if ((event.keyCode == 65) && (turningMode != 1)) //65 = A { turningTimestamp = getTimer(); turningMode = 1; } else if ((event.keyCode == 68) && (turningMode != 2)) //68 = D { turningTimestamp = getTimer(); turningMode = 2; } } private final function keyUp(event:KeyboardEvent):void { if ((event.keyCode == 87) && (moving)) moving = false; //87 = W if (((event.keyCode == 65) || (event.keyCode == 68)) && (turningMode != 0)) turningMode = 0; //65 = A, 68 = D } } } The results were as I expected. Absolutely no improvement. I really hope that someone has another suggestion as this thing needs fixing. Also, I doubt it's my system as I have a pretty good one (8GB RAM, Q9550 QuadCore intel, ATI Radeon 4870 512MB). Also, everyone else I asked so far had the same issue with my client. Update 5: another example of a smooth flash game just to demonstrate that my movement definitely is different! See http://www.spel.nl/game/bumpercraft.html Update 4: I traced the time before rendering (EVENT.RENDER) and right after rendering (EVENT.ENTER_FRAME), the results: rendering took: 14 ms rendering took: 14 ms rendering took: 12 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 14 ms rendering took: 12 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 14 ms rendering took: 12 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 14 ms rendering took: 14 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 16 ms rendering took: 12 ms rendering took: 24 ms rendering took: 18 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 16 ms rendering took: 12 ms rendering took: 232 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 12 ms rendering took: 14 ms rendering took: 16 ms rendering took: 12 ms rendering took: 14 ms rendering took: 12 ms The range is 12-16 ms. During these differences, the shocking/warping/flickering movement was already going on. There is also 1 peak of 232ms, at this time there was a relatively big warp. This is however not the biggest problme, the biggest problem are the continuous small warps during normal movement. Does this give anyone a clue? Update 3: After testing, I know that the following factors are not causing my problem: Bitmap's quality - changed with photoshop to an uglier 8 colours optimized graphic, no improvement at all. Constant rotation of image while turning - disabled it, no improvement at all Browser rendering - tried to use the flash player standalone, no improvement at all I am 100% convinced that the problem lies in either my code or in my algorithm. Please, help me out. It has been almost two weeks (1 week that I asked this question on SO) now and I still have to get my golden answer. Update 1: see bottom for full flex project source and a live demo demonstrating my problem. I'm working on a 2d flash game. Player ships are created as an object: ships[id] = new GameShip(); When movement and rotation information is available, this is being directed to the corresponding ship: ships[id].setMovementMode(1); //move forward Now, within this GameShip object movement works using the "Event.ENTER_FRAME" event: addEventListener(Event.ENTER_FRAME, movementHandler); The following function is then being run: private final function movementHandler(event:Event):void { var newTimeStamp:uint = UtilLib.getTimeStamp(); //set current timeStamp var distance:Number = (newTimeStamp - movementTimeStamp) / 1000 * movementSpeed; //speed = x pixels forward every 1 second movementTimeStamp = newTimeStamp; //update old timeStamp var diagonalChange:Array = getDiagonalChange(movementAngle, distance); //the diagonal position update based on angle and distance charX += diagonalChange[0]; charY += diagonalChange[1]; if (shipContainer) { //when the container is ready to be worked with shipContainer.x = charX; shipContainer.y = charY; } } private final function getDiagonalChange(angle:Number, distance:Number):Array { var rAngle:Number = angle * Math.PI/180; //convert degrees to radian return [Math.sin(rAngle) * distance, (Math.cos(rAngle) * distance) * -1]; } When the object is no longer moving, the event listener will be removed. The same method is being used for rotation. Everything works almost perfect. I've set the project's target FPS to 100 and created a FPS counter. According to the FPS counter, the average FPS in firefox is around 100, while the top is 1000 and the bottom is 22. I think that the bottom and top FPSs are only happening during the initialization of the client (startup). The problem is that the ship appears to be almost perfectly smooth, while it should be just that without the "almost" part. It's almost as if the ship is "flickering" very very fast, you can't actually see it but it's hard to focus on the object while it's moving with your eyes. Also, every now and then, there seems to be a bit of a framerate spike, as if the client is skipping a couple of frames, you then see it quickly warp. It is very difficult to explain what the real problem is, but in general it's that the movement is not perfectly smooth. So, do you have any suggestions on how to make the movement or transition of objects perfectly smooth? Update 1: I re-created the client to demonstrate my problem. Please check it out. The client: http://feedpostal.com/test/MovementTest.html The Actionscript Project (full source): http://feedpostal.com/test/MovementTest.rar An example of a smooth flash game (not created by me): http://www.gamesforwork.com/games/swf/Mission%20Racing_august_10th_2009.swf It took me a pretty long time to recreate this client side version, I hope this will help with solving the problem. Please note: yes, it is actually pretty smooth. But it is definitely not smooth enough.

    Read the article

  • New CentOS/cPanel servers showing high load averages at idle

    - by Jax
    I have taken delivery of two identically specced CentOS/cPanel servers, showing the same behaviour of a resting load average of 1.30, 1.21, 1.16 and yet the CPU is sitting 100% idle. Hardware: Xeon(R) CPU E3-1270 4GB RAM Behavior:- top shows CPU 99.9% idle virtually no disk IO Some command output :- uname -a Linux server.myserver.com 2.6.18-308.4.1.el5PAE #1 SMP Tue Apr 17 17:47:38 EDT 2012 i686 i686 i386 GNU/Linux top top - 10:37:50 up 1:47, 1 user, load average: 1.28, 1.20, 1.17 Tasks: 199 total, 1 running, 198 sleeping, 0 stopped, 0 zombie Cpu(s): 0.0%us, 0.0%sy, 0.0%ni, 99.9%id, 0.1%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 4125104k total, 438764k used, 3686340k free, 25788k buffers Swap: 2096440k total, 0k used, 2096440k free, 291080k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1 root 15 0 2160 640 552 S 0.0 0.0 0:00.89 init 2 root RT -5 0 0 0 S 0.0 0.0 0:00.00 migration/0 3 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/0 4 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/0 5 root RT -5 0 0 0 S 0.0 0.0 0:00.00 migration/1 6 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/1 7 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/1 8 root RT -5 0 0 0 S 0.0 0.0 0:00.00 migration/2 9 root 35 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/2 10 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/2 11 root RT -5 0 0 0 S 0.0 0.0 0:00.00 migration/3 12 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/3 13 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/3 14 root RT -5 0 0 0 S 0.0 0.0 0:00.00 migration/4 15 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/4 16 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/4 17 root RT -5 0 0 0 S 0.0 0.0 0:00.00 migration/5 18 root 38 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/5 19 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/5 20 root RT -5 0 0 0 S 0.0 0.0 0:00.00 migration/6 21 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/6 22 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/6 23 root RT -5 0 0 0 S 0.0 0.0 0:00.00 migration/7 24 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/7 25 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/7 26 root 10 -5 0 0 0 S 0.0 0.0 0:06.42 events/0 27 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 events/1 28 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 events/2 29 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 events/3 30 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 events/4 31 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 events/5 32 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 events/6 33 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 events/7 34 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 khelper 35 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kthread 45 root 13 -5 0 0 0 S 0.0 0.0 0:00.00 kblockd/0 46 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kblockd/1 47 root 14 -5 0 0 0 S 0.0 0.0 0:00.00 kblockd/2 48 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kblockd/3 49 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kblockd/4 50 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kblockd/5 51 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kblockd/6 52 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kblockd/7 53 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kacpid 189 root 11 -5 0 0 0 S 0.0 0.0 0:00.00 cqueue/0 190 root 11 -5 0 0 0 S 0.0 0.0 0:00.00 cqueue/1 191 root 12 -5 0 0 0 S 0.0 0.0 0:00.00 cqueue/2 192 root 12 -5 0 0 0 S 0.0 0.0 0:00.00 cqueue/3 193 root 13 -5 0 0 0 S 0.0 0.0 0:00.00 cqueue/4 194 root 13 -5 0 0 0 S 0.0 0.0 0:00.00 cqueue/5 195 root 14 -5 0 0 0 S 0.0 0.0 0:00.00 cqueue/6 196 root 14 -5 0 0 0 S 0.0 0.0 0:00.00 cqueue/7 199 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 khubd ps axf PID TTY STAT TIME COMMAND 1 ? Ss 0:00 init [3] 2 ? S< 0:00 [migration/0] 3 ? SN 0:00 [ksoftirqd/0] 4 ? S< 0:00 [watchdog/0] 5 ? S< 0:00 [migration/1] 6 ? SN 0:00 [ksoftirqd/1] 7 ? S< 0:00 [watchdog/1] 8 ? S< 0:00 [migration/2] 9 ? SN 0:00 [ksoftirqd/2] 10 ? S< 0:00 [watchdog/2] 11 ? S< 0:00 [migration/3] 12 ? SN 0:00 [ksoftirqd/3] 13 ? S< 0:00 [watchdog/3] 14 ? S< 0:00 [migration/4] 15 ? SN 0:00 [ksoftirqd/4] 16 ? S< 0:00 [watchdog/4] 17 ? S< 0:00 [migration/5] 18 ? SN 0:00 [ksoftirqd/5] 19 ? S< 0:00 [watchdog/5] 20 ? S< 0:00 [migration/6] 21 ? SN 0:00 [ksoftirqd/6] 22 ? S< 0:00 [watchdog/6] 23 ? S< 0:00 [migration/7] 24 ? SN 0:00 [ksoftirqd/7] 25 ? S< 0:00 [watchdog/7] 26 ? S< 0:06 [events/0] 27 ? S< 0:00 [events/1] 28 ? S< 0:00 [events/2] 29 ? S< 0:00 [events/3] 30 ? S< 0:00 [events/4] 31 ? S< 0:00 [events/5] 32 ? S< 0:00 [events/6] 33 ? S< 0:00 [events/7] 34 ? S< 0:00 [khelper] 35 ? S< 0:00 [kthread] 45 ? S< 0:00 \_ [kblockd/0] 46 ? S< 0:00 \_ [kblockd/1] 47 ? S< 0:00 \_ [kblockd/2] 48 ? S< 0:00 \_ [kblockd/3] 49 ? S< 0:00 \_ [kblockd/4] 50 ? S< 0:00 \_ [kblockd/5] 51 ? S< 0:00 \_ [kblockd/6] 52 ? S< 0:00 \_ [kblockd/7] 53 ? S< 0:00 \_ [kacpid] 189 ? S< 0:00 \_ [cqueue/0] 190 ? S< 0:00 \_ [cqueue/1] 191 ? S< 0:00 \_ [cqueue/2] 192 ? S< 0:00 \_ [cqueue/3] 193 ? S< 0:00 \_ [cqueue/4] 194 ? S< 0:00 \_ [cqueue/5] 195 ? S< 0:00 \_ [cqueue/6] 196 ? S< 0:00 \_ [cqueue/7] 199 ? S< 0:00 \_ [khubd] 201 ? S< 0:00 \_ [kseriod] 301 ? S 0:00 \_ [khungtaskd] 302 ? S 0:00 \_ [pdflush] 303 ? S 0:00 \_ [pdflush] 304 ? S< 0:00 \_ [kswapd0] 305 ? S< 0:00 \_ [aio/0] 306 ? S< 0:00 \_ [aio/1] 307 ? S< 0:00 \_ [aio/2] 308 ? S< 0:00 \_ [aio/3] 309 ? S< 0:00 \_ [aio/4] 310 ? S< 0:00 \_ [aio/5] 311 ? S< 0:00 \_ [aio/6] 312 ? S< 0:00 \_ [aio/7] 472 ? S< 0:00 \_ [kpsmoused] 551 ? S< 0:00 \_ [ata/0] 552 ? S< 0:00 \_ [ata/1] 553 ? S< 0:00 \_ [ata/2] 554 ? S< 0:00 \_ [ata/3] 555 ? S< 0:00 \_ [ata/4] 556 ? S< 0:00 \_ [ata/5] 557 ? S< 0:00 \_ [ata/6] 558 ? S< 0:00 \_ [ata/7] 559 ? S< 0:00 \_ [ata_aux] 569 ? S< 0:00 \_ [scsi_eh_0] 570 ? S< 0:00 \_ [scsi_eh_1] 571 ? S< 0:00 \_ [scsi_eh_2] 572 ? S< 0:00 \_ [scsi_eh_3] 573 ? S< 0:00 \_ [scsi_eh_4] 574 ? S< 0:00 \_ [scsi_eh_5] 593 ? S< 0:00 \_ [kstriped] 630 ? S< 0:00 \_ [kjournald] 655 ? S< 0:00 \_ [kauditd] 1860 ? S< 0:00 \_ [kmpathd/0] 1861 ? S< 0:00 \_ [kmpathd/1] 1862 ? S< 0:00 \_ [kmpathd/2] 1863 ? S< 0:00 \_ [kmpathd/3] 1864 ? S< 0:00 \_ [kmpathd/4] 1865 ? S< 0:00 \_ [kmpathd/5] 1866 ? S< 0:00 \_ [kmpathd/6] 1867 ? S< 0:00 \_ [kmpathd/7] 1868 ? S< 0:00 \_ [kmpath_handlerd] 1902 ? S< 0:00 \_ [kjournald] 1904 ? S< 0:00 \_ [kjournald] 1906 ? S< 0:00 \_ [kjournald] 1908 ? S< 0:00 \_ [kjournald] 1910 ? S< 0:00 \_ [kjournald] 2184 ? S< 0:00 \_ [iscsi_eh] 2288 ? S< 0:00 \_ [cnic_wq] 2298 ? S< 0:00 \_ [bnx2i_thread/0] 2299 ? S< 0:00 \_ [bnx2i_thread/1] 2300 ? S< 0:00 \_ [bnx2i_thread/2] 2301 ? S< 0:00 \_ [bnx2i_thread/3] 2302 ? S< 0:00 \_ [bnx2i_thread/4] 2303 ? S< 0:00 \_ [bnx2i_thread/5] 2304 ? S< 0:00 \_ [bnx2i_thread/6] 2305 ? S< 0:00 \_ [bnx2i_thread/7] 2330 ? S< 0:00 \_ [ib_addr] 2359 ? S< 0:00 \_ [ib_mcast] 2360 ? S< 0:00 \_ [ib_inform] 2361 ? S< 0:00 \_ [local_sa] 2371 ? S< 0:00 \_ [iw_cm_wq] 2381 ? S< 0:00 \_ [ib_cm/0] 2382 ? S< 0:00 \_ [ib_cm/1] 2383 ? S< 0:00 \_ [ib_cm/2] 2384 ? S< 0:00 \_ [ib_cm/3] 2385 ? S< 0:00 \_ [ib_cm/4] 2386 ? S< 0:00 \_ [ib_cm/5] 2387 ? S< 0:00 \_ [ib_cm/6] 2388 ? S< 0:00 \_ [ib_cm/7] 2398 ? S< 0:00 \_ [rdma_cm] 2684 ? S< 0:00 \_ [bond0] 2882 ? S< 0:00 \_ [bond1] 3195 ? S< 0:00 \_ [kondemand/0] 3197 ? S< 0:00 \_ [kondemand/1] 3198 ? S< 0:00 \_ [kondemand/2] 3199 ? S< 0:00 \_ [kondemand/3] 3200 ? S< 0:00 \_ [kondemand/4] 3201 ? S< 0:00 \_ [kondemand/5] 3202 ? S< 0:00 \_ [kondemand/6] 3203 ? S< 0:00 \_ [kondemand/7] 688 ? S<s 0:00 /sbin/udevd -d 2425 ? S<Lsl 0:00 iscsiuio 2432 ? Ss 0:00 iscsid 2434 ? S<Ls 0:00 iscsid 3061 ? S<sl 0:00 auditd 3063 ? S<sl 0:00 \_ /sbin/audispd 3121 ? Ss 0:00 syslogd -m 0 3124 ? Ss 0:00 klogd -x 3220 ? Ss 0:00 irqbalance 3278 ? Ss 0:00 dbus-daemon --system 3324 ? Ss 0:00 /usr/sbin/acpid 3337 ? Ss 0:00 hald 3338 ? S 0:00 \_ hald-runner 3345 ? S 0:00 \_ hald-addon-acpi: listening on acpid socket /var/run/acpid.socket 3349 ? S 0:00 \_ hald-addon-keyboard: listening on /dev/input/event1 3360 ? S 0:00 \_ hald-addon-storage: polling /dev/sr0 3413 ? Ssl 0:00 automount 3435 ? Ssl 0:00 /usr/sbin/named -u named 3466 ? Ss 0:00 /usr/sbin/sshd 4072 ? Ss 0:00 \_ sshd: root@pts/0 4078 pts/0 Ss 0:00 \_ -bash 5436 pts/0 R+ 0:00 \_ ps axf 3484 ? Ss 0:00 xinetd -stayalive -pidfile /var/run/xinetd.pid 3500 ? SLs 0:00 ntpd -u ntp:ntp -p /var/run/ntpd.pid -g 3514 ? S 0:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --pid-file=/var/lib/mysql/server.myserver.com.pid 3575 ? Sl 0:00 \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --log-error=/var/lib/mysql/server.myserver.com.err --pid-fil 3687 ? Ss 0:00 /usr/sbin/exim -bd -q1h 3709 ? Ss 0:00 /usr/sbin/dovecot 3710 ? S 0:00 \_ dovecot-auth 3725 ? S 0:00 \_ pop3-login 3726 ? S 0:00 \_ pop3-login 3727 ? S 0:00 \_ imap-login 3728 ? S 0:00 \_ imap-login 3729 ? Ss 0:00 /usr/local/apache/bin/httpd -k start -DSSL 4326 ? S 0:00 \_ /usr/bin/perl /usr/local/cpanel/bin/leechprotect 4332 ? S 0:00 \_ /usr/local/apache/bin/httpd -k start -DSSL 4333 ? S 0:00 \_ /usr/local/apache/bin/httpd -k start -DSSL 4334 ? S 0:00 \_ /usr/local/apache/bin/httpd -k start -DSSL 4335 ? S 0:00 \_ /usr/local/apache/bin/httpd -k start -DSSL 4336 ? S 0:00 \_ /usr/local/apache/bin/httpd -k start -DSSL 4337 ? S 0:00 \_ /usr/local/apache/bin/httpd -k start -DSSL 4382 ? S 0:00 \_ /usr/local/apache/bin/httpd -k start -DSSL 4383 ? S 0:00 \_ /usr/local/apache/bin/httpd -k start -DSSL 4384 ? S 0:00 \_ /usr/local/apache/bin/httpd -k start -DSSL 5389 ? S 0:00 \_ /usr/local/apache/bin/httpd -k start -DSSL 5390 ? S 0:00 \_ /usr/local/apache/bin/httpd -k start -DSSL 3741 ? Ss 0:00 pure-ftpd (SERVER) 3746 ? S 0:00 /usr/sbin/pure-authd -s /var/run/ftpd.sock -r /usr/sbin/pureauth 3759 ? Ss 0:00 crond 3772 ? Ss 0:00 /usr/sbin/atd 3909 ? S 0:00 cpsrvd (SSL) - waiting for connections 5435 ? Z 0:00 \_ [cpsrvd-ssl] <defunct> 3931 ? S 0:00 queueprocd - wait to process a task 3948 ? S 0:00 tailwatchd 3954 ? SN 0:00 cpanellogd - sleeping for logs 4003 ? Ss 0:00 ./nimbus /opt/nimsoft 4016 ? S 0:00 \_ nimbus(controller) 4053 ? Sl 0:00 \_ nimbus(spooler) 4066 ? S 0:00 \_ nimbus(hdb) 4069 ? S 0:00 \_ nimbus(cdm) 4070 ? S 0:00 \_ nimbus(processes) 4023 ? S 0:00 /usr/sbin/smartd -q never 4027 tty1 Ss+ 0:00 /sbin/mingetty tty1 4028 tty2 Ss+ 0:00 /sbin/mingetty tty2 4029 tty3 Ss+ 0:00 /sbin/mingetty tty3 4030 tty4 Ss+ 0:00 /sbin/mingetty tty4 4031 tty5 Ss+ 0:00 /sbin/mingetty tty5 4033 tty6 Ss+ 0:00 /sbin/mingetty tty6 4035 ttyS1 Ss+ 0:00 /sbin/agetty -h -L ttyS1 19200 vt100 vmstat 10 6 procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------ r b swpd free buff cache si so bi bo in cs us sy id wa st 0 0 0 3718136 25684 257424 0 0 8 3 127 189 0 0 100 0 0 0 0 0 3718136 25700 257420 0 0 0 7 1013 1500 0 0 100 0 0 0 0 0 3718136 25700 257424 0 0 0 1 1013 1551 0 0 100 0 0 0 0 0 3718136 25700 257424 0 0 0 0 1012 1469 0 0 100 0 0 1 0 0 3712680 25716 257424 0 0 0 2 1013 1542 0 0 100 0 0 0 0 0 3718376 25740 257424 0 0 0 46 1017 1534 0 0 100 0 0 Can anyone advise me as to what is the cause of and how I may resolve this behaviour? A kernel/driver conflict perhaps? I don't see any processes in R or D state that might inflate the load averages artificially, I realise it may be considered low in an 8 thread system but its higher at idle than any normal behaviour I've previously come across. Thanks in advance for your time. Edit: iotop Total DISK READ: 0.00 B/s | Total DISK WRITE: 0.00 B/s TID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND 26 be/3 root 0.00 B/s 0.00 B/s 0.00 % 0.29 % [events/0] 3205 be/3 root 0.00 B/s 0.00 B/s 0.00 % 0.10 % [kondemand/2] 3208 be/3 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [kondemand/5] 3209 be/3 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [kondemand/6] 3207 be/3 root 0.00 B/s 0.00 B/s 0.10 % 0.00 % [kondemand/4] 3210 be/3 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [kondemand/7] 3227 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % irqbalance 3288 be/3 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [rpciod/1] 3287 be/3 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [rpciod/0] 3206 be/3 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [kondemand/3] 3069 be/3 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % auditd 3070 be/2 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % audispd 655 be/3 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [kauditd] 3619 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % automount 3 be/7 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [ksoftirqd/0] 3068 be/3 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % auditd 29 be/3 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [events/3] 4 rt/3 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [watchdog/0] 7 rt/3 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [watchdog/1] 10 rt/3 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [watchdog/2] 13 rt/3 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [watchdog/3] 16 rt/3 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [watchdog/4] 19 rt/3 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [watchdog/5] 22 rt/3 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [watchdog/6] 25 rt/3 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [watchdog/7] 27 be/3 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [events/1] 28 be/3 root 0.00 B/s 0.00 B/s 0.29 % 0.00 % [events/2] 30 be/3 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [events/4] 31 be/3 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [events/5] 32 be/3 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [events/6] 33 be/3 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [events/7] 34 be/3 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [khelper] 35 be/3 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [kthread] 45 be/3 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [kblockd/0]

    Read the article

  • "Class ref in pre-verified class resolved to unexpected implementation" when running android tests i

    - by Mike
    I have a module that builds an app called MyApp. I have another that builds some testcases for that app, called MyAppTests. They both build their own APKs, and they both work fine from within my IDE. I'd like to build them using ant so that I can take advantage of continuous integration. Building the app module works fine. I'm having difficulty getting the Test module to compile and run. Using Christopher's tip from a previous question, I used android create test-project -p MyAppTests -m ../MyApp -n MyAppTests to create the necessary build files to build and run my test project. This seems to work great (once I remove an unnecessary test case that it constructed for me and revert my AndroidManifest.xml to the one I was using before it got replaced by android create), but I have two problems. The first problem: The project doesn't compile because it's missing libraries. $ ant run-tests Buildfile: build.xml [setup] Project Target: Google APIs [setup] Vendor: Google Inc. [setup] Platform Version: 1.6 [setup] API level: 4 [setup] WARNING: No minSdkVersion value set. Application will install on all Android versions. -install-tested-project: [setup] Project Target: Google APIs [setup] Vendor: Google Inc. [setup] Platform Version: 1.6 [setup] API level: 4 [setup] WARNING: No minSdkVersion value set. Application will install on all Android versions. -compile-tested-if-test: -dirs: [echo] Creating output directories if needed... -resource-src: [echo] Generating R.java / Manifest.java from the resources... -aidl: [echo] Compiling aidl files into Java classes... compile: [javac] Compiling 1 source file to /Users/mike/Projects/myapp/android/MyApp/bin/classes -dex: [echo] Converting compiled files and external libraries into /Users/mike/Projects/myapp/android/MyApp/bin/classes.dex... [echo] -package-resources: [echo] Packaging resources [aaptexec] Creating full resource package... -package-debug-sign: [apkbuilder] Creating MyApp-debug-unaligned.apk and signing it with a debug key... [apkbuilder] Using keystore: /Users/mike/.android/debug.keystore debug: [echo] Running zip align on final apk... [echo] Debug Package: /Users/mike/Projects/myapp/android/MyApp/bin/MyApp-debug.apk install: [echo] Installing /Users/mike/Projects/myapp/android/MyApp/bin/MyApp-debug.apk onto default emulator or device... [exec] 1567 KB/s (288354 bytes in 0.179s) [exec] pkg: /data/local/tmp/MyApp-debug.apk [exec] Success -compile-tested-if-test: -dirs: [echo] Creating output directories if needed... [mkdir] Created dir: /Users/mike/Projects/myapp/android/MyAppTests/gen [mkdir] Created dir: /Users/mike/Projects/myapp/android/MyAppTests/bin [mkdir] Created dir: /Users/mike/Projects/myapp/android/MyAppTests/bin/classes -resource-src: [echo] Generating R.java / Manifest.java from the resources... -aidl: [echo] Compiling aidl files into Java classes... compile: [javac] Compiling 5 source files to /Users/mike/Projects/myapp/android/MyAppTests/bin/classes [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/GsonTest.java:4: package roboguice.test does not exist [javac] import roboguice.test.RoboUnitTestCase; [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/GsonTest.java:8: package com.google.gson does not exist [javac] import com.google.gson.JsonElement; [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/GsonTest.java:9: package com.google.gson does not exist [javac] import com.google.gson.JsonParser; [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/GsonTest.java:11: cannot find symbol [javac] symbol: class RoboUnitTestCase [javac] public class GsonTest extends RoboUnitTestCase<MyApplication> { [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/HttpTest.java:6: package roboguice.test does not exist [javac] import roboguice.test.RoboUnitTestCase; [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/HttpTest.java:7: package roboguice.util does not exist [javac] import roboguice.util.RoboLooperThread; [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/HttpTest.java:11: package com.google.gson does not exist [javac] import com.google.gson.JsonObject; [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/HttpTest.java:15: cannot find symbol [javac] symbol: class RoboUnitTestCase [javac] public class HttpTest extends RoboUnitTestCase<MyApplication> { [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/LinksTest.java:4: package roboguice.test does not exist [javac] import roboguice.test.RoboUnitTestCase; [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/LinksTest.java:12: cannot find symbol [javac] symbol: class RoboUnitTestCase [javac] public class LinksTest extends RoboUnitTestCase<MyApplication> { [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/SafeAsyncTest.java:4: package roboguice.test does not exist [javac] import roboguice.test.RoboUnitTestCase; [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/SafeAsyncTest.java:5: package roboguice.util does not exist [javac] import roboguice.util.RoboAsyncTask; [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/SafeAsyncTest.java:6: package roboguice.util does not exist [javac] import roboguice.util.RoboLooperThread; [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/SafeAsyncTest.java:12: cannot find symbol [javac] symbol: class RoboUnitTestCase [javac] public class SafeAsyncTest extends RoboUnitTestCase<MyApplication> { [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyApp/bin/classes/com/myapp/activity/Stories.class: warning: Cannot find annotation method 'value()' in type 'roboguice.inject.InjectResource': class file for roboguice.inject.InjectResource not found [javac] /Users/mike/Projects/myapp/android/MyApp/bin/classes/com/myapp/activity/Stories.class: warning: Cannot find annotation method 'value()' in type 'roboguice.inject.InjectResource' [javac] /Users/mike/Projects/myapp/android/MyApp/bin/classes/com/myapp/activity/Stories.class: warning: Cannot find annotation method 'value()' in type 'roboguice.inject.InjectView': class file for roboguice.inject.InjectView not found [javac] /Users/mike/Projects/myapp/android/MyApp/bin/classes/com/myapp/activity/Stories.class: warning: Cannot find annotation method 'value()' in type 'roboguice.inject.InjectView' [javac] /Users/mike/Projects/myapp/android/MyApp/bin/classes/com/myapp/activity/Stories.class: warning: Cannot find annotation method 'value()' in type 'roboguice.inject.InjectView' [javac] /Users/mike/Projects/myapp/android/MyApp/bin/classes/com/myapp/activity/Stories.class: warning: Cannot find annotation method 'value()' in type 'roboguice.inject.InjectView' [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/GsonTest.java:15: cannot find symbol [javac] symbol : class JsonParser [javac] location: class com.myapp.test.GsonTest [javac] final JsonParser parser = new JsonParser(); [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/GsonTest.java:15: cannot find symbol [javac] symbol : class JsonParser [javac] location: class com.myapp.test.GsonTest [javac] final JsonParser parser = new JsonParser(); [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/GsonTest.java:18: cannot find symbol [javac] symbol : class JsonElement [javac] location: class com.myapp.test.GsonTest [javac] final JsonElement e = parser.parse(s); [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/GsonTest.java:20: cannot find symbol [javac] symbol : class JsonElement [javac] location: class com.myapp.test.GsonTest [javac] final JsonElement e2 = parser.parse(s2); [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/HttpTest.java:19: cannot find symbol [javac] symbol : method getInstrumentation() [javac] location: class com.myapp.test.HttpTest [javac] assertEquals("MyApp", getInstrumentation().getTargetContext().getResources().getString(com.myapp.R.string.app_name)); [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/HttpTest.java:62: cannot find symbol [javac] symbol : class RoboLooperThread [javac] location: class com.myapp.test.HttpTest [javac] new RoboLooperThread() { [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/HttpTest.java:82: cannot find symbol [javac] symbol : method assertTrue(java.lang.String,boolean) [javac] location: class com.myapp.test.HttpTest [javac] assertTrue(result[0], result[0].contains("Search")); [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/HttpTest.java:87: cannot find symbol [javac] symbol : class JsonObject [javac] location: class com.myapp.test.HttpTest [javac] final JsonObject[] result = {null}; [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/HttpTest.java:90: cannot find symbol [javac] symbol : class RoboLooperThread [javac] location: class com.myapp.test.HttpTest [javac] new RoboLooperThread() { [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/HttpTest.java:117: cannot find symbol [javac] symbol : class JsonObject [javac] location: class com.myapp.test.HttpTest [javac] final JsonObject[] result = {null}; [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/HttpTest.java:120: cannot find symbol [javac] symbol : class RoboLooperThread [javac] location: class com.myapp.test.HttpTest [javac] new RoboLooperThread() { [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/LinksTest.java:27: cannot find symbol [javac] symbol : method assertTrue(boolean) [javac] location: class com.myapp.test.LinksTest [javac] assertTrue(m.matches()); [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/LinksTest.java:28: cannot find symbol [javac] symbol : method assertEquals(java.lang.String,java.lang.String) [javac] location: class com.myapp.test.LinksTest [javac] assertEquals( map.get(url), m.group(1) ); [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/SafeAsyncTest.java:19: cannot find symbol [javac] symbol : method getInstrumentation() [javac] location: class com.myapp.test.SafeAsyncTest [javac] assertEquals("MyApp", getInstrumentation().getTargetContext().getString(com.myapp.R.string.app_name)); [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/SafeAsyncTest.java:27: cannot find symbol [javac] symbol : class RoboLooperThread [javac] location: class com.myapp.test.SafeAsyncTest [javac] new RoboLooperThread() { [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/SafeAsyncTest.java:65: cannot find symbol [javac] symbol : method assertEquals(com.myapp.test.SafeAsyncTest.State,com.myapp.test.SafeAsyncTest.State) [javac] location: class com.myapp.test.SafeAsyncTest [javac] assertEquals(State.TEST_SUCCESS,state[0]); [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/SafeAsyncTest.java:74: cannot find symbol [javac] symbol : class RoboLooperThread [javac] location: class com.myapp.test.SafeAsyncTest [javac] new RoboLooperThread() { [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/SafeAsyncTest.java:105: cannot find symbol [javac] symbol : method assertEquals(com.myapp.test.SafeAsyncTest.State,com.myapp.test.SafeAsyncTest.State) [javac] location: class com.myapp.test.SafeAsyncTest [javac] assertEquals(State.TEST_SUCCESS,state[0]); [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/SafeAsyncTest.java:113: cannot find symbol [javac] symbol : class RoboLooperThread [javac] location: class com.myapp.test.SafeAsyncTest [javac] new RoboLooperThread() { [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/SafeAsyncTest.java:144: cannot find symbol [javac] symbol : method assertEquals(com.myapp.test.SafeAsyncTest.State,com.myapp.test.SafeAsyncTest.State) [javac] location: class com.myapp.test.SafeAsyncTest [javac] assertEquals(State.TEST_SUCCESS,state[0]); [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/SafeAsyncTest.java:154: cannot find symbol [javac] symbol : class RoboLooperThread [javac] location: class com.myapp.test.SafeAsyncTest [javac] new RoboLooperThread() { [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/SafeAsyncTest.java:187: cannot find symbol [javac] symbol : method assertEquals(com.myapp.test.SafeAsyncTest.State,com.myapp.test.SafeAsyncTest.State) [javac] location: class com.myapp.test.SafeAsyncTest [javac] assertEquals(State.TEST_SUCCESS,state[0]); [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/StoriesTest.java:11: cannot access roboguice.activity.GuiceListActivity [javac] class file for roboguice.activity.GuiceListActivity not found [javac] public class StoriesTest extends ActivityUnitTestCase<Stories> { [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/StoriesTest.java:21: cannot access roboguice.application.GuiceApplication [javac] class file for roboguice.application.GuiceApplication not found [javac] setApplication( new MyApplication( getInstrumentation().getTargetContext() ) ); [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/StoriesTest.java:22: incompatible types [javac] found : com.myapp.activity.Stories [javac] required: android.app.Activity [javac] final Activity activity = startActivity(intent, null, null); [javac] ^ [javac] 39 errors [javac] 6 warnings BUILD FAILED /opt/local/android-sdk-mac/platforms/android-1.6/templates/android_rules.xml:248: Compile failed; see the compiler error output for details. Total time: 24 seconds That's not a hard problem to solve. I'm not sure it's the right thing to do, but I copied the missing libraries (roboguice and gson) from the MyApp/libs directory to the MyAppTests/libs directory and everything seems to compile fine. But that leads to the second problem, which I'm currently stuck on. The tests compile fine but they won't run: $ cp ../MyApp/libs/gson-r538.jar libs/ $ cp ../MyApp/libs/roboguice-1.1-SNAPSHOT.jar libs/ 0 10:23 /Users/mike/Projects/myapp/android/MyAppTests $ ant run-testsBuildfile: build.xml [setup] Project Target: Google APIs [setup] Vendor: Google Inc. [setup] Platform Version: 1.6 [setup] API level: 4 [setup] WARNING: No minSdkVersion value set. Application will install on all Android versions. -install-tested-project: [setup] Project Target: Google APIs [setup] Vendor: Google Inc. [setup] Platform Version: 1.6 [setup] API level: 4 [setup] WARNING: No minSdkVersion value set. Application will install on all Android versions. -compile-tested-if-test: -dirs: [echo] Creating output directories if needed... -resource-src: [echo] Generating R.java / Manifest.java from the resources... -aidl: [echo] Compiling aidl files into Java classes... compile: [javac] Compiling 1 source file to /Users/mike/Projects/myapp/android/MyApp/bin/classes -dex: [echo] Converting compiled files and external libraries into /Users/mike/Projects/myapp/android/MyApp/bin/classes.dex... [echo] -package-resources: [echo] Packaging resources [aaptexec] Creating full resource package... -package-debug-sign: [apkbuilder] Creating MyApp-debug-unaligned.apk and signing it with a debug key... [apkbuilder] Using keystore: /Users/mike/.android/debug.keystore debug: [echo] Running zip align on final apk... [echo] Debug Package: /Users/mike/Projects/myapp/android/MyApp/bin/MyApp-debug.apk install: [echo] Installing /Users/mike/Projects/myapp/android/MyApp/bin/MyApp-debug.apk onto default emulator or device... [exec] 1396 KB/s (288354 bytes in 0.201s) [exec] pkg: /data/local/tmp/MyApp-debug.apk [exec] Success -compile-tested-if-test: -dirs: [echo] Creating output directories if needed... -resource-src: [echo] Generating R.java / Manifest.java from the resources... -aidl: [echo] Compiling aidl files into Java classes... compile: [javac] Compiling 5 source files to /Users/mike/Projects/myapp/android/MyAppTests/bin/classes [javac] Note: /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/SafeAsyncTest.java uses unchecked or unsafe operations. [javac] Note: Recompile with -Xlint:unchecked for details. -dex: [echo] Converting compiled files and external libraries into /Users/mike/Projects/myapp/android/MyAppTests/bin/classes.dex... [echo] -package-resources: [echo] Packaging resources [aaptexec] Creating full resource package... -package-debug-sign: [apkbuilder] Creating MyAppTests-debug-unaligned.apk and signing it with a debug key... [apkbuilder] Using keystore: /Users/mike/.android/debug.keystore debug: [echo] Running zip align on final apk... [echo] Debug Package: /Users/mike/Projects/myapp/android/MyAppTests/bin/MyAppTests-debug.apk install: [echo] Installing /Users/mike/Projects/myapp/android/MyAppTests/bin/MyAppTests-debug.apk onto default emulator or device... [exec] 1227 KB/s (94595 bytes in 0.075s) [exec] pkg: /data/local/tmp/MyAppTests-debug.apk [exec] Success run-tests: [echo] Running tests ... [exec] [exec] android.test.suitebuilder.TestSuiteBuilder$FailedToCreateTests:INSTRUMENTATION_RESULT: shortMsg=Class ref in pre-verified class resolved to unexpected implementation [exec] INSTRUMENTATION_RESULT: longMsg=java.lang.IllegalAccessError: Class ref in pre-verified class resolved to unexpected implementation [exec] INSTRUMENTATION_CODE: 0 BUILD SUCCESSFUL Total time: 38 seconds Any idea what's causing the "Class ref in pre-verified class resolved to unexpected implementation" error?

    Read the article

  • Can't build and run an android test project created using "ant create test-project" when tested proj

    - by Mike
    I have a module that builds an app called MyApp. I have another that builds some testcases for that app, called MyAppTests. They both build their own APKs, and they both work fine from within my IDE. I'd like to build them using ant so that I can take advantage of continuous integration. Building the app module works fine. I'm having difficulty getting the Test module to compile and run. Using Christopher's tip from a previous question, I used android create test-project -p MyAppTests -m ../MyApp -n MyAppTests to create the necessary build files to build and run my test project. This seems to work great (once I remove an unnecessary test case that it constructed for me and revert my AndroidManifest.xml to the one I was using before it got replaced by android create), but I have two problems. The first problem: The project doesn't compile because it's missing libraries. $ ant run-tests Buildfile: build.xml [setup] Project Target: Google APIs [setup] Vendor: Google Inc. [setup] Platform Version: 1.6 [setup] API level: 4 [setup] WARNING: No minSdkVersion value set. Application will install on all Android versions. -install-tested-project: [setup] Project Target: Google APIs [setup] Vendor: Google Inc. [setup] Platform Version: 1.6 [setup] API level: 4 [setup] WARNING: No minSdkVersion value set. Application will install on all Android versions. -compile-tested-if-test: -dirs: [echo] Creating output directories if needed... -resource-src: [echo] Generating R.java / Manifest.java from the resources... -aidl: [echo] Compiling aidl files into Java classes... compile: [javac] Compiling 1 source file to /Users/mike/Projects/myapp/android/MyApp/bin/classes -dex: [echo] Converting compiled files and external libraries into /Users/mike/Projects/myapp/android/MyApp/bin/classes.dex... [echo] -package-resources: [echo] Packaging resources [aaptexec] Creating full resource package... -package-debug-sign: [apkbuilder] Creating MyApp-debug-unaligned.apk and signing it with a debug key... [apkbuilder] Using keystore: /Users/mike/.android/debug.keystore debug: [echo] Running zip align on final apk... [echo] Debug Package: /Users/mike/Projects/myapp/android/MyApp/bin/MyApp-debug.apk install: [echo] Installing /Users/mike/Projects/myapp/android/MyApp/bin/MyApp-debug.apk onto default emulator or device... [exec] 1567 KB/s (288354 bytes in 0.179s) [exec] pkg: /data/local/tmp/MyApp-debug.apk [exec] Success -compile-tested-if-test: -dirs: [echo] Creating output directories if needed... [mkdir] Created dir: /Users/mike/Projects/myapp/android/MyAppTests/gen [mkdir] Created dir: /Users/mike/Projects/myapp/android/MyAppTests/bin [mkdir] Created dir: /Users/mike/Projects/myapp/android/MyAppTests/bin/classes -resource-src: [echo] Generating R.java / Manifest.java from the resources... -aidl: [echo] Compiling aidl files into Java classes... compile: [javac] Compiling 5 source files to /Users/mike/Projects/myapp/android/MyAppTests/bin/classes [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/GsonTest.java:4: package roboguice.test does not exist [javac] import roboguice.test.RoboUnitTestCase; [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/GsonTest.java:8: package com.google.gson does not exist [javac] import com.google.gson.JsonElement; [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/GsonTest.java:9: package com.google.gson does not exist [javac] import com.google.gson.JsonParser; [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/GsonTest.java:11: cannot find symbol [javac] symbol: class RoboUnitTestCase [javac] public class GsonTest extends RoboUnitTestCase<MyApplication> { [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/HttpTest.java:6: package roboguice.test does not exist [javac] import roboguice.test.RoboUnitTestCase; [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/HttpTest.java:7: package roboguice.util does not exist [javac] import roboguice.util.RoboLooperThread; [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/HttpTest.java:11: package com.google.gson does not exist [javac] import com.google.gson.JsonObject; [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/HttpTest.java:15: cannot find symbol [javac] symbol: class RoboUnitTestCase [javac] public class HttpTest extends RoboUnitTestCase<MyApplication> { [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/LinksTest.java:4: package roboguice.test does not exist [javac] import roboguice.test.RoboUnitTestCase; [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/LinksTest.java:12: cannot find symbol [javac] symbol: class RoboUnitTestCase [javac] public class LinksTest extends RoboUnitTestCase<MyApplication> { [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/SafeAsyncTest.java:4: package roboguice.test does not exist [javac] import roboguice.test.RoboUnitTestCase; [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/SafeAsyncTest.java:5: package roboguice.util does not exist [javac] import roboguice.util.RoboAsyncTask; [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/SafeAsyncTest.java:6: package roboguice.util does not exist [javac] import roboguice.util.RoboLooperThread; [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/SafeAsyncTest.java:12: cannot find symbol [javac] symbol: class RoboUnitTestCase [javac] public class SafeAsyncTest extends RoboUnitTestCase<MyApplication> { [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyApp/bin/classes/com/myapp/activity/Stories.class: warning: Cannot find annotation method 'value()' in type 'roboguice.inject.InjectResource': class file for roboguice.inject.InjectResource not found [javac] /Users/mike/Projects/myapp/android/MyApp/bin/classes/com/myapp/activity/Stories.class: warning: Cannot find annotation method 'value()' in type 'roboguice.inject.InjectResource' [javac] /Users/mike/Projects/myapp/android/MyApp/bin/classes/com/myapp/activity/Stories.class: warning: Cannot find annotation method 'value()' in type 'roboguice.inject.InjectView': class file for roboguice.inject.InjectView not found [javac] /Users/mike/Projects/myapp/android/MyApp/bin/classes/com/myapp/activity/Stories.class: warning: Cannot find annotation method 'value()' in type 'roboguice.inject.InjectView' [javac] /Users/mike/Projects/myapp/android/MyApp/bin/classes/com/myapp/activity/Stories.class: warning: Cannot find annotation method 'value()' in type 'roboguice.inject.InjectView' [javac] /Users/mike/Projects/myapp/android/MyApp/bin/classes/com/myapp/activity/Stories.class: warning: Cannot find annotation method 'value()' in type 'roboguice.inject.InjectView' [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/GsonTest.java:15: cannot find symbol [javac] symbol : class JsonParser [javac] location: class com.myapp.test.GsonTest [javac] final JsonParser parser = new JsonParser(); [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/GsonTest.java:15: cannot find symbol [javac] symbol : class JsonParser [javac] location: class com.myapp.test.GsonTest [javac] final JsonParser parser = new JsonParser(); [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/GsonTest.java:18: cannot find symbol [javac] symbol : class JsonElement [javac] location: class com.myapp.test.GsonTest [javac] final JsonElement e = parser.parse(s); [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/GsonTest.java:20: cannot find symbol [javac] symbol : class JsonElement [javac] location: class com.myapp.test.GsonTest [javac] final JsonElement e2 = parser.parse(s2); [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/HttpTest.java:19: cannot find symbol [javac] symbol : method getInstrumentation() [javac] location: class com.myapp.test.HttpTest [javac] assertEquals("MyApp", getInstrumentation().getTargetContext().getResources().getString(com.myapp.R.string.app_name)); [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/HttpTest.java:62: cannot find symbol [javac] symbol : class RoboLooperThread [javac] location: class com.myapp.test.HttpTest [javac] new RoboLooperThread() { [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/HttpTest.java:82: cannot find symbol [javac] symbol : method assertTrue(java.lang.String,boolean) [javac] location: class com.myapp.test.HttpTest [javac] assertTrue(result[0], result[0].contains("Search")); [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/HttpTest.java:87: cannot find symbol [javac] symbol : class JsonObject [javac] location: class com.myapp.test.HttpTest [javac] final JsonObject[] result = {null}; [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/HttpTest.java:90: cannot find symbol [javac] symbol : class RoboLooperThread [javac] location: class com.myapp.test.HttpTest [javac] new RoboLooperThread() { [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/HttpTest.java:117: cannot find symbol [javac] symbol : class JsonObject [javac] location: class com.myapp.test.HttpTest [javac] final JsonObject[] result = {null}; [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/HttpTest.java:120: cannot find symbol [javac] symbol : class RoboLooperThread [javac] location: class com.myapp.test.HttpTest [javac] new RoboLooperThread() { [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/LinksTest.java:27: cannot find symbol [javac] symbol : method assertTrue(boolean) [javac] location: class com.myapp.test.LinksTest [javac] assertTrue(m.matches()); [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/LinksTest.java:28: cannot find symbol [javac] symbol : method assertEquals(java.lang.String,java.lang.String) [javac] location: class com.myapp.test.LinksTest [javac] assertEquals( map.get(url), m.group(1) ); [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/SafeAsyncTest.java:19: cannot find symbol [javac] symbol : method getInstrumentation() [javac] location: class com.myapp.test.SafeAsyncTest [javac] assertEquals("MyApp", getInstrumentation().getTargetContext().getString(com.myapp.R.string.app_name)); [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/SafeAsyncTest.java:27: cannot find symbol [javac] symbol : class RoboLooperThread [javac] location: class com.myapp.test.SafeAsyncTest [javac] new RoboLooperThread() { [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/SafeAsyncTest.java:65: cannot find symbol [javac] symbol : method assertEquals(com.myapp.test.SafeAsyncTest.State,com.myapp.test.SafeAsyncTest.State) [javac] location: class com.myapp.test.SafeAsyncTest [javac] assertEquals(State.TEST_SUCCESS,state[0]); [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/SafeAsyncTest.java:74: cannot find symbol [javac] symbol : class RoboLooperThread [javac] location: class com.myapp.test.SafeAsyncTest [javac] new RoboLooperThread() { [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/SafeAsyncTest.java:105: cannot find symbol [javac] symbol : method assertEquals(com.myapp.test.SafeAsyncTest.State,com.myapp.test.SafeAsyncTest.State) [javac] location: class com.myapp.test.SafeAsyncTest [javac] assertEquals(State.TEST_SUCCESS,state[0]); [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/SafeAsyncTest.java:113: cannot find symbol [javac] symbol : class RoboLooperThread [javac] location: class com.myapp.test.SafeAsyncTest [javac] new RoboLooperThread() { [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/SafeAsyncTest.java:144: cannot find symbol [javac] symbol : method assertEquals(com.myapp.test.SafeAsyncTest.State,com.myapp.test.SafeAsyncTest.State) [javac] location: class com.myapp.test.SafeAsyncTest [javac] assertEquals(State.TEST_SUCCESS,state[0]); [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/SafeAsyncTest.java:154: cannot find symbol [javac] symbol : class RoboLooperThread [javac] location: class com.myapp.test.SafeAsyncTest [javac] new RoboLooperThread() { [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/SafeAsyncTest.java:187: cannot find symbol [javac] symbol : method assertEquals(com.myapp.test.SafeAsyncTest.State,com.myapp.test.SafeAsyncTest.State) [javac] location: class com.myapp.test.SafeAsyncTest [javac] assertEquals(State.TEST_SUCCESS,state[0]); [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/StoriesTest.java:11: cannot access roboguice.activity.GuiceListActivity [javac] class file for roboguice.activity.GuiceListActivity not found [javac] public class StoriesTest extends ActivityUnitTestCase<Stories> { [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/StoriesTest.java:21: cannot access roboguice.application.GuiceApplication [javac] class file for roboguice.application.GuiceApplication not found [javac] setApplication( new MyApplication( getInstrumentation().getTargetContext() ) ); [javac] ^ [javac] /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/StoriesTest.java:22: incompatible types [javac] found : com.myapp.activity.Stories [javac] required: android.app.Activity [javac] final Activity activity = startActivity(intent, null, null); [javac] ^ [javac] 39 errors [javac] 6 warnings BUILD FAILED /opt/local/android-sdk-mac/platforms/android-1.6/templates/android_rules.xml:248: Compile failed; see the compiler error output for details. Total time: 24 seconds That's not a hard problem to solve. I'm not sure it's the right thing to do, but I copied the missing libraries (roboguice and gson) from the MyApp/libs directory to the MyAppTests/libs directory and everything seems to compile fine. But that leads to the second problem, which I'm currently stuck on. The tests compile fine but they won't run: $ cp ../MyApp/libs/gson-r538.jar libs/ $ cp ../MyApp/libs/roboguice-1.1-SNAPSHOT.jar libs/ 0 10:23 /Users/mike/Projects/myapp/android/MyAppTests $ ant run-testsBuildfile: build.xml [setup] Project Target: Google APIs [setup] Vendor: Google Inc. [setup] Platform Version: 1.6 [setup] API level: 4 [setup] WARNING: No minSdkVersion value set. Application will install on all Android versions. -install-tested-project: [setup] Project Target: Google APIs [setup] Vendor: Google Inc. [setup] Platform Version: 1.6 [setup] API level: 4 [setup] WARNING: No minSdkVersion value set. Application will install on all Android versions. -compile-tested-if-test: -dirs: [echo] Creating output directories if needed... -resource-src: [echo] Generating R.java / Manifest.java from the resources... -aidl: [echo] Compiling aidl files into Java classes... compile: [javac] Compiling 1 source file to /Users/mike/Projects/myapp/android/MyApp/bin/classes -dex: [echo] Converting compiled files and external libraries into /Users/mike/Projects/myapp/android/MyApp/bin/classes.dex... [echo] -package-resources: [echo] Packaging resources [aaptexec] Creating full resource package... -package-debug-sign: [apkbuilder] Creating MyApp-debug-unaligned.apk and signing it with a debug key... [apkbuilder] Using keystore: /Users/mike/.android/debug.keystore debug: [echo] Running zip align on final apk... [echo] Debug Package: /Users/mike/Projects/myapp/android/MyApp/bin/MyApp-debug.apk install: [echo] Installing /Users/mike/Projects/myapp/android/MyApp/bin/MyApp-debug.apk onto default emulator or device... [exec] 1396 KB/s (288354 bytes in 0.201s) [exec] pkg: /data/local/tmp/MyApp-debug.apk [exec] Success -compile-tested-if-test: -dirs: [echo] Creating output directories if needed... -resource-src: [echo] Generating R.java / Manifest.java from the resources... -aidl: [echo] Compiling aidl files into Java classes... compile: [javac] Compiling 5 source files to /Users/mike/Projects/myapp/android/MyAppTests/bin/classes [javac] Note: /Users/mike/Projects/myapp/android/MyAppTests/src/com/myapp/test/SafeAsyncTest.java uses unchecked or unsafe operations. [javac] Note: Recompile with -Xlint:unchecked for details. -dex: [echo] Converting compiled files and external libraries into /Users/mike/Projects/myapp/android/MyAppTests/bin/classes.dex... [echo] -package-resources: [echo] Packaging resources [aaptexec] Creating full resource package... -package-debug-sign: [apkbuilder] Creating MyAppTests-debug-unaligned.apk and signing it with a debug key... [apkbuilder] Using keystore: /Users/mike/.android/debug.keystore debug: [echo] Running zip align on final apk... [echo] Debug Package: /Users/mike/Projects/myapp/android/MyAppTests/bin/MyAppTests-debug.apk install: [echo] Installing /Users/mike/Projects/myapp/android/MyAppTests/bin/MyAppTests-debug.apk onto default emulator or device... [exec] 1227 KB/s (94595 bytes in 0.075s) [exec] pkg: /data/local/tmp/MyAppTests-debug.apk [exec] Success run-tests: [echo] Running tests ... [exec] [exec] android.test.suitebuilder.TestSuiteBuilder$FailedToCreateTests:INSTRUMENTATION_RESULT: shortMsg=Class ref in pre-verified class resolved to unexpected implementation [exec] INSTRUMENTATION_RESULT: longMsg=java.lang.IllegalAccessError: Class ref in pre-verified class resolved to unexpected implementation [exec] INSTRUMENTATION_CODE: 0 BUILD SUCCESSFUL Total time: 38 seconds Any idea what's causing the "Class ref in pre-verified class resolved to unexpected implementation" error?

    Read the article

  • How to Submit Form Given Specific Json Response

    - by dentalhero
    I'm new to Json, so please excuse the newb question. I have a form in which I'm conducting an Ajax post to submit address information to a backend script for validation. Here's the form: <form name="Form" id="Forms" method="post" action="WebCatPageServer.exe" class="uniForm"> <input name="Action" type="hidden" value="SHIPTOVALIDATE"/> <input name="IsAjax" type="hidden" value="Yes"/> <!-- <input name="Action" type="hidden" value="VerifyOrder"/>--> <fieldset class="inlineLabels top"> <h2>Order Details</h2> <div class="ctrlHolder first"> <label for="orderdesc">Order Description</label> <input name="Order Desc" id="OrderDesc" type="text" class="textInput small" tabindex="1" value=""/> </div> <div class="ctrlHolder"> <label for="po">PO # <span class="redasterisk">*</span></label> <input name="Cust Po" id="PoJobNo" type="text" class="textInput small required" maxlength="20" tabindex="2" value="dgnfg"/> </div> <!-- <div class="ctrlHolder"> <label for="jobname">Job Name</label> <input name="Job Name" id="CustJobName" type="text" class="textInput small" maxlength="15" tabindex="3" value=""/> </div> --> <div class="ctrlHolder"> <label for="shipvia">Ship Via <span class="redasterisk">*</span></label> <select name="Ship Via" id="shipvia" class="selectInput small required" tabindex="4"/> <option value="" class="default">Select Ship Method</option> <option value="OT - Our Truck" class="del" selected>Our Truck</option> <option value="WC - Will Call" class="pick">Will Call</option> </select> </div> <div class="ctrlHolder" id="pickupdate"> <label for="datepickup">Requested Pickup Date <span class="redasterisk">*</span></label> <input name="datepickup" id="datepickup" type="text" class="textInput small" tabindex="5" value="11/09/2012"> </div> <div class="ctrlHolder" id="shipdate"> <label for="dateship">Requested Delivery Date <span class="redasterisk">*</span></label> <input name="dateship" id="dateship" type="text" class="textInput small" value="" tabindex="6"> </div> <div class="ctrlHolder" id="shipto"> <label for="ShipTo">Ship To <span class="redasterisk">*</span></label> <select name="ShipTos" id="ShipTos" class="selectInput auto required" tabindex="7"> <option value="">Select an Option</option> <option value="ShipToManual" class="manual">Manually Enter Address</option> <option value="0">A ACTION AIR*, 5241 YANCEYVILLE, COLUMBIA, SC 29214-0001</option> <option value="1">A ACTION AIR*, 649 spring lane, sanford, NC 27330</option> <option value="2">A ACTION AIR*, 1313 south briggs avenue, durham, NC 27703</option> <option value="3">A ACTION AIR*, 112 cricket hill lane, cary, NC 27513</option> <option value="4">A ACTION AIR*, 2911 duke homestead road, durham, NC 27705</option> <option value="5">A ACTION AIR*, chickem poop, atlanta, GA 60609</option> </select> <br /> </div> </fieldset> <fieldset class="inlineLabels" id="shipinfo"> <h2>Shipping Information</h2> <div class="ctrlHolder first"> <label for="YourName">Your Name <span class="redasterisk">*</span></label> <input name="Your Name" id="Your_Name" type="text" class="textInput small required" tabindex="8" value="" /> </div> <div class="ctrlHolder"> <label for="CompanyName">Company Name <span class="redasterisk">*</span></label> <input name="Company Name" id="CompanyName" type="text" class="textInput small required" tabindex="9" value="A ACTION AIR*"/> </div> <div class="ctrlHolder"> <label for="Address1">Address 1 <span class="redasterisk">*</span></label> <input name="Address_1" id="Address_1" type="text" maxlength="30" class="textInput small required" tabindex="10" value="5241 YANCEYVILLE"/> </div> <div class="ctrlHolder"> <label for="Address2">Address 2</label> <input name="Address_2" id="Address_2" type="text" maxlength="30" class="textInput small" tabindex="11" value=""/> </div> <div class="ctrlHolder"> <label for="City">City <span class="redasterisk">*</span></label> <input name="City" id="City" type="text" maxlength="25" class="textInput small required" tabindex="12" value="COLUMBIA"/> </div> <div class="ctrlHolder"> <label for="State">State <span class="redasterisk">*</span></label> <select name="State" id="State" class="selectInput small required" tabindex="13"> <option value="">Select State</option> <option value="AL">Alabama</option> <option value="AK">Alaska</option> <option value="AZ">Arizona</option> <option value="AR">Arkansas</option> <option value="CA">California</option> <option value="CO">Colorado</option> <option value="CT">Connecticut</option> <option value="DE">Delaware</option> <option value="FL">Florida</option> <option value="GA">Georgia</option> <option value="HI">Hawaii</option> <option value="ID">Idaho</option> <option value="IL">Illinois</option> <option value="IN">Indiana</option> <option value="IA">Iowa</option> <option value="KS">Kansas</option> <option value="KY">Kentucky</option> <option value="LA">Louisiana</option> <option value="ME">Maine</option> <option value="MD">Maryland</option> <option value="MA">Massachussetts</option> <option value="MI">Michigan</option> <option value="MN">Minnesota</option> <option value="MS">Mississippi</option> <option value="MO">Missouri</option> <option value="MT">Montana</option> <option value="NE">Nebraska</option> <option value="NV">Nevada</option> <option value="NH">New Hampshire</option> <option value="NJ">New Jersey</option> <option value="NM">New Mexico</option> <option value="NY">New York</option> <option value="NC">North Carolina</option> <option value="ND">North Dakota</option> <option value="OH">Ohio</option> <option value="OK">Oklahoma</option> <option value="OR">Oregon</option> <option value="PA">Pennsylvania</option> <option value="RI">Rhode Island</option> <option value="SC" selected>South Carolina</option> <option value="SD">South Dakota</option> <option value="TN">Tennessee</option> <option value="TX">Texas</option> <option value="UT">Utah</option> <option value="VT">Vermont</option> <option value="VA">Virginia</option> <option value="WA">Washington</option> <option value="WV">West Virginia</option> <option value="WI">Wisconsin</option> <option value="WY">Wyoming</option> </select> </div> <div class="ctrlHolder"> <label for="ZipCode">Zip Code <span class="redasterisk">*</span></label> <input name="Zip" id="Zip" type="text" maxlength="10" class="textInput small required zipcode" tabindex="14" value=""/> </div> <div class="ctrlHolder"> <label for="Phone">Phone <span class="redasterisk">*</span></label> <input name="Phone Number" id="Phone" type="text" class="textInput small required phone" alt="phone-us" tabindex="15" value="(336)954-5009"/> </div> <div class="ctrlHolder"> <label for="Fax">Fax</label> <input name="FaxNumber" id="Fax Number" type="text" class="textInput small fax" alt="phone-us" tabindex="16" value=""/> </div> <div class="ctrlHolder"> <label for="">E-mail <span class="redasterisk">*</span></label> <input name="Email" id="Email" type="text" class="textInput small required email" tabindex="17" value=""/> </div> </fieldset> <fieldset class="inlineLabels"> <h2>Order/Shipping Notes</h2> <div class="ctrlHolder first"> <label for="notes">Order Notes </label> <textarea name="OrderNotes" id="ta" cols="26" rows="7" tabindex="18"></textarea><br /> <p class="formHint"><b>(Maximum characters: 175) &nbsp; <span id="charLeft"></span> &nbsp; Characters left</b><br /> (Cross streets, special instructions, etc.)</p> <br /> </div> </fieldset> <fieldset class="inlineLabels"> <h2>Continue To Next Step</h2> <div class="buttonHolder"> <label for="freightmsg">**Applicable freight charges will be applied at the time of invoicing.**</label> <input name="continuetocheckout" type="submit" class="button red smallrounded" value="Continue &gt;" alt="Continue to Next Step" tabindex="20"/> </div> </fieldset> </form> AJAX Call Here's the AJAX call: $(function() { $("#Forms").submit(function() { $.ajax({ type: 'post', url: 'WebCatPageServer.exe', dataType : 'json', data: $("#Forms").serialize(), complete:function(data){ alert(data); } }); return false; }); }); JSON Response Here's the JSON response: {"DidValidate":true,"Company Name":"A ACTION AIR*","AddrLine1":"5241 YANCEYVILLE","AddrLine2":"","City":"COLUMBIA","State":"SC","Zip":"","Modified":false,"AddressError":false,"ZipError":false} Question: How do I submit the form programatically if both AddressError and ZipError return with a false?

    Read the article

  • Agile Development

    - by James Oloo Onyango
    Alot of literature has and is being written about agile developement and its surrounding philosophies. In my quest to find the best way to express the importance of agile methodologies, i have found Robert C. Martin's "A Satire Of Two Companies" to be both the most concise and thorough! Enjoy the read! Rufus Inc Project Kick Off Your name is Bob. The date is January 3, 2001, and your head still aches from the recent millennial revelry. You are sitting in a conference room with several managers and a group of your peers. You are a project team leader. Your boss is there, and he has brought along all of his team leaders. His boss called the meeting. "We have a new project to develop," says your boss's boss. Call him BB. The points in his hair are so long that they scrape the ceiling. Your boss's points are just starting to grow, but he eagerly awaits the day when he can leave Brylcream stains on the acoustic tiles. BB describes the essence of the new market they have identified and the product they want to develop to exploit this market. "We must have this new project up and working by fourth quarter October 1," BB demands. "Nothing is of higher priority, so we are cancelling your current project." The reaction in the room is stunned silence. Months of work are simply going to be thrown away. Slowly, a murmur of objection begins to circulate around the conference table.   His points give off an evil green glow as BB meets the eyes of everyone in the room. One by one, that insidious stare reduces each attendee to quivering lumps of protoplasm. It is clear that he will brook no discussion on this matter. Once silence has been restored, BB says, "We need to begin immediately. How long will it take you to do the analysis?" You raise your hand. Your boss tries to stop you, but his spitwad misses you and you are unaware of his efforts.   "Sir, we can't tell you how long the analysis will take until we have some requirements." "The requirements document won't be ready for 3 or 4 weeks," BB says, his points vibrating with frustration. "So, pretend that you have the requirements in front of you now. How long will you require for analysis?" No one breathes. Everyone looks around to see whether anyone has some idea. "If analysis goes beyond April 1, we have a problem. Can you finish the analysis by then?" Your boss visibly gathers his courage: "We'll find a way, sir!" His points grow 3 mm, and your headache increases by two Tylenol. "Good." BB smiles. "Now, how long will it take to do the design?" "Sir," you say. Your boss visibly pales. He is clearly worried that his 3 mms are at risk. "Without an analysis, it will not be possible to tell you how long design will take." BB's expression shifts beyond austere.   "PRETEND you have the analysis already!" he says, while fixing you with his vacant, beady little eyes. "How long will it take you to do the design?" Two Tylenol are not going to cut it. Your boss, in a desperate attempt to save his new growth, babbles: "Well, sir, with only six months left to complete the project, design had better take no longer than 3 months."   "I'm glad you agree, Smithers!" BB says, beaming. Your boss relaxes. He knows his points are secure. After a while, he starts lightly humming the Brylcream jingle. BB continues, "So, analysis will be complete by April 1, design will be complete by July 1, and that gives you 3 months to implement the project. This meeting is an example of how well our new consensus and empowerment policies are working. Now, get out there and start working. I'll expect to see TQM plans and QIT assignments on my desk by next week. Oh, and don't forget that your crossfunctional team meetings and reports will be needed for next month's quality audit." "Forget the Tylenol," you think to yourself as you return to your cubicle. "I need bourbon."   Visibly excited, your boss comes over to you and says, "Gosh, what a great meeting. I think we're really going to do some world shaking with this project." You nod in agreement, too disgusted to do anything else. "Oh," your boss continues, "I almost forgot." He hands you a 30-page document. "Remember that the SEI is coming to do an evaluation next week. This is the evaluation guide. You need to read through it, memorize it, and then shred it. It tells you how to answer any questions that the SEI auditors ask you. It also tells you what parts of the building you are allowed to take them to and what parts to avoid. We are determined to be a CMM level 3 organization by June!"   You and your peers start working on the analysis of the new project. This is difficult because you have no requirements. But from the 10-minute introduction given by BB on that fateful morning, you have some idea of what the product is supposed to do.   Corporate process demands that you begin by creating a use case document. You and your team begin enumerating use cases and drawing oval and stick diagrams. Philosophical debates break out among the team members. There is disagreement as to whether certain use cases should be connected with <<extends>> or <<includes>> relationships. Competing models are created, but nobody knows how to evaluate them. The debate continues, effectively paralyzing progress.   After a week, somebody finds the iceberg.com Web site, which recommends disposing entirely of <<extends>> and <<includes>> and replacing them with <<precedes>> and <<uses>>. The documents on this Web site, authored by Don Sengroiux, describes a method known as stalwart-analysis, which claims to be a step-by-step method for translating use cases into design diagrams. More competing use case models are created using this new scheme, but again, people can't agree on how to evaluate them. The thrashing continues. More and more, the use case meetings are driven by emotion rather than by reason. If it weren't for the fact that you don't have requirements, you'd be pretty upset by the lack of progress you are making. The requirements document arrives on February 15. And then again on February 20, 25, and every week thereafter. Each new version contradicts the previous one. Clearly, the marketing folks who are writing the requirements, empowered though they might be, are not finding consensus.   At the same time, several new competing use case templates have been proposed by the various team members. Each template presents its own particularly creative way of delaying progress. The debates rage on. On March 1, Prudence Putrigence, the process proctor, succeeds in integrating all the competing use case forms and templates into a single, all-encompassing form. Just the blank form is 15 pages long. She has managed to include every field that appeared on all the competing templates. She also presents a 159- page document describing how to fill out the use case form. All current use cases must be rewritten according to the new standard.   You marvel to yourself that it now requires 15 pages of fill-in-the-blank and essay questions to answer the question: What should the system do when the user presses Return? The corporate process (authored by L. E. Ott, famed author of "Holistic Analysis: A Progressive Dialectic for Software Engineers") insists that you discover all primary use cases, 87 percent of all secondary use cases, and 36.274 percent of all tertiary use cases before you can complete analysis and enter the design phase. You have no idea what a tertiary use case is. So in an attempt to meet this requirement, you try to get your use case document reviewed by the marketing department, which you hope will know what a tertiary use case is.   Unfortunately, the marketing folks are too busy with sales support to talk to you. Indeed, since the project started, you have not been able to get a single meeting with marketing, which has provided a never-ending stream of changing and contradictory requirements documents.   While one team has been spinning endlessly on the use case document, another team has been working out the domain model. Endless variations of UML documents are pouring out of this team. Every week, the model is reworked.   The team members can't decide whether to use <<interfaces>> or <<types>> in the model. A huge disagreement has been raging on the proper syntax and application of OCL. Others on the team just got back from a 5-day class on catabolism, and have been producing incredibly detailed and arcane diagrams that nobody else can fathom.   On March 27, with one week to go before analysis is to be complete, you have produced a sea of documents and diagrams but are no closer to a cogent analysis of the problem than you were on January 3. **** And then, a miracle happens.   **** On Saturday, April 1, you check your e-mail from home. You see a memo from your boss to BB. It states unequivocally that you are done with the analysis! You phone your boss and complain. "How could you have told BB that we were done with the analysis?" "Have you looked at a calendar lately?" he responds. "It's April 1!" The irony of that date does not escape you. "But we have so much more to think about. So much more to analyze! We haven't even decided whether to use <<extends>> or <<precedes>>!" "Where is your evidence that you are not done?" inquires your boss, impatiently. "Whaaa . . . ." But he cuts you off. "Analysis can go on forever; it has to be stopped at some point. And since this is the date it was scheduled to stop, it has been stopped. Now, on Monday, I want you to gather up all existing analysis materials and put them into a public folder. Release that folder to Prudence so that she can log it in the CM system by Monday afternoon. Then get busy and start designing."   As you hang up the phone, you begin to consider the benefits of keeping a bottle of bourbon in your bottom desk drawer. They threw a party to celebrate the on-time completion of the analysis phase. BB gave a colon-stirring speech on empowerment. And your boss, another 3 mm taller, congratulated his team on the incredible show of unity and teamwork. Finally, the CIO takes the stage to tell everyone that the SEI audit went very well and to thank everyone for studying and shredding the evaluation guides that were passed out. Level 3 now seems assured and will be awarded by June. (Scuttlebutt has it that managers at the level of BB and above are to receive significant bonuses once the SEI awards level 3.)   As the weeks flow by, you and your team work on the design of the system. Of course, you find that the analysis that the design is supposedly based on is flawedno, useless; no, worse than useless. But when you tell your boss that you need to go back and work some more on the analysis to shore up its weaker sections, he simply states, "The analysis phase is over. The only allowable activity is design. Now get back to it."   So, you and your team hack the design as best you can, unsure of whether the requirements have been properly analyzed. Of course, it really doesn't matter much, since the requirements document is still thrashing with weekly revisions, and the marketing department still refuses to meet with you.     The design is a nightmare. Your boss recently misread a book named The Finish Line in which the author, Mark DeThomaso, blithely suggested that design documents should be taken down to code-level detail. "If we are going to be working at that level of detail," you ask, "why don't we simply write the code instead?" "Because then you wouldn't be designing, of course. And the only allowable activity in the design phase is design!" "Besides," he continues, "we have just purchased a companywide license for Dandelion! This tool enables 'Round the Horn Engineering!' You are to transfer all design diagrams into this tool. It will automatically generate our code for us! It will also keep the design diagrams in sync with the code!" Your boss hands you a brightly colored shrinkwrapped box containing the Dandelion distribution. You accept it numbly and shuffle off to your cubicle. Twelve hours, eight crashes, one disk reformatting, and eight shots of 151 later, you finally have the tool installed on your server. You consider the week your team will lose while attending Dandelion training. Then you smile and think, "Any week I'm not here is a good week." Design diagram after design diagram is created by your team. Dandelion makes it very difficult to draw these diagrams. There are dozens and dozens of deeply nested dialog boxes with funny text fields and check boxes that must all be filled in correctly. And then there's the problem of moving classes between packages. At first, these diagram are driven from the use cases. But the requirements are changing so often that the use cases rapidly become meaningless. Debates rage about whether VISITOR or DECORATOR design patterns should be used. One developer refuses to use VISITOR in any form, claiming that it's not a properly object-oriented construct. Someone refuses to use multiple inheritance, since it is the spawn of the devil. Review meetings rapidly degenerate into debates about the meaning of object orientation, the definition of analysis versus design, or when to use aggregation versus association. Midway through the design cycle, the marketing folks announce that they have rethought the focus of the system. Their new requirements document is completely restructured. They have eliminated several major feature areas and replaced them with feature areas that they anticipate customer surveys will show to be more appropriate. You tell your boss that these changes mean that you need to reanalyze and redesign much of the system. But he says, "The analysis phase is system. But he says, "The analysis phase is over. The only allowable activity is design. Now get back to it."   You suggest that it might be better to create a simple prototype to show to the marketing folks and even some potential customers. But your boss says, "The analysis phase is over. The only allowable activity is design. Now get back to it." Hack, hack, hack, hack. You try to create some kind of a design document that might reflect the new requirements documents. However, the revolution of the requirements has not caused them to stop thrashing. Indeed, if anything, the wild oscillations of the requirements document have only increased in frequency and amplitude.   You slog your way through them.   On June 15, the Dandelion database gets corrupted. Apparently, the corruption has been progressive. Small errors in the DB accumulated over the months into bigger and bigger errors. Eventually, the CASE tool just stopped working. Of course, the slowly encroaching corruption is present on all the backups. Calls to the Dandelion technical support line go unanswered for several days. Finally, you receive a brief e-mail from Dandelion, informing you that this is a known problem and that the solution is to purchase the new version, which they promise will be ready some time next quarter, and then reenter all the diagrams by hand.   ****   Then, on July 1 another miracle happens! You are done with the design!   Rather than go to your boss and complain, you stock your middle desk drawer with some vodka.   **** They threw a party to celebrate the on-time completion of the design phase and their graduation to CMM level 3. This time, you find BB's speech so stirring that you have to use the restroom before it begins. New banners and plaques are all over your workplace. They show pictures of eagles and mountain climbers, and they talk about teamwork and empowerment. They read better after a few scotches. That reminds you that you need to clear out your file cabinet to make room for the brandy. You and your team begin to code. But you rapidly discover that the design is lacking in some significant areas. Actually, it's lacking any significance at all. You convene a design session in one of the conference rooms to try to work through some of the nastier problems. But your boss catches you at it and disbands the meeting, saying, "The design phase is over. The only allowable activity is coding. Now get back to it."   ****   The code generated by Dandelion is really hideous. It turns out that you and your team were using association and aggregation the wrong way, after all. All the generated code has to be edited to correct these flaws. Editing this code is extremely difficult because it has been instrumented with ugly comment blocks that have special syntax that Dandelion needs in order to keep the diagrams in sync with the code. If you accidentally alter one of these comments, the diagrams will be regenerated incorrectly. It turns out that "Round the Horn Engineering" requires an awful lot of effort. The more you try to keep the code compatible with Dandelion, the more errors Dandelion generates. In the end, you give up and decide to keep the diagrams up to date manually. A second later, you decide that there's no point in keeping the diagrams up to date at all. Besides, who has time?   Your boss hires a consultant to build tools to count the number of lines of code that are being produced. He puts a big thermometer graph on the wall with the number 1,000,000 on the top. Every day, he extends the red line to show how many lines have been added. Three days after the thermometer appears on the wall, your boss stops you in the hall. "That graph isn't growing quickly enough. We need to have a million lines done by October 1." "We aren't even sh-sh-sure that the proshect will require a m-million linezh," you blather. "We have to have a million lines done by October 1," your boss reiterates. His points have grown again, and the Grecian formula he uses on them creates an aura of authority and competence. "Are you sure your comment blocks are big enough?" Then, in a flash of managerial insight, he says, "I have it! I want you to institute a new policy among the engineers. No line of code is to be longer than 20 characters. Any such line must be split into two or more preferably more. All existing code needs to be reworked to this standard. That'll get our line count up!"   You decide not to tell him that this will require two unscheduled work months. You decide not to tell him anything at all. You decide that intravenous injections of pure ethanol are the only solution. You make the appropriate arrangements. Hack, hack, hack, and hack. You and your team madly code away. By August 1, your boss, frowning at the thermometer on the wall, institutes a mandatory 50-hour workweek.   Hack, hack, hack, and hack. By September 1st, the thermometer is at 1.2 million lines and your boss asks you to write a report describing why you exceeded the coding budget by 20 percent. He institutes mandatory Saturdays and demands that the project be brought back down to a million lines. You start a campaign of remerging lines. Hack, hack, hack, and hack. Tempers are flaring; people are quitting; QA is raining trouble reports down on you. Customers are demanding installation and user manuals; salespeople are demanding advance demonstrations for special customers; the requirements document is still thrashing, the marketing folks are complaining that the product isn't anything like they specified, and the liquor store won't accept your credit card anymore. Something has to give.    On September 15, BB calls a meeting. As he enters the room, his points are emitting clouds of steam. When he speaks, the bass overtones of his carefully manicured voice cause the pit of your stomach to roll over. "The QA manager has told me that this project has less than 50 percent of the required features implemented. He has also informed me that the system crashes all the time, yields wrong results, and is hideously slow. He has also complained that he cannot keep up with the continuous train of daily releases, each more buggy than the last!" He stops for a few seconds, visibly trying to compose himself. "The QA manager estimates that, at this rate of development, we won't be able to ship the product until December!" Actually, you think it's more like March, but you don't say anything. "December!" BB roars with such derision that people duck their heads as though he were pointing an assault rifle at them. "December is absolutely out of the question. Team leaders, I want new estimates on my desk in the morning. I am hereby mandating 65-hour work weeks until this project is complete. And it better be complete by November 1."   As he leaves the conference room, he is heard to mutter: "Empowermentbah!" * * * Your boss is bald; his points are mounted on BB's wall. The fluorescent lights reflecting off his pate momentarily dazzle you. "Do you have anything to drink?" he asks. Having just finished your last bottle of Boone's Farm, you pull a bottle of Thunderbird from your bookshelf and pour it into his coffee mug. "What's it going to take to get this project done? " he asks. "We need to freeze the requirements, analyze them, design them, and then implement them," you say callously. "By November 1?" your boss exclaims incredulously. "No way! Just get back to coding the damned thing." He storms out, scratching his vacant head.   A few days later, you find that your boss has been transferred to the corporate research division. Turnover has skyrocketed. Customers, informed at the last minute that their orders cannot be fulfilled on time, have begun to cancel their orders. Marketing is re-evaluating whether this product aligns with the overall goals of the company. Memos fly, heads roll, policies change, and things are, overall, pretty grim. Finally, by March, after far too many sixty-five hour weeks, a very shaky version of the software is ready. In the field, bug-discovery rates are high, and the technical support staff are at their wits' end, trying to cope with the complaints and demands of the irate customers. Nobody is happy.   In April, BB decides to buy his way out of the problem by licensing a product produced by Rupert Industries and redistributing it. The customers are mollified, the marketing folks are smug, and you are laid off.     Rupert Industries: Project Alpha   Your name is Robert. The date is January 3, 2001. The quiet hours spent with your family this holiday have left you refreshed and ready for work. You are sitting in a conference room with your team of professionals. The manager of the division called the meeting. "We have some ideas for a new project," says the division manager. Call him Russ. He is a high-strung British chap with more energy than a fusion reactor. He is ambitious and driven but understands the value of a team. Russ describes the essence of the new market opportunity the company has identified and introduces you to Jane, the marketing manager, who is responsible for defining the products that will address it. Addressing you, Jane says, "We'd like to start defining our first product offering as soon as possible. When can you and your team meet with me?" You reply, "We'll be done with the current iteration of our project this Friday. We can spare a few hours for you between now and then. After that, we'll take a few people from the team and dedicate them to you. We'll begin hiring their replacements and the new people for your team immediately." "Great," says Russ, "but I want you to understand that it is critical that we have something to exhibit at the trade show coming up this July. If we can't be there with something significant, we'll lose the opportunity."   "I understand," you reply. "I don't yet know what it is that you have in mind, but I'm sure we can have something by July. I just can't tell you what that something will be right now. In any case, you and Jane are going to have complete control over what we developers do, so you can rest assured that by July, you'll have the most important things that can be accomplished in that time ready to exhibit."   Russ nods in satisfaction. He knows how this works. Your team has always kept him advised and allowed him to steer their development. He has the utmost confidence that your team will work on the most important things first and will produce a high-quality product.   * * *   "So, Robert," says Jane at their first meeting, "How does your team feel about being split up?" "We'll miss working with each other," you answer, "but some of us were getting pretty tired of that last project and are looking forward to a change. So, what are you people cooking up?" Jane beams. "You know how much trouble our customers currently have . . ." And she spends a half hour or so describing the problem and possible solution. "OK, wait a second" you respond. "I need to be clear about this." And so you and Jane talk about how this system might work. Some of her ideas aren't fully formed. You suggest possible solutions. She likes some of them. You continue discussing.   During the discussion, as each new topic is addressed, Jane writes user story cards. Each card represents something that the new system has to do. The cards accumulate on the table and are spread out in front of you. Both you and Jane point at them, pick them up, and make notes on them as you discuss the stories. The cards are powerful mnemonic devices that you can use to represent complex ideas that are barely formed.   At the end of the meeting, you say, "OK, I've got a general idea of what you want. I'm going to talk to the team about it. I imagine they'll want to run some experiments with various database structures and presentation formats. Next time we meet, it'll be as a group, and we'll start identifying the most important features of the system."   A week later, your nascent team meets with Jane. They spread the existing user story cards out on the table and begin to get into some of the details of the system. The meeting is very dynamic. Jane presents the stories in the order of their importance. There is much discussion about each one. The developers are concerned about keeping the stories small enough to estimate and test. So they continually ask Jane to split one story into several smaller stories. Jane is concerned that each story have a clear business value and priority, so as she splits them, she makes sure that this stays true.   The stories accumulate on the table. Jane writes them, but the developers make notes on them as needed. Nobody tries to capture everything that is said; the cards are not meant to capture everything but are simply reminders of the conversation.   As the developers become more comfortable with the stories, they begin writing estimates on them. These estimates are crude and budgetary, but they give Jane an idea of what the story will cost.   At the end of the meeting, it is clear that many more stories could be discussed. It is also clear that the most important stories have been addressed and that they represent several months worth of work. Jane closes the meeting by taking the cards with her and promising to have a proposal for the first release in the morning.   * * *   The next morning, you reconvene the meeting. Jane chooses five cards and places them on the table. "According to your estimates, these cards represent about one perfect team-week's worth of work. The last iteration of the previous project managed to get one perfect team-week done in 3 real weeks. If we can get these five stories done in 3 weeks, we'll be able to demonstrate them to Russ. That will make him feel very comfortable about our progress." Jane is pushing it. The sheepish look on her face lets you know that she knows it too. You reply, "Jane, this is a new team, working on a new project. It's a bit presumptuous to expect that our velocity will be the same as the previous team's. However, I met with the team yesterday afternoon, and we all agreed that our initial velocity should, in fact, be set to one perfectweek for every 3 real-weeks. So you've lucked out on this one." "Just remember," you continue, "that the story estimates and the story velocity are very tentative at this point. We'll learn more when we plan the iteration and even more when we implement it."   Jane looks over her glasses at you as if to say "Who's the boss around here, anyway?" and then smiles and says, "Yeah, don't worry. I know the drill by now."Jane then puts 15 more cards on the table. She says, "If we can get all these cards done by the end of March, we can turn the system over to our beta test customers. And we'll get good feedback from them."   You reply, "OK, so we've got our first iteration defined, and we have the stories for the next three iterations after that. These four iterations will make our first release."   "So," says Jane, can you really do these five stories in the next 3 weeks?" "I don't know for sure, Jane," you reply. "Let's break them down into tasks and see what we get."   So Jane, you, and your team spend the next several hours taking each of the five stories that Jane chose for the first iteration and breaking them down into small tasks. The developers quickly realize that some of the tasks can be shared between stories and that other tasks have commonalities that can probably be taken advantage of. It is clear that potential designs are popping into the developers' heads. From time to time, they form little discussion knots and scribble UML diagrams on some cards.   Soon, the whiteboard is filled with the tasks that, once completed, will implement the five stories for this iteration. You start the sign-up process by saying, "OK, let's sign up for these tasks." "I'll take the initial database generation." Says Pete. "That's what I did on the last project, and this doesn't look very different. I estimate it at two of my perfect workdays." "OK, well, then, I'll take the login screen," says Joe. "Aw, darn," says Elaine, the junior member of the team, "I've never done a GUI, and kinda wanted to try that one."   "Ah, the impatience of youth," Joe says sagely, with a wink in your direction. "You can assist me with it, young Jedi." To Jane: "I think it'll take me about three of my perfect workdays."   One by one, the developers sign up for tasks and estimate them in terms of their own perfect workdays. Both you and Jane know that it is best to let the developers volunteer for tasks than to assign the tasks to them. You also know full well that you daren't challenge any of the developers' estimates. You know these people, and you trust them. You know that they are going to do the very best they can.   The developers know that they can't sign up for more perfect workdays than they finished in the last iteration they worked on. Once each developer has filled his or her schedule for the iteration, they stop signing up for tasks.   Eventually, all the developers have stopped signing up for tasks. But, of course, tasks are still left on the board.   "I was worried that that might happen," you say, "OK, there's only one thing to do, Jane. We've got too much to do in this iteration. What stories or tasks can we remove?" Jane sighs. She knows that this is the only option. Working overtime at the beginning of a project is insane, and projects where she's tried it have not fared well.   So Jane starts to remove the least-important functionality. "Well, we really don't need the login screen just yet. We can simply start the system in the logged-in state." "Rats!" cries Elaine. "I really wanted to do that." "Patience, grasshopper." says Joe. "Those who wait for the bees to leave the hive will not have lips too swollen to relish the honey." Elaine looks confused. Everyone looks confused. "So . . .," Jane continues, "I think we can also do away with . . ." And so, bit by bit, the list of tasks shrinks. Developers who lose a task sign up for one of the remaining ones.   The negotiation is not painless. Several times, Jane exhibits obvious frustration and impatience. Once, when tensions are especially high, Elaine volunteers, "I'll work extra hard to make up some of the missing time." You are about to correct her when, fortunately, Joe looks her in the eye and says, "When once you proceed down the dark path, forever will it dominate your destiny."   In the end, an iteration acceptable to Jane is reached. It's not what Jane wanted. Indeed, it is significantly less. But it's something the team feels that can be achieved in the next 3 weeks.   And, after all, it still addresses the most important things that Jane wanted in the iteration. "So, Jane," you say when things had quieted down a bit, "when can we expect acceptance tests from you?" Jane sighs. This is the other side of the coin. For every story the development team implements,   Jane must supply a suite of acceptance tests that prove that it works. And the team needs these long before the end of the iteration, since they will certainly point out differences in the way Jane and the developers imagine the system's behaviour.   "I'll get you some example test scripts today," Jane promises. "I'll add to them every day after that. You'll have the entire suite by the middle of the iteration."   * * *   The iteration begins on Monday morning with a flurry of Class, Responsibilities, Collaborators sessions. By midmorning, all the developers have assembled into pairs and are rapidly coding away. "And now, my young apprentice," Joe says to Elaine, "you shall learn the mysteries of test-first design!"   "Wow, that sounds pretty rad," Elaine replies. "How do you do it?" Joe beams. It's clear that he has been anticipating this moment. "OK, what does the code do right now?" "Huh?" replied Elaine, "It doesn't do anything at all; there is no code."   "So, consider our task; can you think of something the code should do?" "Sure," Elaine said with youthful assurance, "First, it should connect to the database." "And thereupon, what must needs be required to connecteth the database?" "You sure talk weird," laughed Elaine. "I think we'd have to get the database object from some registry and call the Connect() method. "Ah, astute young wizard. Thou perceives correctly that we requireth an object within which we can cacheth the database object." "Is 'cacheth' really a word?" "It is when I say it! So, what test can we write that we know the database registry should pass?" Elaine sighs. She knows she'll just have to play along. "We should be able to create a database object and pass it to the registry in a Store() method. And then we should be able to pull it out of the registry with a Get() method and make sure it's the same object." "Oh, well said, my prepubescent sprite!" "Hay!" "So, now, let's write a test function that proves your case." "But shouldn't we write the database object and registry object first?" "Ah, you've much to learn, my young impatient one. Just write the test first." "But it won't even compile!" "Are you sure? What if it did?" "Uh . . ." "Just write the test, Elaine. Trust me." And so Joe, Elaine, and all the other developers began to code their tasks, one test case at a time. The room in which they worked was abuzz with the conversations between the pairs. The murmur was punctuated by an occasional high five when a pair managed to finish a task or a difficult test case.   As development proceeded, the developers changed partners once or twice a day. Each developer got to see what all the others were doing, and so knowledge of the code spread generally throughout the team.   Whenever a pair finished something significant whether a whole task or simply an important part of a task they integrated what they had with the rest of the system. Thus, the code base grew daily, and integration difficulties were minimized.   The developers communicated with Jane on a daily basis. They'd go to her whenever they had a question about the functionality of the system or the interpretation of an acceptance test case.   Jane, good as her word, supplied the team with a steady stream of acceptance test scripts. The team read these carefully and thereby gained a much better understanding of what Jane expected the system to do. By the beginning of the second week, there was enough functionality to demonstrate to Jane. She watched eagerly as the demonstration passed test case after test case. "This is really cool," Jane said as the demonstration finally ended. "But this doesn't seem like one-third of the tasks. Is your velocity slower than anticipated?"   You grimace. You'd been waiting for a good time to mention this to Jane but now she was forcing the issue. "Yes, unfortunately, we are going more slowly than we had expected. The new application server we are using is turning out to be a pain to configure. Also, it takes forever to reboot, and we have to reboot it whenever we make even the slightest change to its configuration."   Jane eyes you with suspicion. The stress of last Monday's negotiations had still not entirely dissipated. She says, "And what does this mean to our schedule? We can't slip it again, we just can't. Russ will have a fit! He'll haul us all into the woodshed and ream us some new ones."   You look Jane right in the eyes. There's no pleasant way to give someone news like this. So you just blurt out, "Look, if things keep going like they're going, we're not going to be done with everything by next Friday. Now it's possible that we'll figure out a way to go faster. But, frankly, I wouldn't depend on that. You should start thinking about one or two tasks that could be removed from the iteration without ruining the demonstration for Russ. Come hell or high water, we are going to give that demonstration on Friday, and I don't think you want us to choose which tasks to omit."   "Aw forchrisakes!" Jane barely manages to stifle yelling that last word as she stalks away, shaking her head. Not for the first time, you say to yourself, "Nobody ever promised me project management would be easy." You are pretty sure it won't be the last time, either.   Actually, things went a bit better than you had hoped. The team did, in fact, have to drop one task from the iteration, but Jane had chosen wisely, and the demonstration for Russ went without a hitch. Russ was not impressed with the progress, but neither was he dismayed. He simply said, "This is pretty good. But remember, we have to be able to demonstrate this system at the trade show in July, and at this rate, it doesn't look like you'll have all that much to show." Jane, whose attitude had improved dramatically with the completion of the iteration, responded to Russ by saying, "Russ, this team is working hard, and well. When July comes around, I am confident that we'll have something significant to demonstrate. It won't be everything, and some of it may be smoke and mirrors, but we'll have something."   Painful though the last iteration was, it had calibrated your velocity numbers. The next iteration went much better. Not because your team got more done than in the last iteration but simply because the team didn't have to remove any tasks or stories in the middle of the iteration.   By the start of the fourth iteration, a natural rhythm has been established. Jane, you, and the team know exactly what to expect from one another. The team is running hard, but the pace is sustainable. You are confident that the team can keep up this pace for a year or more.   The number of surprises in the schedule diminishes to near zero; however, the number of surprises in the requirements does not. Jane and Russ frequently look over the growing system and make recommendations or changes to the existing functionality. But all parties realize that these changes take time and must be scheduled. So the changes do not cause anyone's expectations to be violated. In March, there is a major demonstration of the system to the board of directors. The system is very limited and is not yet in a form good enough to take to the trade show, but progress is steady, and the board is reasonably impressed.   The second release goes even more smoothly than the first. By now, the team has figured out a way to automate Jane's acceptance test scripts. The team has also refactored the design of the system to the point that it is really easy to add new features and change old ones. The second release was done by the end of June and was taken to the trade show. It had less in it than Jane and Russ would have liked, but it did demonstrate the most important features of the system. Although customers at the trade show noticed that certain features were missing, they were very impressed overall. You, Russ, and Jane all returned from the trade show with smiles on your faces. You all felt as though this project was a winner.   Indeed, many months later, you are contacted by Rufus Inc. That company had been working on a system like this for its internal operations. Rufus has canceled the development of that system after a death-march project and is negotiating to license your technology for its environment.   Indeed, things are looking up!

    Read the article

< Previous Page | 89 90 91 92 93