httpcontext cache - Page 284

actionscript3: reflect-class applied on rotationY

- by algro

Hi, I'm using a class which applies a visual reflection-effect to defined movieclips. I use a reflection-class from here: link to source. It works like a charm except when I apply a rotation to the movieclip. In my case the reflection is still visible but only a part of it. What am I doing wrong? How could I pass/include the rotation to the Reflection-Class ? Thanks in advance! This is how you apply the Reflection Class to your movieclip: var ref_mc:MovieClip = new MoviClip(); addChild(ref_mc); var r1:Reflect = new Reflect({mc:ref_mc, alpha:50, ratio:50,distance:0, updateTime:0,reflectionDropoff:1}); Now I apply a rotation to my movieclip: ref_mc.rotationY = 30; And Here the Reflect-Class: package com.pixelfumes.reflect{ import flash.display.MovieClip; import flash.display.DisplayObject; import flash.display.BitmapData; import flash.display.Bitmap; import flash.geom.Matrix; import flash.display.GradientType; import flash.display.SpreadMethod; import flash.utils.setInterval; import flash.utils.clearInterval; public class Reflect extends MovieClip{ //Created By Ben Pritchard of Pixelfumes 2007 //Thanks to Mim, Jasper, Jason Merrill and all the others who //have contributed to the improvement of this class //static var for the version of this class private static var VERSION:String = "4.0"; //reference to the movie clip we are reflecting private var mc:MovieClip; //the BitmapData object that will hold a visual copy of the mc private var mcBMP:BitmapData; //the BitmapData object that will hold the reflected image private var reflectionBMP:Bitmap; //the clip that will act as out gradient mask private var gradientMask_mc:MovieClip; //how often the reflection should update (if it is video or animated) private var updateInt:Number; //the size the reflection is allowed to reflect within private var bounds:Object; //the distance the reflection is vertically from the mc private var distance:Number = 0; function Reflect(args:Object){ /*the args object passes in the following variables /we set the values of our internal vars to math the args*/ //the clip being reflected mc = args.mc; //the alpha level of the reflection clip var alpha:Number = args.alpha/100; //the ratio opaque color used in the gradient mask var ratio:Number = args.ratio; //update time interval var updateTime:Number = args.updateTime; //the distance at which the reflection visually drops off at var reflectionDropoff:Number = args.reflectionDropoff; //the distance the reflection starts from the bottom of the mc var distance:Number = args.distance; //store width and height of the clip var mcHeight = mc.height; var mcWidth = mc.width; //store the bounds of the reflection bounds = new Object(); bounds.width = mcWidth; bounds.height = mcHeight; //create the BitmapData that will hold a snapshot of the movie clip mcBMP = new BitmapData(bounds.width, bounds.height, true, 0xFFFFFF); mcBMP.draw(mc); //create the BitmapData the will hold the reflection reflectionBMP = new Bitmap(mcBMP); //flip the reflection upside down reflectionBMP.scaleY = -1; //move the reflection to the bottom of the movie clip reflectionBMP.y = (bounds.height*2) + distance; //add the reflection to the movie clip's Display Stack var reflectionBMPRef:DisplayObject = mc.addChild(reflectionBMP); reflectionBMPRef.name = "reflectionBMP"; //add a blank movie clip to hold our gradient mask var gradientMaskRef:DisplayObject = mc.addChild(new MovieClip()); gradientMaskRef.name = "gradientMask_mc"; //get a reference to the movie clip - cast the DisplayObject that is returned as a MovieClip gradientMask_mc = mc.getChildByName("gradientMask_mc") as MovieClip; //set the values for the gradient fill var fillType:String = GradientType.LINEAR; var colors:Array = [0xFFFFFF, 0xFFFFFF]; var alphas:Array = [alpha, 0]; var ratios:Array = [0, ratio]; var spreadMethod:String = SpreadMethod.PAD; //create the Matrix and create the gradient box var matr:Matrix = new Matrix(); //set the height of the Matrix used for the gradient mask var matrixHeight:Number; if (reflectionDropoff<=0) { matrixHeight = bounds.height; } else { matrixHeight = bounds.height/reflectionDropoff; } matr.createGradientBox(bounds.width, matrixHeight, (90/180)*Math.PI, 0, 0); //create the gradient fill gradientMask_mc.graphics.beginGradientFill(fillType, colors, alphas, ratios, matr, spreadMethod); gradientMask_mc.graphics.drawRect(0,0,bounds.width,bounds.height); //position the mask over the reflection clip gradientMask_mc.y = mc.getChildByName("reflectionBMP").y - mc.getChildByName("reflectionBMP").height; //cache clip as a bitmap so that the gradient mask will function gradientMask_mc.cacheAsBitmap = true; mc.getChildByName("reflectionBMP").cacheAsBitmap = true; //set the mask for the reflection as the gradient mask mc.getChildByName("reflectionBMP").mask = gradientMask_mc; //if we are updating the reflection for a video or animation do so here if(updateTime > -1){ updateInt = setInterval(update, updateTime, mc); } } public function setBounds(w:Number,h:Number):void{ //allows the user to set the area that the reflection is allowed //this is useful for clips that move within themselves bounds.width = w; bounds.height = h; gradientMask_mc.width = bounds.width; redrawBMP(mc); } public function redrawBMP(mc:MovieClip):void { // redraws the bitmap reflection - Mim Gamiet [2006] mcBMP.dispose(); mcBMP = new BitmapData(bounds.width, bounds.height, true, 0xFFFFFF); mcBMP.draw(mc); } private function update(mc):void { //updates the reflection to visually match the movie clip mcBMP = new BitmapData(bounds.width, bounds.height, true, 0xFFFFFF); mcBMP.draw(mc); reflectionBMP.bitmapData = mcBMP; } public function destroy():void{ //provides a method to remove the reflection mc.removeChild(mc.getChildByName("reflectionBMP")); reflectionBMP = null; mcBMP.dispose(); clearInterval(updateInt); mc.removeChild(mc.getChildByName("gradientMask_mc")); } } }

Read the article

Asset Pipeline acting up

- by Abram

Ok, so my asset pipeline has suddenly started acting up on my development machine. JS functions that previously worked are now throwing "not a function" errors.. I know I must be doing something wrong. A minute ago the datatables jquery function was working, then it was throwing an error, then it was working, and now it's not working or throwing an error. Here is my application.js //= require jquery //= require jquery-ui //= require jquery_ujs //= require_self //= require_tree . //= require dataTables/jquery.dataTables //= require dataTables/jquery.dataTables.bootstrap //= require bootstrap //= require bootstrap-tooltip //= require bootstrap-popover //= require bootstrap-tab //= require bootstrap-modal //= require bootstrap-alert //= require bootstrap-dropdown //= require jquery.ui.addresspicker //= require raty //= require jquery.alphanumeric //= require jquery.formrestrict //= require select2 //= require chosen/chosen.jquery //= require highcharts //= require jquery.lazyload Here is some of my layout header: <%= stylesheet_link_tag "application", media: "all" %> <%= yield(:scripthead) %> <%= javascript_include_tag "application" %> <%= csrf_meta_tags %> <%= yield(:head) %> Above I am using the yield to load up online scripts from google as they're only needed on some pages, and generally slow down the site if included in the application layout. I tried removing the yield but things were still broken, even after clearing the browser cache and running rake assets:clean (just to be on the safe side). Here's what shows up between CSS and metatags (for a page with nothin in the yield scripthead): <script src="/assets/jquery.js?body=1" type="text/javascript"></script> <script src="/assets/jquery-ui.js?body=1" type="text/javascript"></script> <script src="/assets/jquery_ujs.js?body=1" type="text/javascript"></script> <script src="/assets/application.js?body=1" type="text/javascript"></script> <script src="/assets/aidmodels.js?body=1" type="text/javascript"></script> <script src="/assets/audio.js?body=1" type="text/javascript"></script> <script src="/assets/bootstrap-alert.js?body=1" type="text/javascript"></script> <script src="/assets/bootstrap-dropdown.js?body=1" type="text/javascript"></script> <script src="/assets/bootstrap-modal.js?body=1" type="text/javascript"></script> <script src="/assets/bootstrap-popover.js?body=1" type="text/javascript"></script> <script src="/assets/bootstrap-tab.js?body=1" type="text/javascript"></script> <script src="/assets/bootstrap-tooltip.js?body=1" type="text/javascript"></script> <script src="/assets/branches.js?body=1" type="text/javascript"></script> <script src="/assets/charts.js?body=1" type="text/javascript"></script> <script src="/assets/chosen/backup_chosen.jquery.js?body=1" type="text/javascript"></script> <script src="/assets/chosen/chosen.jquery.js?body=1" type="text/javascript"></script> <script src="/assets/consumers.js?body=1" type="text/javascript"></script> <script src="/assets/dispensers.js?body=1" type="text/javascript"></script> <script src="/assets/favorites.js?body=1" type="text/javascript"></script> <script src="/assets/features.js?body=1" type="text/javascript"></script> <script src="/assets/generic_styles.js?body=1" type="text/javascript"></script> <script src="/assets/gmaps4rails/gmaps4rails.base.js?body=1" type="text/javascript"></script> <script src="/assets/gmaps4rails/gmaps4rails.bing.js?body=1" type="text/javascript"></script> <script src="/assets/gmaps4rails/gmaps4rails.googlemaps.js?body=1" type="text/javascript"></script> <script src="/assets/gmaps4rails/gmaps4rails.mapquest.js?body=1" type="text/javascript"></script> <script src="/assets/gmaps4rails/gmaps4rails.openlayers.js?body=1" type="text/javascript"></script> <script src="/assets/highcharts.js?body=1" type="text/javascript"></script> <script src="/assets/jquery-ui-1.8.18.custom.min.js?body=1" type="text/javascript"></script> <script src="/assets/jquery.alphanumeric.js?body=1" type="text/javascript"></script> <script src="/assets/jquery.formrestrict.js?body=1" type="text/javascript"></script> <script src="/assets/jquery.lazyload.js?body=1" type="text/javascript"></script> <script src="/assets/jquery.ui.addresspicker.js?body=1" type="text/javascript"></script> <script src="/assets/likes.js?body=1" type="text/javascript"></script> <script src="/assets/messages.js?body=1" type="text/javascript"></script> <script src="/assets/overalls.js?body=1" type="text/javascript"></script> <script src="/assets/pages.js?body=1" type="text/javascript"></script> <script src="/assets/questions.js?body=1" type="text/javascript"></script> <script src="/assets/raty.js?body=1" type="text/javascript"></script> <script src="/assets/reviews.js?body=1" type="text/javascript"></script> <script src="/assets/sessions.js?body=1" type="text/javascript"></script> <script src="/assets/styles.js?body=1" type="text/javascript"></script> <script src="/assets/tickets.js?body=1" type="text/javascript"></script> <script src="/assets/universities.js?body=1" type="text/javascript"></script> <script src="/assets/users.js?body=1" type="text/javascript"></script> <script src="/assets/dataTables/jquery.dataTables.js?body=1" type="text/javascript"></script> <script src="/assets/dataTables/jquery.dataTables.bootstrap.js?body=1" type="text/javascript"></script> <script src="/assets/bootstrap-transition.js?body=1" type="text/javascript"></script> <script src="/assets/bootstrap-affix.js?body=1" type="text/javascript"></script> <script src="/assets/bootstrap-button.js?body=1" type="text/javascript"></script> <script src="/assets/bootstrap-carousel.js?body=1" type="text/javascript"></script> <script src="/assets/bootstrap-collapse.js?body=1" type="text/javascript"></script> <script src="/assets/bootstrap-scrollspy.js?body=1" type="text/javascript"></script> <script src="/assets/bootstrap-typeahead.js?body=1" type="text/javascript"></script> <script src="/assets/bootstrap.js?body=1" type="text/javascript"></script> <script src="/assets/select2.js?body=1" type="text/javascript"></script> From application.rb: config.assets.initialize_on_precompile = false # Enable the asset pipeline config.assets.enabled = true config.action_controller.assets_dir = "#{File.dirname(File.dirname(__FILE__))}/public" # Version of your assets, change this if you want to expire all your assets config.assets.version = '1.0' I'm sorry, I'm not sure what else to include to help with this puzzle, but any advise would be appreciated. I was having no problems before I started trying to upload to heroku and now everything's gone haywire. EDIT: In the console at the moment I'm seeing Uncaught TypeError: Cannot read property 'Constructor' of undefined bootstrap-popover.js:33 Uncaught ReferenceError: google is not defined jquery.ui.addresspicker.js:25 Uncaught TypeError: Object [object Object] has no method 'popover' overall:476

Read the article

Need guidance on a Google Map application that has to show 250 000 polylines.

- by lucian.jp

I am looking for advice for an application I am developing that uses Google Map. Summary: A user has a list of criteria for searching a street segment that fulfills the criteria. The street segments will be colored with 3 colors for showing those below average, average and over average. Then the user clicks on the street segment to see an information window showing the properties of that specific segment hiding those not selected until he/she closes the window and other polyline becomes visible again. This looks quite like the Monopoly City Streets game Hasbro made some month ago the difference being I do not use Flash, I can’t use Open Street Map because it doesn’t list street segment (if it does the IDs won’t be the same anyway) and I do not have to show Google sketch building over. Information: I have a database of street segments with IDs, polyline points and centroid. The database has 6,000,000 street segment records in it. To narrow the generated data a bit we focus on city. The largest city we must show has 250,000 street segments. This means 250,000 line segment polyline to show. Our longest polyline uses 9600 characters which is stored in two 8000 varchar columns in SQL Server 2008. We need to use the API v3 because it is faster than the API v2 and the application will be ported to iPhone. For now it's an ASP.NET 3.5 with SQl Server 2008 application. Performance is a priority. Problems: Most of the demo projects that do this are made with API v2. So besides tutorial on the Google API v3 reference page I have nothing to compare performance or technology use to achieve my goal. There is no available .NET wrapper for the API v3 yet. Generating a 250,000 line segment polyline creates a heavy file which takes time to transfer and parse. (I have found a demo of one polyline of 390,000 points. I think the encoder would be far less efficient with more polylines with less points since there will be less rounding.) Since streets segments are shown based on criteria, polylines must be dynamically created and cache can't be used. Some thoughts: KML/KMZ: Pros: Since it is a standard we can easily load Bing maps, Yahoo! maps, Google maps, Google Earth, with the same KML file. The data generation would be the same. Cons: LineString in KML cannot be encoded polyline like the Google map API can handle. So it would probably be bigger and slower to display. Zipping the file at the size it will take more processing time and require the client side to uncompress the data and I am not quite sure with 250,000 data how an iPhone would handle this and how a server would handle 40 users browsing at the same time. JavaScript file: Pros: JavaScript file can have encoded polyline and would significantly reduce the file to transfer. Cons: Have to create my own stripped version of API v3 to add overlays, create polyline, etc. It is more complex than just create a KML file and point to the source. GeoRSS: This option isn't adapted for my needs I think, but I could be wrong. MapServer: I saw some post suggesting using MapServer to generate overlays. Not quite sure for the connection with our database and the performance it would give. Plus it requires a plugin for generating KML. It seems to me that it wouldn't allow me to do better than creating my own KML or JavaScript file. Maintenance would be simpler without. Monopoly City Streets: The game is now over, but for those who know what I am talking about Monopoly City Streets was showing at max zoom level only the streets that the centroid was inside the Bounds of the window. Moving the map was sending request to the server for the new streets to show. While I think this was ingenious, I have no idea how to implement something similar. The only thing I thought about was to compare if the long was inside the bound of map area X and same with Y. While this could improve performance significantly at high zoom level, this would give nothing when showing a whole city. Clustering: While cluster is awesome for marker, it seems we cannot cluster polylines. I would have liked something like MarkerClusterer for polylines and be able to cluster by my 3 polyline colors. This will probably stay as a “would have been freaking awesome but forget it”. Arrow: I will have in a future version to show a direction for the polyline and will have to show an arrow at the centroid. Loading an image or marker will only double my data so creating a custom overlay will probably be my only option. I have found that demo for something similar I would like to achieve. Unfortunately, the demo is very slow, but I only wish to show 1 arrow per polyline and not multiple like the demo. This functionality will depend on the format of data since I don't think KML support custom overlays. Criteria: While the application is done with ASP.NET 3.5, the port to the iPhone won't use the web to show the application and be limited in screen size for selecting the criteria. This is why I was more orienting on a service or page generating the file based on criteria passed in parameters. The service would than generate the file I need to display the polylines on the map. I could also create an aspx page that does this. The aspx page is more documented than the service way. There should be a reason. Questions: Should I create a web service to returns the street segments file or create an aspx page that return the file? Should I create a JavaScript file with encoded polyline or a KML with longitude/latitude based on the fact that maximum longitude/latitude polyline have 9600 characters and I have to render maximum 250,000 line segment polyline. Or should I go with a MapServer that generate the overlay? Will I be able to display simple arrow on the polyline on the next version. In case of KML generation is it faster to create the file with XDocument, XmlDocument, XmlWriter and this manually or just serialize the street segment in the stream? This is more a brainstorming Stack Overflow question than an actual code problem. Any answer helping narrow the possibilities is as good as someone having all the knowledge to point me out a better choice.

Read the article

how to display the below xml in the flex datagrid

- by Kishore

<rows> - <row rowno="1"> - <row-value> <colno>1</colno> <value>1</value> </row-value> - <row-value> <colno>2</colno> <value>sel</value> </row-value> - <row-value> <colno>3</colno> <value>select * from qt_query</value> </row-value> - <row-value> <colno>4</colno> <value>kishore</value> </row-value> - <row-value> <colno>5</colno> <value>2009-04-10 00:00:00.0</value> </row-value> - <row-value> <colno>6</colno> </row-value> - <row-value> <colno>7</colno> </row-value> </row> - <row rowno="2"> - <row-value> <colno>1</colno> <value>2</value> </row-value> - <row-value> <colno>2</colno> <value>sel12345</value> </row-value> - <row-value> <colno>3</colno> <value>select * from qt_query</value> </row-value> - <row-value> <colno>4</colno> <value>kishore</value> </row-value> - <row-value> <colno>5</colno> <value>2009-04-10 00:00:00.0</value> </row-value> - <row-value> <colno>6</colno> <value>krajkumard</value> </row-value> - <row-value> <colno>7</colno> <value>2010-05-21 00:00:00.0</value> </row-value> </row> - <row rowno="3"> - <row-value> <colno>1</colno> <value>3</value> </row-value> - <row-value> <colno>2</colno> <value>sele123</value> </row-value> - <row-value> <colno>3</colno> - <value> select * from cache where %2scache_name% = ? and %1icache_type% = ? </value> </row-value> - <row-value> <colno>4</colno> <value>krajkumard</value> </row-value> - <row-value> <colno>5</colno> <value>2010-05-21 00:00:00.0</value> </row-value> - <row-value> <colno>6</colno> </row-value> - <row-value> <colno>7</colno> </row-value> </row> - <row rowno="4"> - <row-value> <colno>1</colno> <value>4</value> </row-value> - <row-value> <colno>2</colno> <value>upd</value> </row-value> - <row-value> <colno>3</colno> <value>select * from qt_qtool where %sname=?%</value> </row-value> - <row-value> <colno>4</colno> <value>krajkumard</value> </row-value> - <row-value> <colno>5</colno> <value>2010-05-31 00:00:00.0</value> </row-value> - <row-value> <colno>6</colno> </row-value> - <row-value> <colno>7</colno> <value>2010-06-07 00:00:00.0</value> </row-value> </row> - <row rowno="5"> - <row-value> <colno>1</colno> <value>5</value> </row-value> - <row-value> <colno>2</colno> <value>Pron_errors</value> </row-value> - <row-value> <colno>3</colno> <value>select * from proanalyzer_errors</value> </row-value> - <row-value> <colno>4</colno> <value>tjothy</value> </row-value> - <row-value> <colno>5</colno> <value>2010-06-07 00:00:00.0</value> </row-value> - <row-value> <colno>6</colno> </row-value> - <row-value> <colno>7</colno> </row-value> </row> </rows>

Read the article

Out of memory exception during scrolling of listview

- by user1761316

I am using facebook data like postedpicture,profile picture,name,message in my listview.I am getting an OOM error while doing fast scrolling of listview. I am also having scrollviewlistener in my application that loads more data when the scrollbar reaches the bottom of the screen.I just want to know whether I need to change anything in this class. imageLoader.DisplayImage(postimage.get(position).replace(" ", "%20"), postimg) ; I am using the above line to call the method in this imageloader class to set the bitmap to imageview. Here is my imageloader class import java.io.File; import java.io.FileInputStream; import java.io.FileNotFoundException; import java.io.FileOutputStream; import java.io.InputStream; import java.io.OutputStream; import java.net.HttpURLConnection; import java.net.URL; import java.util.Collections; import java.util.Map; import java.util.Stack; import java.util.WeakHashMap; import com.stellent.beerbro.Wall; import android.app.Activity; import android.content.Context; import android.graphics.Bitmap; import android.graphics.Bitmap.Config; import android.graphics.BitmapFactory; import android.graphics.Canvas; import android.graphics.Paint; import android.graphics.PorterDuff.Mode; import android.graphics.PorterDuffXfermode; import android.graphics.Rect; import android.graphics.RectF; import android.graphics.drawable.BitmapDrawable; import android.util.Log; import android.widget.ImageView; public class ImageLoader { MemoryCache memoryCache=new MemoryCache(); FileCache fileCache; private Map<ImageView, String> imageViews=Collections.synchronizedMap(new WeakHashMap<ImageView, String>()); public ImageLoader(Context context){ //Make the background thead low priority. This way it will not affect the UI performance photoLoaderThread.setPriority(Thread.NORM_PRIORITY-1); fileCache=new FileCache(context); } // final int stub_id=R.drawable.stub; public void DisplayImage(String url,ImageView imageView) { imageViews.put(imageView, url); System.gc(); // Bitmap bitmap=createScaledBitmap(memoryCache.get(url), 100,100,0); Bitmap bitmap=memoryCache.get(url); // Bitmap bitmaps=bitmap.createScaledBitmap(bitmap, 0, 100, 100); if(bitmap!=null) { imageView.setBackgroundDrawable(new BitmapDrawable(bitmap)); // imageView.setImageBitmap(getRoundedCornerBitmap( bitmap, 10,70,70)); // imageView.setImageBitmap(bitmap); // Log.v("first", "first"); } else { queuePhoto(url, imageView); // Log.v("second", "second"); } } private Bitmap createScaledBitmap(Bitmap bitmap, int i, int j, int k) { // TODO Auto-generated method stub return null; } private void queuePhoto(String url, ImageView imageView) { //This ImageView may be used for other images before. So there may be some old tasks in the queue. We need to discard them. photosQueue.Clean(imageView); PhotoToLoad p=new PhotoToLoad(url, imageView); synchronized(photosQueue.photosToLoad){ photosQueue.photosToLoad.push(p); photosQueue.photosToLoad.notifyAll(); } //start thread if it's not started yet if(photoLoaderThread.getState()==Thread.State.NEW) photoLoaderThread.start(); } public Bitmap getBitmap(String url) { File f=fileCache.getFile(url); //from SD cache Bitmap b = decodeFile(f); if(b!=null) return b; //from web try { Bitmap bitmap=null; URL imageUrl = new URL(url); HttpURLConnection conn = (HttpURLConnection)imageUrl.openConnection(); conn.setConnectTimeout(30000); conn.setReadTimeout(30000); InputStream is=conn.getInputStream(); OutputStream os = new FileOutputStream(f); Utils.CopyStream(is, os); os.close(); bitmap = decodeFile(f); return bitmap; } catch (Exception ex){ ex.printStackTrace(); return null; } }//Lalit //decodes image and scales it to reduce memory consumption private Bitmap decodeFile(File f){ try { //decode image size BitmapFactory.Options o = new BitmapFactory.Options(); o.inJustDecodeBounds = true; BitmapFactory.decodeStream(new FileInputStream(f),null,o); //Find the correct scale value. It should be the power of 2. final int REQUIRED_SIZE=Wall.width; final int REQUIRED_SIZE1=Wall.height; // final int REQUIRED_SIZE=250; // int width_tmp=o.outWidth, height_tmp=o.outHeight; int scale=1; // while(o.outWidth/scale/2>=REQUIRED_SIZE && o.outHeight/scale/2>=REQUIRED_SIZE) //// scale*=2; while(true){ if(width_tmp/2<REQUIRED_SIZE && height_tmp/2<REQUIRED_SIZE1) break; width_tmp/=2; height_tmp/=2; scale*=2; } //decode with inSampleSize BitmapFactory.Options o2 = new BitmapFactory.Options(); o2.inSampleSize=scale; // o2.inSampleSize=2; return BitmapFactory.decodeStream(new FileInputStream(f), null, o2); } catch (FileNotFoundException e) {} return null; } //Task for the queue private class PhotoToLoad { public String url; public ImageView imageView; public PhotoToLoad(String u, ImageView i){ url=u; imageView=i; } } PhotosQueue photosQueue=new PhotosQueue(); public void stopThread() { photoLoaderThread.interrupt(); } //stores list of photos to download class PhotosQueue { private Stack<PhotoToLoad> photosToLoad=new Stack<PhotoToLoad>(); //removes all instances of this ImageView public void Clean(ImageView image) { for(int j=0 ;j<photosToLoad.size();){ if(photosToLoad.get(j).imageView==image) photosToLoad.remove(j); else ++j; } } } class PhotosLoader extends Thread { public void run() { try { while(true) { //thread waits until there are any images to load in the queue if(photosQueue.photosToLoad.size()==0) synchronized(photosQueue.photosToLoad){ photosQueue.photosToLoad.wait(); } if(photosQueue.photosToLoad.size()!=0) { PhotoToLoad photoToLoad; synchronized(photosQueue.photosToLoad){ photoToLoad=photosQueue.photosToLoad.pop(); } Bitmap bmp=getBitmap(photoToLoad.url); memoryCache.put(photoToLoad.url, bmp); String tag=imageViews.get(photoToLoad.imageView); if(tag!=null && tag.equals(photoToLoad.url)){ BitmapDisplayer bd=new BitmapDisplayer(bmp, photoToLoad.imageView); Activity a=(Activity)photoToLoad.imageView.getContext(); a.runOnUiThread(bd); } } if(Thread.interrupted()) break; } } catch (InterruptedException e) { //allow thread to exit } } } PhotosLoader photoLoaderThread=new PhotosLoader(); //Used to display bitmap in the UI thread class BitmapDisplayer implements Runnable { Bitmap bitmap; ImageView imageView; public BitmapDisplayer(Bitmap b, ImageView i){bitmap=b;imageView=i;} public void run() { if(bitmap!=null) imageView.setBackgroundDrawable(new BitmapDrawable(bitmap)); } } public void clearCache() { memoryCache.clear(); fileCache.clear(); } public static Bitmap getRoundedCornerBitmap(Bitmap bitmap, int pixels,int width,int height) { Bitmap output = Bitmap.createBitmap(width,height, Config.ARGB_8888); Canvas canvas = new Canvas(output); final int color = 0xff424242; final Paint paint = new Paint(); final Rect rect = new Rect(0, 0, bitmap.getWidth(), bitmap.getHeight()); final RectF rectF = new RectF(rect); final float roundPx = pixels; paint.setAntiAlias(true); canvas.drawARGB(0, 0, 0, 0); paint.setColor(color); canvas.drawRoundRect(rectF, roundPx, roundPx, paint); paint.setXfermode(new PorterDuffXfermode(Mode.SRC_IN)); canvas.drawBitmap(bitmap, rect, rect, paint); return output; } }

Read the article

Need help debugging a very basic PHP SOAP Hello world app

- by WarDoGG

I have been breaking my head at this, reading almost every article and tutorial there is on the web, but nothing doing.. i still cannot get my first web service application to work. I would really appreciate it if anyone could debug this code for me and provide me with a good explanation as to what is wrong and why. This will help indeed ! Thanks ! I have pasted below the entire codes that i am using making it easier to debug. I'm using the PHP5 SOAP extension. Here is my WSDL: <?xml version="1.0" encoding="utf-8"?> <wsdl:definitions name="testWebservice" xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/" xmlns:tm="http://microsoft.com/wsdl/mime/textMatching/" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:mime="http://schemas.xmlsoap.org/wsdl/mime/" xmlns:tns="http://tempuri.org/" xmlns:s1="http://microsoft.com/wsdl/types/" xmlns:s="http://www.w3.org/2001/XMLSchema" xmlns:soap12="http://schemas.xmlsoap.org/wsdl/soap12/" xmlns:http="http://schemas.xmlsoap.org/wsdl/http/" targetNamespace="http://tempuri.org/" xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/"> <wsdl:types> <s:schema elementFormDefault="qualified" targetNamespace="http://tempuri.org/"> <s:import namespace="http://microsoft.com/wsdl/types/" /> <s:element name="getUser"> <s:complexType> <s:sequence> <s:element minOccurs="0" maxOccurs="1" name="username" type="s:string" /> <s:element minOccurs="0" maxOccurs="1" name="password" type="s:string" /> </s:sequence> </s:complexType> </s:element> <s:element name="getUserResponse"> <s:complexType> <s:sequence> <s:element minOccurs="0" maxOccurs="1" name="getUserResult" type="tns:userInfo" /> </s:sequence> </s:complexType> </s:element> <s:complexType name="userInfo"> <s:sequence> <s:element minOccurs="1" maxOccurs="1" name="ID" type="s:int" /> <s:element minOccurs="1" maxOccurs="1" name="authkey" type="s:int" /> </s:sequence> </s:complexType> </s:schema> </wsdl:types> <wsdl:message name="getUserSoapIn"> <wsdl:part name="parameters" element="tns:getUser" /> </wsdl:message> <wsdl:message name="getUserSoapOut"> <wsdl:part name="parameters" element="tns:getUserResponse" /> </wsdl:message> <wsdl:portType name="testWebservice"> <wsdl:operation name="getUser"> <wsdl:input message="tns:getUserSoapIn" /> <wsdl:output message="tns:getUserSoapOut" /> </wsdl:operation> </wsdl:portType> <wsdl:binding name="testWebserviceBinding" type="tns:testWebservice"> <soap:binding style="document" transport="http://schemas.xmlsoap.org/soap/http" /> <wsdl:operation name="getUser"> <soap:operation soapAction="http://tempuri.org/getUser" /> <wsdl:input> <soap:body use="literal" /> </wsdl:input> <wsdl:output> <soap:body use="literal" /> </wsdl:output> </wsdl:operation> </wsdl:binding> <wsdl:service name="testWebserviceService"> <wsdl:port name="testWebservicePort" binding="tns:testWebserviceBinding"> <soap:address location="http://127.0.0.1/nusoap/storytruck/index.php" /> </wsdl:port> </wsdl:service> </wsdl:definitions> and here is the PHP Code i use to setup the server: <?php function getUser($user,$pass) { return array('ID'=>1); } ini_set("soap.wsdl_cache_enabled", "0"); // disabling WSDL cache $server = new SoapServer("http://127.0.0.1/mywsdl.wsdl"); $server->addFunction('getUser'); $server->handle(); ?> and the code for the client: <?php $client = new SoapClient("http://127.0.0.1/index.php?wsdl", array('exceptions' => 0)); try { $result = $client->getUser("username","pass"); print_r($result); } catch (SoapFault $result) { print_r($result); } ?> Here is the ERROR output i am getting on the browser : SoapFault Object ( [message:protected] => Error cannot find parameter [string:Exception:private] => [code:protected] => 0 [file:protected] => C:\xampp\htdocs\client.php [line:protected] => 6 [trace:Exception:private] => Array ( [0] => Array ( [function] => __call [class] => SoapClient [type] => -> [args] => Array ( [0] => getUser [1] => Array ( [0] => username [1] => pass ) ) ) [1] => Array ( [file] => C:\xampp\htdocs\client.php [line] => 6 [function] => getUser [class] => SoapClient [type] => -> [args] => Array ( [0] => username [1] => pass ) ) ) [previous:Exception:private] => [faultstring] => Error cannot find parameter [faultcode] => SOAP-ENV:Client )

Read the article

Authoritative sources about Database vs. Flatfile decision

- by FastAl

<tldr>looking for a reference to a book or other undeniably authoritative source that gives reasons when you should choose a database vs. when you should choose other storage methods. I have provided an un-authoritative list of reasons about 2/3 of the way down this post.</tldr> I have a situation at my company where a database is being used where it would be better to use another solution (in this case, an auto-generated piece of source code that contains a static lookup table, searched by binary sort). Normally, a database would be an OK solution even though the problem does not require a database, e.g, none of the elements of ACID are needed, as it is read-only data, updated about every 3-5 years (also requiring other sourcecode changes), and fits in memory, and can be keyed into via binary search (a tad faster than db, but speed is not an issue). The problem is that this code runs on our enterprise server, but is shared with several PC platforms (some disconnected, some use a central DB, etc.), and parts of it are managed by multiple programming units, parts by the DBAs, parts even by mathematicians in another department, etc. These hit their own platform’s version of their databases (containing their own copy of the static data). What happens is that every implementation, every little change, something different goes wrong. There are many other issues as well. I can’t even use a flatfile, because one mode of running on our enterprise server does not have permission to read files (only databases, and of course, its own literal storage, e.g., in-source table). Of course, other parts of the system use databases in proper, less obscure manners; there is no problem with those parts. So why don’t we just change it? I don’t have administrative ability to force a change. But I’m affected because sometimes I have to help fix the problems, but mostly because it causes outages and tons of extra IT time by other programmers and d*mmit that makes me mad! The reason neither management, nor the designers of the system, can see the problem is that they propose a solution that won’t work: increase communication; implement more safeguards and standards; etc. But every time, in a different part of the already-pared-down but still multi-step processes, a few different diligent, hard-working, top performing IT personnel make a unique subtle error that causes it to fail, sometimes after the last round of testing! And in general these are not single-person failures, but understandable miscommunications. And communication at our company is actually better than most. People just don't think that's the case because they haven't dug into the matter. However, I have it on very good word from somebody with extensive formal study of sociology and psychology that the relatively small amount of less-than-proper database usage in this gigantic cross-platform multi-source, multi-language project is bureaucratically un-maintainable. Impossible. No chance. At least with Human Beings in the loop, and it can’t be automated. In addition, the management and developers who could change this, though intelligent and capable, don’t understand the rigidity of this ‘how humans are’ issue, and are not convincible on the matter. The reason putting the static data in sourcecode will solve the problem is, although the solution is less sexy than a database, it would function with no technical drawbacks; and since the sharing of sourcecode already works very well, you basically erase any database-related effort from this section of the project, along with all the drawbacks of it that are causing problems. OK, that’s the background, for the curious. I won’t be able to convince management that this is an unfixable sociological problem, and that the real solution is coding around these limits of human nature, just as you would code around a bug in a 3rd party component that you can’t change. So what I have to do is exploit the unsuitableness of the database solution, and not do it using logic, but rather authority. I am aware of many reasons, and posts on this site giving reasons for one over the other; I’m not looking for lists of reasons like these (although you can add a comment if I've miss a doozy): WHY USE A DATABASE? instead of flatfile/other DB vs. file: if you need... Random Read / Transparent search optimization Advanced / varied / customizable Searching and sorting capabilities Transaction/rollback Locks, semaphores Concurrency control / Shared users Security 1-many/m-m is easier Easy modification Scalability Load Balancing Random updates / inserts / deletes Advanced query Administrative control of design, etc. SQL / learning curve Debugging / Logging Centralized / Live Backup capabilities Cached queries / dvlp & cache execution plans Interleaved update/read Referential integrity, avoid redundant/missing/corrupt/out-of-sync data Reporting (from on olap or oltp db) / turnkey generation tools [Disadvantages:] Important to get right the first time - professional design - but only b/c it's meant to last s/w & h/w cost Usu. over a network, speed issue (best vs. best design vs. local=even then a separate process req's marshalling/netwk layers/inter-p comm) indicies and query processing can stand in the way of simple processing (vs. flatfile) WHY USE FLATFILE: If you only need... Sequential Row processing only Limited usage append only (no reading, no master key/update) Only Update the record you're reading (fixed length recs only) Too big to fit into memory If Local disk / read-ahead network connection Portability / small system Email / cut & Paste / store as document by novice - simple format Low design learning curve but high cost later WHY USE IN-MEMORY/TABLE (tables, arrays, etc.): if you need... Processing a single db/ff record that was imported Known size of data Static data if hardcoding the table Narrow, unchanging use (e.g., one program or proc) -includes a class that will be shared, but encapsulates its data manipulation Extreme speed needed / high transaction frequency Random access - but search is dependent on implementation Following are some other posts about the topic: http://stackoverflow.com/questions/1499239/database-vs-flat-text-file-what-are-some-technical-reasons-for-choosing-one-over http://stackoverflow.com/questions/332825/are-flat-file-databases-any-good http://stackoverflow.com/questions/2356851/database-vs-flat-files http://stackoverflow.com/questions/514455/databases-vs-plain-text/514530 What I’d like to know is if anybody could recommend a hard, authoritative source containing these reasons. I’m looking for a paper book I can buy, or a reputable website with whitepapers about the issue (e.g., Microsoft, IBM), not counting the user-generated content on those sites. This will have a greater change to elicit a change that I’m looking for: less wasted programmer time, and more reliable programs. Thanks very much for your help. You win a prize for reading such a large post!

Read the article

How do I prove I should put a table of values in source code instead of a database table?

- by FastAl

<tldr>looking for a reference to a book or other undeniably authoritative source that gives reasons when you should choose a database vs. when you should choose other storage methods. I have provided an un-authoritative list of reasons about 2/3 of the way down this post.</tldr> I have a situation at my company where a database is being used where it would be better to use another solution (in this case, an auto-generated piece of source code that contains a static lookup table, searched by binary sort). Normally, a database would be an OK solution even though the problem does not require a database, e.g, none of the elements of ACID are needed, as it is read-only data, updated about every 3-5 years (also requiring other sourcecode changes), and fits in memory, and can be keyed into via binary search (a tad faster than db, but speed is not an issue). The problem is that this code runs on our enterprise server, but is shared with several PC platforms (some disconnected, some use a central DB, etc.), and parts of it are managed by multiple programming units, parts by the DBAs, parts even by mathematicians in another department, etc. These hit their own platform’s version of their databases (containing their own copy of the static data). What happens is that every implementation, every little change, something different goes wrong. There are many other issues as well. I can’t even use a flatfile, because one mode of running on our enterprise server does not have permission to read files (only databases, and of course, its own literal storage, e.g., in-source table). Of course, other parts of the system use databases in proper, less obscure manners; there is no problem with those parts. So why don’t we just change it? I don’t have administrative ability to force a change. But I’m affected because sometimes I have to help fix the problems, but mostly because it causes outages and tons of extra IT time by other programmers and d*mmit that makes me mad! The reason neither management, nor the designers of the system, can see the problem is that they propose a solution that won’t work: increase communication; implement more safeguards and standards; etc. But every time, in a different part of the already-pared-down but still multi-step processes, a few different diligent, hard-working, top performing IT personnel make a unique subtle error that causes it to fail, sometimes after the last round of testing! And in general these are not single-person failures, but understandable miscommunications. And communication at our company is actually better than most. People just don't think that's the case because they haven't dug into the matter. However, I have it on very good word from somebody with extensive formal study of sociology and psychology that the relatively small amount of less-than-proper database usage in this gigantic cross-platform multi-source, multi-language project is bureaucratically un-maintainable. Impossible. No chance. At least with Human Beings in the loop, and it can’t be automated. In addition, the management and developers who could change this, though intelligent and capable, don’t understand the rigidity of this ‘how humans are’ issue, and are not convincible on the matter. The reason putting the static data in sourcecode will solve the problem is, although the solution is less sexy than a database, it would function with no technical drawbacks; and since the sharing of sourcecode already works very well, you basically erase any database-related effort from this section of the project, along with all the drawbacks of it that are causing problems. OK, that’s the background, for the curious. I won’t be able to convince management that this is an unfixable sociological problem, and that the real solution is coding around these limits of human nature, just as you would code around a bug in a 3rd party component that you can’t change. So what I have to do is exploit the unsuitableness of the database solution, and not do it using logic, but rather authority. I am aware of many reasons, and posts on this site giving reasons for one over the other; I’m not looking for lists of reasons like these (although you can add a comment if I've miss a doozy): WHY USE A DATABASE? instead of flatfile/other DB vs. file: if you need... Random Read / Transparent search optimization Advanced / varied / customizable Searching and sorting capabilities Transaction/rollback Locks, semaphores Concurrency control / Shared users Security 1-many/m-m is easier Easy modification Scalability Load Balancing Random updates / inserts / deletes Advanced query Administrative control of design, etc. SQL / learning curve Debugging / Logging Centralized / Live Backup capabilities Cached queries / dvlp & cache execution plans Interleaved update/read Referential integrity, avoid redundant/missing/corrupt/out-of-sync data Reporting (from on olap or oltp db) / turnkey generation tools [Disadvantages:] Important to get right the first time - professional design - but only b/c it's meant to last s/w & h/w cost Usu. over a network, speed issue (best vs. best design vs. local=even then a separate process req's marshalling/netwk layers/inter-p comm) indicies and query processing can stand in the way of simple processing (vs. flatfile) WHY USE FLATFILE: If you only need... Sequential Row processing only Limited usage append only (no reading, no master key/update) Only Update the record you're reading (fixed length recs only) Too big to fit into memory If Local disk / read-ahead network connection Portability / small system Email / cut & Paste / store as document by novice - simple format Low design learning curve but high cost later WHY USE IN-MEMORY/TABLE (tables, arrays, etc.): if you need... Processing a single db/ff record that was imported Known size of data Static data if hardcoding the table Narrow, unchanging use (e.g., one program or proc) -includes a class that will be shared, but encapsulates its data manipulation Extreme speed needed / high transaction frequency Random access - but search is dependent on implementation Following are some other posts about the topic: http://stackoverflow.com/questions/1499239/database-vs-flat-text-file-what-are-some-technical-reasons-for-choosing-one-over http://stackoverflow.com/questions/332825/are-flat-file-databases-any-good http://stackoverflow.com/questions/2356851/database-vs-flat-files http://stackoverflow.com/questions/514455/databases-vs-plain-text/514530 What I’d like to know is if anybody could recommend a hard, authoritative source containing these reasons. I’m looking for a paper book I can buy, or a reputable website with whitepapers about the issue (e.g., Microsoft, IBM), not counting the user-generated content on those sites. This will have a greater change to elicit a change that I’m looking for: less wasted programmer time, and more reliable programs. Thanks very much for your help. You win a prize for reading such a large post!

Read the article

Display database content which is last inserted using last inserted ID

- by user2330772

What i am doing is i am displaying last inserted data when form data is submitted,the form is multipart/form-data. I am getting this form data using jquery,here i am sending this data to php file using Ajax POST.In that php file i am inserting that data in db table..where i am getting the id of inserted data..on success of Ajax call i am sending that id to another PHP file..where using that id i am displaying the last inserted data... My form is: <form method="post" enctype="multipart/form-data" name="upload_form" id="data"> <select id="sel"> <option>Select the Project Stream</option> <option value="1">Computer Science</option> <option value="2">Mechanical</option> <option value="3">IT</option> <option value="4">Web Development</option> <option value="5">MCA</option> <option value="6">Civil</option> </select><br /> <input type="text" id="title" placeholder="Project Title"/><br /> <input type="text" id="vurl" placeholder="If You have any video about project write your video url path here" style="width:435px;"/><br /> <textarea id="prjdesc" name="prjdesc" rows="20" cols="80" style="border-style:groove;box-shadow: 10px 10px 10px 10px #888888;"placeholder="Please describe Your Project"></textarea> <label for="file">Filename:</label> <input type="file" name="file" id="file"/><br /> <button>Submit</button> </form> My js file: $("form#data").submit(function() { alert("update"); var sid=$("#sel").val(); alert(sid); var ttle = $("#title").val(); alert(ttle); var text = $("#prjdesc").val(); var vurl = $("#vurl").val(); /*var dataString = 'param='+text+'&param1='+vurl+'&param2='+ttle+'&param3='+id;*/ var formData = new FormData($(this)[0]); formData.append('param',text); formData.append('param1',vurl); formData.append('param2',ttle ); formData.append('param3',sid ); $.ajax({ type:'POST', data:formData, url:'insert.php', success:function(id) { alert(id); window.location ="another.php?id="+id; }, cache: false, contentType: false, processData: false }); return false; }); insert.php: <?php print_r($_FILES); $desc = $_POST['param']; echo $desc; $video = $_POST['param1']; echo $video ; $title = $_POST['param2']; echo $title; $tech_id=$_POST['param3']; echo $tech_id; $host="localhost"; $username="root"; $password=""; $db_name="geny"; $tbl_name="project_details"; mysql_connect("$host", "$username", "$password")or die("cannot connect"); mysql_select_db("$db_name")or die("cannot select DB"); $allowedExts = array("gif", "jpeg", "jpg", "png"); $extension = end(explode(".", $_FILES["file"]["name"])); $url_dir = "C:/wamp/www/WebsiteTemplate4/upload/"; if ((($_FILES["file"]["type"] == "image/gif") || ($_FILES["file"]["type"] == "image/jpeg") || ($_FILES["file"]["type"] == "image/jpg") || ($_FILES["file"]["type"] == "image/pjpeg") || ($_FILES["file"]["type"] == "image/x-png") || ($_FILES["file"]["type"] == "image/png")) && ($_FILES["file"]["size"] < 50000) && in_array($extension, $allowedExts)) { if ($_FILES["file"]["error"] > 0) { echo "Return Code: " . $_FILES["file"]["error"] . "<br>"; } else { if (file_exists($url_dir . $_FILES["file"]["name"])) { echo $_FILES["file"]["name"] . " already exists. "; } else { move_uploaded_file($_FILES["file"]["tmp_name"],$url_dir. $_FILES["file"]["name"]); // echo "Stored in: " . "upload/" . $_FILES["file"]["name"]; $tmp = "C:/wamp/www/WebsiteTemplate4/upload/" . $_FILES["file"]["name"]; $sql="INSERT INTO $tbl_name (title, content, img_path, video_url, project_tech_Id) VALUES ('$title','$desc','$tmp','$video','$tech_id')"; if(mysql_query($sql)) { echo mysql_insert_id(); } else { echo "Cannot Insert"; } } } } else { echo "Invalid file"; } ?> another.php: <?php $temp=$_GET['id']; echo $temp; $host="localhost"; $username="root"; $password=""; $db_name="geny"; $tbl_name="project_details"; mysql_connect("$host", "$username", "$password")or die("cannot connect"); mysql_select_db("$db_name")or die("cannot select DB"); $query = mysql_query("SELECT content FROM project_details WHERE id=". $temp); if (!$query) { echo 'Could not run query: ' . mysql_error(); exit; } $row = mysql_fetch_row($query); echo "<div id='uprjct' style='background:#336699;'> <p>$row[0]</p> </div>"; ?> but the ID what i am returning in insert.php containg array of elements...i dont want all these thing i want only ID(which is number).. please tell me what is wrong in my code...

Read the article

F# - Facebook Hacker Cup - Double Squares

- by Jacob

I'm working on strengthening my F#-fu and decided to tackle the Facebook Hacker Cup Double Squares problem. I'm having some problems with the run-time and was wondering if anyone could help me figure out why it is so much slower than my C# equivalent. There's a good description from another post; Source: Facebook Hacker Cup Qualification Round 2011 A double-square number is an integer X which can be expressed as the sum of two perfect squares. For example, 10 is a double-square because 10 = 3^2 + 1^2. Given X, how can we determine the number of ways in which it can be written as the sum of two squares? For example, 10 can only be written as 3^2 + 1^2 (we don't count 1^2 + 3^2 as being different). On the other hand, 25 can be written as 5^2 + 0^2 or as 4^2 + 3^2. You need to solve this problem for 0 = X = 2,147,483,647. Examples: 10 = 1 25 = 2 3 = 0 0 = 1 1 = 1 My basic strategy (which I'm open to critique on) is to; Create a dictionary (for memoize) of the input numbers initialzed to 0 Get the largest number (LN) and pass it to count/memo function Get the LN square root as int Calculate squares for all numbers 0 to LN and store in dict Sum squares for non repeat combinations of numbers from 0 to LN If sum is in memo dict, add 1 to memo Finally, output the counts of the original numbers. Here is the F# code (See code changes at bottom) I've written that I believe corresponds to this strategy (Runtime: ~8:10); open System open System.Collections.Generic open System.IO /// Get a sequence of values let rec range min max = seq { for num in [min .. max] do yield num } /// Get a sequence starting from 0 and going to max let rec zeroRange max = range 0 max /// Find the maximum number in a list with a starting accumulator (acc) let rec maxNum acc = function | [] -> acc | p::tail when p > acc -> maxNum p tail | p::tail -> maxNum acc tail /// A helper for finding max that sets the accumulator to 0 let rec findMax nums = maxNum 0 nums /// Build a collection of combinations; ie [1,2,3] = (1,1), (1,2), (1,3), (2,2), (2,3), (3,3) let rec combos range = seq { let count = ref 0 for inner in range do for outer in Seq.skip !count range do yield (inner, outer) count := !count + 1 } let rec squares nums = let dict = new Dictionary<int, int>() for s in nums do dict.[s] <- (s * s) dict /// Counts the number of possible double squares for a given number and keeps track of other counts that are provided in the memo dict. let rec countDoubleSquares (num: int) (memo: Dictionary<int, int>) = // The highest relevent square is the square root because it squared plus 0 squared is the top most possibility let maxSquare = System.Math.Sqrt((float)num) // Our relevant squares are 0 to the highest possible square; note the cast to int which shouldn't hurt. let relSquares = range 0 ((int)maxSquare) // calculate the squares up front; let calcSquares = squares relSquares // Build up our square combinations; ie [1,2,3] = (1,1), (1,2), (1,3), (2,2), (2,3), (3,3) for (sq1, sq2) in combos relSquares do let v = calcSquares.[sq1] + calcSquares.[sq2] // Memoize our relevant results if memo.ContainsKey(v) then memo.[v] <- memo.[v] + 1 // return our count for the num passed in memo.[num] // Read our numbers from file. //let lines = File.ReadAllLines("test2.txt") //let nums = [ for line in Seq.skip 1 lines -> Int32.Parse(line) ] // Optionally, read them from straight array let nums = [1740798996; 1257431873; 2147483643; 602519112; 858320077; 1048039120; 415485223; 874566596; 1022907856; 65; 421330820; 1041493518; 5; 1328649093; 1941554117; 4225; 2082925; 0; 1; 3] // Initialize our memoize dictionary let memo = new Dictionary<int, int>() for num in nums do memo.[num] <- 0 // Get the largest number in our set, all other numbers will be memoized along the way let maxN = findMax nums // Do the memoize let maxCount = countDoubleSquares maxN memo // Output our results. for num in nums do printfn "%i" memo.[num] // Have a little pause for when we debug let line = Console.Read() And here is my version in C# (Runtime: ~1:40: using System; using System.Collections.Generic; using System.Diagnostics; using System.IO; using System.Linq; using System.Text; namespace FBHack_DoubleSquares { public class TestInput { public int NumCases { get; set; } public List<int> Nums { get; set; } public TestInput() { Nums = new List<int>(); } public int MaxNum() { return Nums.Max(); } } class Program { static void Main(string[] args) { // Read input from file. //TestInput input = ReadTestInput("live.txt"); // As example, load straight. TestInput input = new TestInput { NumCases = 20, Nums = new List<int> { 1740798996, 1257431873, 2147483643, 602519112, 858320077, 1048039120, 415485223, 874566596, 1022907856, 65, 421330820, 1041493518, 5, 1328649093, 1941554117, 4225, 2082925, 0, 1, 3, } }; var maxNum = input.MaxNum(); Dictionary<int, int> memo = new Dictionary<int, int>(); foreach (var num in input.Nums) { if (!memo.ContainsKey(num)) memo.Add(num, 0); } DoMemoize(maxNum, memo); StringBuilder sb = new StringBuilder(); foreach (var num in input.Nums) { //Console.WriteLine(memo[num]); sb.AppendLine(memo[num].ToString()); } Console.Write(sb.ToString()); var blah = Console.Read(); //File.WriteAllText("out.txt", sb.ToString()); } private static int DoMemoize(int num, Dictionary<int, int> memo) { var highSquare = (int)Math.Floor(Math.Sqrt(num)); var squares = CreateSquareLookup(highSquare); var relSquares = squares.Keys.ToList(); Debug.WriteLine("Starting - " + num.ToString()); Debug.WriteLine("RelSquares.Count = {0}", relSquares.Count); int sum = 0; var index = 0; foreach (var square in relSquares) { foreach (var inner in relSquares.Skip(index)) { sum = squares[square] + squares[inner]; if (memo.ContainsKey(sum)) memo[sum]++; } index++; } if (memo.ContainsKey(num)) return memo[num]; return 0; } private static TestInput ReadTestInput(string fileName) { var lines = File.ReadAllLines(fileName); var input = new TestInput(); input.NumCases = int.Parse(lines[0]); foreach (var lin in lines.Skip(1)) { input.Nums.Add(int.Parse(lin)); } return input; } public static Dictionary<int, int> CreateSquareLookup(int maxNum) { var dict = new Dictionary<int, int>(); int square; foreach (var num in Enumerable.Range(0, maxNum)) { square = num * num; dict[num] = square; } return dict; } } } Thanks for taking a look. UPDATE Changing the combos function slightly will result in a pretty big performance boost (from 8 min to 3:45): /// Old and Busted... let rec combosOld range = seq { let rangeCache = Seq.cache range let count = ref 0 for inner in rangeCache do for outer in Seq.skip !count rangeCache do yield (inner, outer) count := !count + 1 } /// The New Hotness... let rec combos maxNum = seq { for i in 0..maxNum do for j in i..maxNum do yield i,j }

Read the article

Azure Diagnostics wrt Custom Logs and honoring scheduledTransferPeriod

- by kjsteuer

I have implemented my own TraceListener similar to http://blogs.technet.com/b/meamcs/archive/2013/05/23/diagnostics-of-cloud-services-custom-trace-listener.aspx . One thing I noticed is that that logs show up immediately in My Azure Table Storage. I wonder if this is expected with Custom Trace Listeners or because I am in a development environment. My diagnosics.wadcfg <?xml version="1.0" encoding="utf-8"?> <DiagnosticMonitorConfiguration configurationChangePollInterval="PT1M""overallQuotaInMB="4096" xmlns="http://schemas.microsoft.com/ServiceHosting/2010/10/DiagnosticsConfiguration"> <DiagnosticInfrastructureLogs scheduledTransferLogLevelFilter="Information" /> <Directories scheduledTransferPeriod="PT1M"> <IISLogs container="wad-iis-logfiles" /> <CrashDumps container="wad-crash-dumps" /> </Directories> <Logs bufferQuotaInMB="0" scheduledTransferPeriod="PT30M" scheduledTransferLogLevelFilter="Information" /> </DiagnosticMonitorConfiguration> I have changed my approach a bit. Now I am defining in the web config of my webrole. I notice when I set autoflush to true in the webconfig, every thing works but scheduledTransferPeriod is not honored because the flush method pushes to the table storage. I would like to have scheduleTransferPeriod trigger the flush or trigger flush after a certain number of log entries like the buffer is full. Then I can also flush on server shutdown. Is there any method or event on the CustomTraceListener where I can listen to the scheduleTransferPeriod? <system.diagnostics>  <trace> <listeners> <add name="TableTraceListener" type="Pos.Services.Implementation.TableTraceListener, Pos.Services.Implementation" /> <remove name="Default" /> </listeners> </trace> </system.diagnostics> I have modified the custom trace listener to the following: namespace Pos.Services.Implementation { class TableTraceListener : TraceListener { #region Fields //connection string for azure storage readonly string _connectionString; //Custom sql storage table for logs. //TODO put in config readonly string _diagnosticsTable; [ThreadStatic] static StringBuilder _messageBuffer; readonly object _initializationSection = new object(); bool _isInitialized; CloudTableClient _tableStorage; readonly object _traceLogAccess = new object(); readonly List<LogEntry> _traceLog = new List<LogEntry>(); #endregion #region Constructors public TableTraceListener() : base("TableTraceListener") { _connectionString = RoleEnvironment.GetConfigurationSettingValue("DiagConnection"); _diagnosticsTable = RoleEnvironment.GetConfigurationSettingValue("DiagTableName"); } #endregion #region Methods /// <summary> /// Flushes the entries to the storage table /// </summary> public override void Flush() { if (!_isInitialized) { lock (_initializationSection) { if (!_isInitialized) { Initialize(); } } } var context = _tableStorage.GetTableServiceContext(); context.MergeOption = MergeOption.AppendOnly; lock (_traceLogAccess) { _traceLog.ForEach(entry => context.AddObject(_diagnosticsTable, entry)); _traceLog.Clear(); } if (context.Entities.Count > 0) { context.BeginSaveChangesWithRetries(SaveChangesOptions.None, (ar) => context.EndSaveChangesWithRetries(ar), null); } } /// <summary> /// Creates the storage table object. This class does not need to be locked because the caller is locked. /// </summary> private void Initialize() { var account = CloudStorageAccount.Parse(_connectionString); _tableStorage = account.CreateCloudTableClient(); _tableStorage.GetTableReference(_diagnosticsTable).CreateIfNotExists(); _isInitialized = true; } public override bool IsThreadSafe { get { return true; } } #region Trace and Write Methods /// <summary> /// Writes the message to a string buffer /// </summary> /// <param name="message">the Message</param> public override void Write(string message) { if (_messageBuffer == null) _messageBuffer = new StringBuilder(); _messageBuffer.Append(message); } /// <summary> /// Writes the message with a line breaker to a string buffer /// </summary> /// <param name="message"></param> public override void WriteLine(string message) { if (_messageBuffer == null) _messageBuffer = new StringBuilder(); _messageBuffer.AppendLine(message); } /// <summary> /// Appends the trace information and message /// </summary> /// <param name="eventCache">the Event Cache</param> /// <param name="source">the Source</param> /// <param name="eventType">the Event Type</param> /// <param name="id">the Id</param> /// <param name="message">the Message</param> public override void TraceEvent(TraceEventCache eventCache, string source, TraceEventType eventType, int id, string message) { base.TraceEvent(eventCache, source, eventType, id, message); AppendEntry(id, eventType, eventCache); } /// <summary> /// Adds the trace information to a collection of LogEntry objects /// </summary> /// <param name="id">the Id</param> /// <param name="eventType">the Event Type</param> /// <param name="eventCache">the EventCache</param> private void AppendEntry(int id, TraceEventType eventType, TraceEventCache eventCache) { if (_messageBuffer == null) _messageBuffer = new StringBuilder(); var message = _messageBuffer.ToString(); _messageBuffer.Length = 0; if (message.EndsWith(Environment.NewLine)) message = message.Substring(0, message.Length - Environment.NewLine.Length); if (message.Length == 0) return; var entry = new LogEntry() { PartitionKey = string.Format("{0:D10}", eventCache.Timestamp >> 30), RowKey = string.Format("{0:D19}", eventCache.Timestamp), EventTickCount = eventCache.Timestamp, Level = (int)eventType, EventId = id, Pid = eventCache.ProcessId, Tid = eventCache.ThreadId, Message = message }; lock (_traceLogAccess) _traceLog.Add(entry); } #endregion #endregion } }

Read the article

interactive login on cgi script

- by raindrop18

I am new for perl-cgi script. and my objective is to create user/pass interactive script to log and get information from multiple device at once. instead of add the user/pass credential on the script itself. since i am new if some one show me how to write the interactive part of the script. thanks much!!! here is the current code usr/local/bin/perl -wl use CGI ':standard'; use Net::Telnet::Cisco; ### set the error fields to nulls $selerror = ""; ### Input from the screen - make sure data was input if (param() and param('Switches') ne "" and (param('Mac') ne "" or param('Interface Description') ne "" or param('VLAN') ne "" )) { ### Put the input devices into an array. @devices = param('Switches'); ### format the header data print header(); print start_html(-title=>"ShowSwitches",-BGCOLOR=>'aqua'); print "\n",h1("<CENTER>Show Switches</CENTER>"); print "\n",hr(),"\n"; ### Go thru the device array. foreach(@devices) { $error_msg = ""; $TAC_login_error = ""; $open_error = ""; $retry_open_error = ""; $prompt_error = ""; $password_error = ""; ### Take input host and use to send to Telnet $host = $_; $session = Net::Telnet::Cisco->new(Errmode => 'return', Timeout => 30); ### Connect to the host $session->open(Host =>"$host", Timeout => 15); $open_error = $session->errmsg; ### Login with TACACS if host can be connected to if ($open_error eq "") { $session->login('USER', 'PASS'); $TAC_login_error = $session->errmsg; ### Login with TACACS failed - try standard login if ($TAC_login_error ne "") { ### Connect to host $session->open(Host =>"$host", Timeout => 15); $retry_open_error = $session->errmsg; ### Wait for password prompt - multiple matches - devices may have different device prompts. if ($retry_open_error eq "") { $session->waitfor(Match => '/Password:.*$/', Match => '/Enter password:.*$/', Timeout => 20); $prompt_error = $session->errmsg; if ($prompt_error eq "") { ### Input password $session->print('getmeout'); $password_error = $session->errmsg; $session->waitfor('/.*>$/'); $password_error = $session->errmsg; } } } } ### No errors, then issue "show commands". if ($open_error eq "" and ($TAC_login_error eq "" or $retry_open_error eq "") and $prompt_error eq "" and $password_error eq "") { ### Show Mac if (param('Mac')) { $cmd = 'sh mac'; @output = $session->cmd("$cmd"); $show_error = ""; $show_error = $session->errmsg; print "\n",h2($host . ' - ' . $cmd); if ($show_error ne "") { $error_msg = 'Error for show mac - ' . $show_error; print b($error_msg),(br); print hr(),"\n"; $error_msg = ''; } else { print pre(@output); print hr(),"\n"; } } ### Show Interface Description if (param('Interface Description')) { $cmd = 'sh interface description'; @output = $session->cmd("$cmd"); $show_error = ""; $show_error = $session->errmsg; print "\n",h2($host . ' - ' . $cmd); if ($show_error ne "") { $error_msg = 'Error for show mac - ' . $show_error; print b($error_msg),(br); print hr(),"\n"; $error_msg = ''; } else { print pre(@output); print hr(),"\n"; } } ### Show VLAN if (param('VLAN')) { $cmd = 'sh vlan'; @output = $session->cmd("$cmd"); $show_error = ""; $show_error = $session->errmsg; print "\n",h2($host . ' - ' . $cmd); if ($show_error ne "") { $error_msg = 'Error for show vlan - ' . $show_error; print b($error_msg),(br); print hr(),"\n"; $error_msg = ''; } else { print pre(@output); print hr(),"\n"; } } } elsif ($TAC_login_error ne "" and $password_error ne "") { $error_msg = "Error - $host " . $TAC_login_error . ' - possible incorrect TACACS or standard password parameters on device.'; } elsif ($open_error ne "") { $error_msg = "Error - $host " . $open_error . ' - cannot connect to host - is it down??'; } elsif ($prompt_error ne "") { $error_msg = "Error - $host " . $prompt_error . ' - password prompt not recognized - invalid TACACS (or user) password.'; } elsif ($password_error ne "") { $error_msg = "Error - $host " . $password_error . ' - possible incorrect user/password parameters on device.'; } if ($error_msg ne "" ) { print b($error_msg),(br); print hr(),"\n"; } print hr(),"\n"; print end_html(),"\n"; } } else { ### No Show command was selected. if (param('Submit') and param('Mac') eq "" and param('Interface Description') eq "" and param('VLAN') eq "" ) { $selerror = 'No Show Displays were selected. Try again please!!'; } elsif ### No switch was selected. (param('Submit') and param('Switches') eq "") { $selerror = 'No devices were selected. Try again please!!'; } ### This formats the initial Show Web page. print header(-Pragma='no-cache'), start_html(-title=>"Show Displays",-BGCOLOR=>'aqua'), h1("<CENTER>Show Switches</CENTER>"),hr(), start_form(), b("Select Show Commands:"), br(), br(), checkbox(-name=>'Mac'), checkbox(-name=>'Interface Description'), checkbox(-name=>'VLAN'), br(),br(),hr(),br(), b("Select One or More Devices:"), br(), br(), scrolling_list (-name => 'Switches', -default=> "NONE", -values => ['cs6a', 'cs7a', 'cs7b', 'cs8b', 'cs9a', 'c9b', 'csa' ], -multiple => 'true', -size => 7, ), p(submit('Submit'),reset('Reset')), b($selerror), end_form(),hr(), end_html(); } #

Read the article

interactive login on perl cgi script- updated question [closed]

- by raindrop18

I am new for perl-cgi script. and my objective is to create user/pass interactive script to log and get information from multiple device at once. instead of add the user/pass credential on the script itself. since i am new if some one show me how to write the interactive part of the script. thanks much!!! here is the current code usr/local/bin/perl -wl use CGI ':standard'; use Net::Telnet::Cisco; # ### set the error fields to nulls $selerror = ""; # ### Input from the screen - make sure data was input if (param() and param('Switches') ne "" and (param('Mac') ne "" or param('Interface Description') ne "" or param('VLAN') ne "" )) { # ### Put the input devices into an array. @devices = param('Switches'); # ### format the header data print header(); print start_html(-title=>"ShowSwitches",-BGCOLOR=>'aqua'); print "\n",h1("<CENTER>Show Switches</CENTER>"); print "\n",hr(),"\n"; # ### Go thru the device array. foreach(@devices) { $error_msg = ""; $TAC_login_error = ""; $open_error = ""; $retry_open_error = ""; $prompt_error = ""; $password_error = ""; # ### Take input host and use to send to Telnet $host = $_; $session = Net::Telnet::Cisco->new(Errmode => 'return', Timeout => 30); # ### Connect to the host $session->open(Host =>"$host", Timeout => 15); $open_error = $session->errmsg; # ### Login with TACACS if host can be connected to if ($open_error eq "") { $session->login('USER', 'PASS'); $TAC_login_error = $session->errmsg; # ### Login with TACACS failed - try standard login if ($TAC_login_error ne "") { # ### Connect to host $session->open(Host =>"$host", Timeout => 15); $retry_open_error = $session->errmsg; # ### Wait for password prompt - multiple matches - devices may have different device prompts. if ($retry_open_error eq "") { $session->waitfor(Match => '/Password:.*$/', Match => '/Enter password:.*$/', Timeout => 20); $prompt_error = $session->errmsg; if ($prompt_error eq "") { # ### Input password $session->print('getmeout'); $password_error = $session->errmsg; $session->waitfor('/.*>$/'); $password_error = $session->errmsg; } } } } # ### No errors, then issue "show commands". if ($open_error eq "" and ($TAC_login_error eq "" or $retry_open_error eq "") and $prompt_error eq "" and $password_error eq "") { # ### Show Mac if (param('Mac')) { $cmd = 'sh mac'; @output = $session->cmd("$cmd"); $show_error = ""; $show_error = $session->errmsg; print "\n",h2($host . ' - ' . $cmd); if ($show_error ne "") { $error_msg = 'Error for show mac - ' . $show_error; print b($error_msg),(br); print hr(),"\n"; $error_msg = ''; } else { print pre(@output); print hr(),"\n"; } } # ### Show Interface Description if (param('Interface Description')) { $cmd = 'sh interface description'; @output = $session->cmd("$cmd"); $show_error = ""; $show_error = $session->errmsg; print "\n",h2($host . ' - ' . $cmd); if ($show_error ne "") { $error_msg = 'Error for show mac - ' . $show_error; print b($error_msg),(br); print hr(),"\n"; $error_msg = ''; } else { print pre(@output); print hr(),"\n"; } } # ### Show VLAN if (param('VLAN')) { $cmd = 'sh vlan'; @output = $session->cmd("$cmd"); $show_error = ""; $show_error = $session->errmsg; print "\n",h2($host . ' - ' . $cmd); if ($show_error ne "") { $error_msg = 'Error for show vlan - ' . $show_error; print b($error_msg),(br); print hr(),"\n"; $error_msg = ''; } else { print pre(@output); print hr(),"\n"; } } } elsif ($TAC_login_error ne "" and $password_error ne "") { $error_msg = "Error - $host " . $TAC_login_error . ' - possible incorrect TACACS or standard password parameters on device.'; } elsif ($open_error ne "") { $error_msg = "Error - $host " . $open_error . ' - cannot connect to host - is it down??'; } elsif ($prompt_error ne "") { $error_msg = "Error - $host " . $prompt_error . ' - password prompt not recognized - invalid TACACS (or user) password.'; } elsif ($password_error ne "") { $error_msg = "Error - $host " . $password_error . ' - possible incorrect user/password parameters on device.'; } if ($error_msg ne "" ) { print b($error_msg),(br); print hr(),"\n"; } print hr(),"\n"; print end_html(),"\n"; } } else { # ### No Show command was selected. if (param('Submit') and param('Mac') eq "" and param('Interface Description') eq "" and param('VLAN') eq "" ) { $selerror = 'No Show Displays were selected. Try again please!!'; } elsif # ### No switch was selected. (param('Submit') and param('Switches') eq "") { $selerror = 'No devices were selected. Try again please!!'; } # ### This formats the initial Show Web page. print header(-Pragma=>'no-cache'), start_html(-title=>"Show Displays",-BGCOLOR=>'aqua'), h1("<CENTER>Show Switches</CENTER>"),hr(), start_form(), b("Select Show Commands:"), br(), br(), checkbox(-name=>'Mac'), checkbox(-name=>'Interface Description'), checkbox(-name=>'VLAN'), br(),br(),hr(),br(), b("Select One or More Devices:"), br(), br(), scrolling_list (-name => 'Switches', -default=> "NONE", -values => ['cs6a', 'cs7a', 'cs7b', 'cs8b', 'cs9a', 'c9b', 'csa' ], -multiple => 'true', -size => 7, ), p(submit('Submit'),reset('Reset')), b($selerror), end_form(),hr(), end_html(); } #

Read the article

Session is working in Localhost Properly but not Online (Cpanel)

- by nando pandi

Hello guys Sorry for my stupid question regarding to my yesterday question its not solved yet even the advice you have given but still not working. i have removed all of spaces but still showing the problem for me. it's working perfect in localhost but not in CPANEL. Here is the errors which give: Warning: session_start() [function.session-start]: Cannot send session cookie - headers already sent by (output started at /home/scalepro/public_html/Admin Panel/Remote Employee/main.php:1) in /home/scalepro/public_html/Admin Panel/Remote Employee/main.php on line 1 Warning: session_start() [function.session-start]: Cannot send session cache limiter - headers already sent (output started at /home/scalepro/public_html/Admin Panel/Remote Employee/main.php:1) in /home/scalepro/public_html/Admin Panel/Remote Employee/main.php on line 1 Warning: Cannot modify header information - headers already sent by (output started at /home/scalepro/public_html/Admin Panel/Remote Employee/main.php:1) in /home/scalepro/public_html/Admin Panel/Remote Employee/main.php on line 13 Warning: Unknown: Your script possibly relies on a session side-effect which existed until PHP 4.2.3. Please be advised that the session extension does not consider global variables as a source of data, unless register_globals is enabled. You can disable this functionality and this warning by setting session.bug_compat_42 or session.bug_compat_warn to off, respectively in Unknown on line 0 ANY ONE PLEASE ??? Here is my code: <?php session_start(); require_once('../../Admin Panel/db.php'); if(isset($_POST['email']) && !empty($_POST['email']) && isset($_POST['password']) && !empty($_POST['password'])) { $email = $_POST['email']; $password = $_POST['password']; $query="SELECT RemoteEmployeeFullName, RemoteEmployeeEmail, RemoteEmployeePassword FROM remoteemployees WHERE RemoteEmployeeEmail='".$email."' AND RemoteEmployeePassword='".$password."'"; $queryrun=$connection->query($query); if($queryrun->num_rows > 0) { $_SESSION['email']=$RemoteEmployeeFullName; header("Location: /home/scalepro/public_html/Admin Panel/Remote Employee/REPLists.php"); } else { echo 'Email: <b>'.$email. '</b> or Password <b>'. $password.'</b> Is Not Typed Correctly Try Again Please!.'; header( "refresh:5;url= /home/scalepro/public_html/spd/myaccount.php" ); } } else { header( "refresh:5;url= /home/scalepro/public_html/spd/myaccount.php" ); } ?> if the condition gets true this will be redirected to a page by the name of REPLists.php here is the page. <?php session_start(); require_once('../../Admin Panel/db.php'); ?> <html> <head> <style> .wrapper { width:1250px; height:auto; border:solid 1px #000; margin:0 auto; padding:5px; border-radius:5px; -webkit-border-radius:5px; -moz-border-radius:5px; -ms-border-radius:5px; } .wrapper .header { width:1250px; height:20px; border-bottom:solid 1px #f0eeee; margin:auto 0; margin-bottom:12px; } .wrapper .header div { text-decoration:none; color:#F60; } .wrapper .header div a { text-decoration:none; color:#F60; } .wrapper .Labelcon { width:1250px; height:29px; border-bottom:solid 1px #ccc; } .wrapper .Labelcon .Label { width:125px; height:20px; float:left; text-align:center; border-left:1px solid #f0eeee; font:Verdana, Geneva, sans-serif; font-size:14.3px; font-weight:bold; } .wrapper .Valuecon { width:1250px; height:29px; border-bottom:solid 1px #ccc; color:#F60; text-decoration:none; } .wrapper .Valuecon .Value { width:125px; height:20px; float:left; text-align:center; border-left:1px solid #f0eeee; font-size:14px; } </style> </head> <body> <div class="wrapper"> <div class="header"> <div style="float:left;"><font color="#000000">Email: </font> <?php if(isset($_SESSION['email'])) { echo $_SESSION['email']; } ?> </div> <div style="float:right;"> <a href="#">My Profile</a> | <a href="logout.php">Logout</a></div> </div> <div class="Labelcon"> <div class="Label">Property ID</div> <div class="Label">Property Type</div> <div class="Label">Property Deal Type</div> <div class="Label">Property Owner</div> <div class="Label">Proposted Price</div> </div> <?php if(!isset($_SESSION['email'])) { header('Location:../../spd/myaccount.php'); } else { $query = "SELECT properties.PropertyID, properties.PropertyType, properties.PropertyDealType, properties.Status, properties.PropostedPrice, remoteemployees.RemoteEmployeeFullName, propertyowners.PropertyOwnerName, propertydealers.PropertyDealerName FROM remoteemployees, propertyowners, propertydealers, properties WHERE properties.PropertyOwnerID=propertyowners.PropertyOwnerID AND properties.PropertyDealerID=propertydealers.PropertyDealerID AND remoteemployees.RemoteEmployeeID=properties.RemoteEmployeeID ORDER BY properties.PropertyID "; $query_run = $connection->query($query); if( $connection->error ) exit( $connection->error ); while($row=$query_run->fetch_assoc()) { ?> <div class="Valuecon"> <div class="Value"><?php echo $row['PropertyID'] ?></div> <div class="Value"><?php echo $row['PropertyType'] ?></div> <div class="Value"><?php echo $row['PropertyDealType']?></div> <div class="Value"><?php echo $row['PropertyOwnerName'] ?></div> <div class="Value"><?php echo $row['PropostedPrice'];?></div> </div> <?php } }?> </div> </body> </html>

Read the article

PHP Pear Installation on CentOS

- by Prabhakar

[root@ip ~]# yum install php-pear* Reducing CentOS-5 Testing to included packages only Finished Setting up Install Process Package 1:php-pear-1.8.1-2.el5.centos.noarch already installed and latest versio n Package php-pear-XML-Util is obsoleted by php-pear, trying to install 1:php-pear -1.8.1-2.el5.centos.noarch instead Package 1:php-pear-1.8.1-2.el5.centos.noarch already installed and latest versio n Package php-pear-DB is obsoleted by php-pear-db, trying to install php-pear-db-1 .7.13-2.el5.rf.noarch instead Resolving Dependencies --> Running transaction check ---> Package php-pear-Auth-RADIUS.noarch 0:1.0.6-1.el5 set to be updated --> Processing Dependency: php-pecl(radius) >= 1.2.5 for package: php-pear-Auth- RADIUS --> Processing Dependency: php-mcrypt for package: php-pear-Auth-RADIUS ---> Package php-pear-Auth-SASL.noarch 0:1.0.4-1.el5 set to be updated ---> Package php-pear-Benchmark.noarch 0:1.2.7-1.el5 set to be updated ---> Package php-pear-CAS.noarch 0:1.1.3-1.el5 set to be updated --> Processing Dependency: php-domxml-php4-php5 for package: php-pear-CAS ---> Package php-pear-Cache-Lite.noarch 0:1.7.5-1.el5 set to be updated ---> Package php-pear-CodeGen.noarch 0:1.0.7-3.el5 set to be updated ---> Package php-pear-CodeGen-PECL.noarch 0:1.1.3-3.el5 set to be updated ---> Package php-pear-Console-CommandLine.noarch 0:1.1.3-3.el5 set to be updated ---> Package php-pear-Console-Getargs.noarch 0:1.3.5-1.el5 set to be updated ---> Package php-pear-Console-ProgressBar.noarch 0:0.5.2-0.2.beta.el5 set to be updated ---> Package php-pear-Console-Table.noarch 0:1.1.1-1.el5 set to be updated ---> Package php-pear-Crypt-Blowfish.noarch 0:1.0.1-1.el5 set to be updated ---> Package php-pear-Crypt-CHAP.noarch 0:1.0.2-1.el5 set to be updated ---> Package php-pear-DB-DataObject.noarch 0:1.8.12-1.el5 set to be updated ---> Package php-pear-DB-DataObject-FormBuilder.noarch 0:1.0.0-1.el5 set to be u pdated ---> Package php-pear-DB-QueryTool.noarch 0:1.1.2-1.el5 set to be updated ---> Package php-pear-Date.noarch 0:1.4.7-2.el5.centos set to be updated ---> Package php-pear-Date-Holidays.noarch 0:0.21.4-1.el5 set to be updated ---> Package php-pear-Date-Holidays-USA.noarch 0:0.1.1-1.el5 set to be updated ---> Package php-pear-Event-Dispatcher.noarch 0:1.1.0-1.el5 set to be updated ---> Package php-pear-File.noarch 0:1.2.2-1.el5.centos set to be updated ---> Package php-pear-File-Find.noarch 0:1.3.0-1.el5 set to be updated ---> Package php-pear-File-Passwd.noarch 0:1.1.7-1.el5 set to be updated ---> Package php-pear-File-SMBPasswd.noarch 0:1.0.3-1.el5 set to be updated ---> Package php-pear-HTML-Common.noarch 0:1.2.5-1.el5 set to be updated ---> Package php-pear-HTML-QuickForm.noarch 0:3.2.12-1.el5 set to be updated ---> Package php-pear-HTML-QuickForm-ElementGrid.noarch 0:0.1.1-1.el5 set to be updated ---> Package php-pear-HTML-QuickForm-advmultiselect.noarch 0:1.4.1-1.el5 set to be updated ---> Package php-pear-HTML-Table.noarch 0:1.7.5-1.el5 set to be updated ---> Package php-pear-HTML-Template-IT.noarch 0:1.3.0-2.el5 set to be updated ---> Package php-pear-HTML_Template_PHPLIB.noarch 0:1.4.0-2.el5 set to be update d ---> Package php-pear-HTTP.noarch 0:1.4.0-7.el5 set to be updated ---> Package php-pear-HTTP-Client.noarch 0:1.1.1-1.el5 set to be updated ---> Package php-pear-HTTP-Request.noarch 0:1.4.4-1.el5 set to be updated ---> Package php-pear-HTTP-Upload.noarch 0:0.9.1-2.el5 set to be updated ---> Package php-pear-Image-Canvas.noarch 0:0.3.1-1.el5 set to be updated ---> Package php-pear-Image-Color.noarch 0:1.0.3-1.el5 set to be updated ---> Package php-pear-Image-Graph.noarch 0:0.8.0-1.el5 set to be updated ---> Package php-pear-Image-GraphViz.noarch 0:1.2.1-4.el5 set to be updated --> Processing Dependency: graphviz for package: php-pear-Image-GraphViz ---> Package php-pear-Log.noarch 0:1.12.7-1.el5 set to be updated ---> Package php-pear-MDB2.noarch 0:2.4.1-2.el5.centos set to be updated ---> Package php-pear-MDB2-Driver-mysql.noarch 0:1.4.1-3.el5.centos set to be up dated ---> Package php-pear-MDB2-Driver-pgsql.noarch 0:1.4.1-1.el5 set to be updated ---> Package php-pear-MDB2-Schema.noarch 0:0.8.0-2.el5 set to be updated ---> Package php-pear-Mail.noarch 0:1.1.14-5.el5.1 set to be updated ---> Package php-pear-Mail-Mime.noarch 0:1.4.0-1.el5.centos set to be updated ---> Package php-pear-Math-Stats.noarch 0:0.9.0-0.1.beta3.el5 set to be updated ---> Package php-pear-Net-Curl.noarch 0:1.2.5-1.el5 set to be updated ---> Package php-pear-Net-DIME.noarch 0:1.0.1-1.el5 set to be updated ---> Package php-pear-Net-FTP.noarch 0:1.3.4-1.el5 set to be updated ---> Package php-pear-Net-POP3.noarch 0:1.3.7-1.el5 set to be updated ---> Package php-pear-Net-Ping.noarch 0:2.4.5-1.el5 set to be updated ---> Package php-pear-Net-SMTP.noarch 0:1.4.4-1.el5 set to be updated ---> Package php-pear-Net-Sieve.noarch 0:1.3.2-1.el5 set to be updated ---> Package php-pear-Net-Socket.noarch 0:1.0.10-1.el5 set to be updated ---> Package php-pear-Net-Traceroute.noarch 0:0.21.3-1.el5 set to be updated ---> Package php-pear-Net-URL.noarch 0:1.0.15-1.el5.centos set to be updated ---> Package php-pear-Net-URL-Mapper.noarch 0:0.9.0-2.el5.1 set to be updated ---> Package php-pear-Net-URL2.noarch 0:0.3.0-1.el5 set to be updated ---> Package php-pear-Net-UserAgent-Detect.noarch 0:2.5.2-1.el5 set to be update d ---> Package php-pear-Numbers-Roman.noarch 0:1.0.2-2.el5 set to be updated ---> Package php-pear-Numbers-Words.noarch 0:0.16.1-1.el5 set to be updated ---> Package php-pear-OLE.noarch 0:1.0.0-0.4.rc1.el5 set to be updated ---> Package php-pear-PHP-CodeSniffer.noarch 0:1.2.2-1.el5 set to be updated ---> Package php-pear-PHP-Compat.noarch 0:1.5.0-1.el5 set to be updated ---> Package php-pear-PHP-CompatInfo.noarch 0:1.4.3-1.el5 set to be updated ---> Package php-pear-PHPUnit.noarch 0:3.3.5-2.el5 set to be updated --> Processing Dependency: php-pecl(Xdebug) >= 2.0.0 for package: php-pear-PHPUn it --> Processing Dependency: php-channel(pear.phpunit.de) for package: php-pear-PH PUnit ---> Package php-pear-Pager.noarch 0:2.4.8-1.el5 set to be updated ---> Package php-pear-Payment-Process.noarch 0:0.6.6-1.el5 set to be updated ---> Package php-pear-Phlickr.noarch 0:0.2.7-2.el5 set to be updated ---> Package php-pear-PhpDocumentor.noarch 0:1.4.3-1.el5 set to be updated --> Processing Dependency: php-Smarty >= 2.6.0 for package: php-pear-PhpDocument or ---> Package php-pear-PhpDocumentor-docs.noarch 0:1.4.3-1.el5 set to be updated ---> Package php-pear-SOAP.noarch 0:0.11.0-2.el5 set to be updated ---> Package php-pear-Spreadsheet-Excel-Writer.noarch 0:0.9.2-2.el5 set to be up dated ---> Package php-pear-Structures-DataGrid.noarch 0:0.8.3-1.el5 set to be updated ---> Package php-pear-Structures-DataGrid-DataSource-Array.noarch 0:0.1.3-1.el5 set to be updated ---> Package php-pear-Structures-DataGrid-DataSource-DataObject.noarch 0:0.1.2-1 .el5 set to be updated ---> Package php-pear-Structures-DataGrid-DataSource-MDB2.noarch 0:0.1.10-1.el5 set to be updated ---> Package php-pear-Structures-DataGrid-DataSource-RSS.noarch 0:0.1.1-1.el5 se t to be updated ---> Package php-pear-Structures-DataGrid-Renderer-Pager.noarch 0:0.1.2-1.el5 se t to be updated ---> Package php-pear-Text-Diff.noarch 0:1.1.0-1.el5 set to be updated ---> Package php-pear-Validate.noarch 0:0.8.3-1.el5 set to be updated ---> Package php-pear-Validate-Finance-CreditCard.noarch 0:0.5.2-1.el5 set to be updated ---> Package php-pear-Var-Dump.noarch 0:1.0.3-2.el5 set to be updated ---> Package php-pear-XML-Beautifier.noarch 0:1.1-3.el5 set to be updated ---> Package php-pear-XML-Parser.noarch 0:1.2.8-1.el5 set to be updated ---> Package php-pear-XML-RSS.noarch 0:1.0.0-1.el5 set to be updated ---> Package php-pear-XML-Serializer.noarch 0:0.20.0-1.el5 set to be updated ---> Package php-pear-date.noarch 0:1.4.6-1.el5.rf set to be updated ---> Package php-pear-db.noarch 0:1.7.13-2.el5.rf set to be updated ---> Package php-pear-excel.noarch 0:0.9.0-1.el5.rf set to be updated ---> Package php-pear-file.noarch 0:1.2.2-1.el5.rf set to be updated ---> Package php-pear-log.noarch 0:1.9.3-1.el5.rf set to be updated ---> Package php-pear-mail_mime.noarch 0:1.3.1-1.el5.rf set to be updated ---> Package php-pear-ole.noarch 0:0.5-2.el5.rf set to be updated --> Running transaction check ---> Package graphviz.i386 0:2.22.0-4.el5.rf set to be updated ---> Package php-Smarty.noarch 0:2.6.26-1.el5 set to be updated ---> Package php-channel-phpunit.noarch 0:1.0-2.el5 set to be updated ---> Package php-domxml-php4-php5.noarch 0:1.21.2-1.el5 set to be updated ---> Package php-mcrypt.i386 0:5.2.9-2.el5.centos.3 set to be updated --> Processing Dependency: php-api = 20041225 for package: php-mcrypt ---> Package php-pecl-radius.i386 0:1.2.5-4.el5 set to be updated --> Processing Dependency: php-api = 20041225 for package: php-pecl-radius ---> Package php-pecl-xdebug.i386 0:2.0.5-1.el5.1 set to be updated --> Processing Dependency: php-api = 20041225 for package: php-pecl-xdebug --> Finished Dependency Resolution php-pecl-xdebug-2.0.5-1.el5.1.i386 from epel has depsolving problems --> Missing Dependency: php-api = 20041225 is needed by package php-pecl-xdebu g-2.0.5-1.el5.1.i386 (epel) php-pecl-radius-1.2.5-4.el5.i386 from epel has depsolving problems --> Missing Dependency: php-api = 20041225 is needed by package php-pecl-radiu s-1.2.5-4.el5.i386 (epel) php-mcrypt-5.2.9-2.el5.centos.3.i386 from c5-testing has depsolving problems --> Missing Dependency: php-api = 20041225 is needed by package php-mcrypt-5.2 .9-2.el5.centos.3.i386 (c5-testing) Error: Missing Dependency: php-api = 20041225 is needed by package php-pecl-radi us-1.2.5-4.el5.i386 (epel) Error: Missing Dependency: php-api = 20041225 is needed by package php-mcrypt-5. 2.9-2.el5.centos.3.i386 (c5-testing) Error: Missing Dependency: php-api = 20041225 is needed by package php-pecl-xdeb ug-2.0.5-1.el5.1.i386 (epel) You could try using --skip-broken to work around the problem You could try running: package-cleanup --problems package-cleanup --dupes rpm -Va --nofiles --nodigest The program package-cleanup is found in the yum-utils package.

Read the article

Using the West Wind Web Toolkit to set up AJAX and REST Services

- by Rick Strahl

I frequently get questions about which option to use for creating AJAX and REST backends for ASP.NET applications. There are many solutions out there to do this actually, but when I have a choice - not surprisingly - I fall back to my own tools in the West Wind West Wind Web Toolkit. I've talked a bunch about the 'in-the-box' solutions in the past so for a change in this post I'll talk about the tools that I use in my own and customer applications to handle AJAX and REST based access to service resources using the West Wind West Wind Web Toolkit. Let me preface this by saying that I like things to be easy. Yes flexible is very important as well but not at the expense of over-complexity. The goal I've had with my tools is make it drop dead easy, with good performance while providing the core features that I'm after, which are: Easy AJAX/JSON Callbacks Ability to return any kind of non JSON content (string, stream, byte[], images) Ability to work with both XML and JSON interchangeably for input/output Access endpoints via POST data, RPC JSON calls, GET QueryString values or Routing interface Easy to use generic JavaScript client to make RPC calls (same syntax, just what you need) Ability to create clean URLS with Routing Ability to use standard ASP.NET HTTP Stack for HTTP semantics It's all about options! In this post I'll demonstrate most of these features (except XML) in a few simple and short samples which you can download. So let's take a look and see how you can build an AJAX callback solution with the West Wind Web Toolkit. Installing the Toolkit Assemblies The easiest and leanest way of using the Toolkit in your Web project is to grab it via NuGet: West Wind Web and AJAX Utilities (Westwind.Web) and drop it into the project by right clicking in your Project and choosing Manage NuGet Packages from anywhere in the Project. When done you end up with your project looking like this: What just happened? Nuget added two assemblies - Westwind.Web and Westwind.Utilities and the client ww.jquery.js library. It also added a couple of references into web.config: The default namespaces so they can be accessed in pages/views and a ScriptCompressionModule that the toolkit optionally uses to compress script resources served from within the assembly (namely ww.jquery.js and optionally jquery.js). Creating a new Service The West Wind Web Toolkit supports several ways of creating and accessing AJAX services, but for this post I'll stick to the lower level approach that works from any plain HTML page or of course MVC, WebForms, WebPages. There's also a WebForms specific control that makes this even easier but I'll leave that for another post. So, to create a new standalone AJAX/REST service we can create a new HttpHandler in the new project either as a pure class based handler or as a generic .ASHX handler. Both work equally well, but generic handlers don't require any web.config configuration so I'll use that here. In the root of the project add a Generic Handler. I'm going to call this one StockService.ashx. Once the handler has been created, edit the code and remove all of the handler body code. Then change the base class to CallbackHandler and add methods that have a [CallbackMethod] attribute. Here's the modified base handler implementation now looks like with an added HelloWorld method: using System; using Westwind.Web; namespace WestWindWebAjax { /// <summary> /// Handler implements CallbackHandler to provide REST/AJAX services /// </summary> public class SampleService : CallbackHandler { [CallbackMethod] public string HelloWorld(string name) { return "Hello " + name + ". Time is: " + DateTime.Now.ToString(); } } } Notice that the class inherits from CallbackHandler and that the HelloWorld service method is marked up with [CallbackMethod]. We're done here. Services Urlbased Syntax Once you compile, the 'service' is live can respond to requests. All CallbackHandlers support input in GET and POST formats, and can return results as JSON or XML. To check our fancy HelloWorld method we can now access the service like this: http://localhost/WestWindWebAjax/StockService.ashx?Method=HelloWorld&name=Rick which produces a default JSON response - in this case a string (wrapped in quotes as it's JSON): (note by default JSON will be downloaded by most browsers not displayed - various options are available to view JSON right in the browser) If I want to return the same data as XML I can tack on a &format=xml at the end of the querystring which produces: <string>Hello Rick. Time is: 11/1/2011 12:11:13 PM</string> Cleaner URLs with Routing Syntax If you want cleaner URLs for each operation you can also configure custom routes on a per URL basis similar to the way that WCF REST does. To do this you need to add a new RouteHandler to your application's startup code in global.asax.cs one for each CallbackHandler based service you create: protected void Application_Start(object sender, EventArgs e) { CallbackHandlerRouteHandler.RegisterRoutes<StockService>(RouteTable.Routes); } With this code in place you can now add RouteUrl properties to any of your service methods. For the HelloWorld method that doesn't make a ton of sense but here is what a routed clean URL might look like in definition: [CallbackMethod(RouteUrl="stocks/HelloWorld/{name}")] public string HelloWorld(string name) { return "Hello " + name + ". Time is: " + DateTime.Now.ToString(); } The same URL I previously used now becomes a bit shorter and more readable with: http://localhost/WestWindWebAjax/HelloWorld/Rick It's an easy way to create cleaner URLs and still get the same functionality. Calling the Service with $.getJSON() Since the result produced is JSON you can now easily consume this data using jQuery's getJSON method. First we need a couple of scripts - jquery.js and ww.jquery.js in the page: <!DOCTYPE html> <html> <head> <link href="Css/Westwind.css" rel="stylesheet" type="text/css" /> <script src="scripts/jquery.min.js" type="text/javascript"></script> <script src="scripts/ww.jquery.min.js" type="text/javascript"></script> </head> <body> Next let's add a small HelloWorld example form (what else) that has a single textbox to type a name, a button and a div tag to receive the result: <fieldset> <legend>Hello World</legend> Please enter a name: <input type="text" name="txtHello" id="txtHello" value="" /> <input type="button" id="btnSayHello" value="Say Hello (POST)" /> <input type="button" id="btnSayHelloGet" value="Say Hello (GET)" /> <div id="divHelloMessage" class="errordisplay" style="display:none;width: 450px;" > </div> </fieldset> Then to call the HelloWorld method a little jQuery is used to hook the document startup and the button click followed by the $.getJSON call to retrieve the data from the server. <script type="text/javascript"> $(document).ready(function () { $("#btnSayHelloGet").click(function () { $.getJSON("SampleService.ashx", { Method: "HelloWorld", name: $("#txtHello").val() }, function (result) { $("#divHelloMessage") .text(result) .fadeIn(1000); }); });</script> .getJSON() expects a full URL to the endpoint of our service, which is the ASHX file. We can either provide a full URL (SampleService.ashx?Method=HelloWorld&name=Rick) or we can just provide the base URL and an object that encodes the query string parameters for us using an object map that has a property that matches each parameter for the server method. We can also use the clean URL routing syntax, but using the object parameter encoding actually is safer as the parameters will get properly encoded by jQuery. The result returned is whatever the result on the server method is - in this case a string. The string is applied to the divHelloMessage element and we're done. Obviously this is a trivial example, but it demonstrates the basics of getting a JSON response back to the browser. AJAX Post Syntax - using ajaxCallMethod() The previous example allows you basic control over the data that you send to the server via querystring parameters. This works OK for simple values like short strings, numbers and boolean values, but doesn't really work if you need to pass something more complex like an object or an array back up to the server. To handle traditional RPC type messaging where the idea is to map server side functions and results to a client side invokation, POST operations can be used. The easiest way to use this functionality is to use ww.jquery.js and the ajaxCallMethod() function. ww.jquery wraps jQuery's AJAX functions and knows implicitly how to call a CallbackServer method with parameters and parse the result. Let's look at another simple example that posts a simple value but returns something more interesting. Let's start with the service method: [CallbackMethod(RouteUrl="stocks/{symbol}")] public StockQuote GetStockQuote(string symbol) { Response.Cache.SetExpires(DateTime.UtcNow.Add(new TimeSpan(0, 2, 0))); StockServer server = new StockServer(); var quote = server.GetStockQuote(symbol); if (quote == null) throw new ApplicationException("Invalid Symbol passed."); return quote; } This sample utilizes a small StockServer helper class (included in the sample) that downloads a stock quote from Yahoo's financial site via plain HTTP GET requests and formats it into a StockQuote object. Lets create a small HTML block that lets us query for the quote and display it: <fieldset> <legend>Single Stock Quote</legend> Please enter a stock symbol: <input type="text" name="txtSymbol" id="txtSymbol" value="msft" /> <input type="button" id="btnStockQuote" value="Get Quote" /> <div id="divStockDisplay" class="errordisplay" style="display:none; width: 450px;"> <div class="label-left">Company:</div> <div id="stockCompany"></div> <div class="label-left">Last Price:</div> <div id="stockLastPrice"></div> <div class="label-left">Quote Time:</div> <div id="stockQuoteTime"></div> </div> </fieldset> The final result looks something like this: Let's hook up the button handler to fire the request and fill in the data as shown: $("#btnStockQuote").click(function () { ajaxCallMethod("SampleService.ashx", "GetStockQuote", [$("#txtSymbol").val()], function (quote) { $("#divStockDisplay").show().fadeIn(1000); $("#stockCompany").text(quote.Company + " (" + quote.Symbol + ")"); $("#stockLastPrice").text(quote.LastPrice); $("#stockQuoteTime").text(quote.LastQuoteTime.formatDate("MMM dd, HH:mm EST")); }, onPageError); }); So we point at SampleService.ashx and the GetStockQuote method, passing a single parameter of the input symbol value. Then there are two handlers for success and failure callbacks. The success handler is the interesting part - it receives the stock quote as a result and assigns its values to various 'holes' in the stock display elements. The data that comes back over the wire is JSON and it looks like this: { "Symbol":"MSFT", "Company":"Microsoft Corpora", "OpenPrice":26.11, "LastPrice":26.01, "NetChange":0.02, "LastQuoteTime":"2011-11-03T02:00:00Z", "LastQuoteTimeString":"Nov. 11, 2011 4:20pm" } which is an object representation of the data. JavaScript can evaluate this JSON string back into an object easily and that's the reslut that gets passed to the success function. The quote data is then applied to existing page content by manually selecting items and applying them. There are other ways to do this more elegantly like using templates, but here we're only interested in seeing how the data is returned. The data in the object is typed - LastPrice is a number and QuoteTime is a date. Note about the date value: JavaScript doesn't have a date literal although the JSON embedded ISO string format used above ("2011-11-03T02:00:00Z") is becoming fairly standard for JSON serializers. However, JSON parsers don't deserialize dates by default and return them by string. This is why the StockQuote actually returns a string value of LastQuoteTimeString for the same date. ajaxMethodCallback always converts dates properly into 'real' dates and the example above uses the real date value along with a .formatDate() data extension (also in ww.jquery.js) to display the raw date properly. Errors and Exceptions So what happens if your code fails? For example if I pass an invalid stock symbol to the GetStockQuote() method you notice that the code does this: if (quote == null) throw new ApplicationException("Invalid Symbol passed."); CallbackHandler automatically pushes the exception message back to the client so it's easy to pick up the error message. Regardless of what kind of error occurs: Server side, client side, protocol errors - any error will fire the failure handler with an error object parameter. The error is returned to the client via a JSON response in the error callback. In the previous examples I called onPageError which is a generic routine in ww.jquery that displays a status message on the bottom of the screen. But of course you can also take over the error handling yourself: $("#btnStockQuote").click(function () { ajaxCallMethod("SampleService.ashx", "GetStockQuote", [$("#txtSymbol").val()], function (quote) { $("#divStockDisplay").fadeIn(1000); $("#stockCompany").text(quote.Company + " (" + quote.Symbol + ")"); $("#stockLastPrice").text(quote.LastPrice); $("#stockQuoteTime").text(quote.LastQuoteTime.formatDate("MMM dd, hh:mmt")); }, function (error, xhr) { $("#divErrorDisplay").text(error.message).fadeIn(1000); }); }); The error object has a isCallbackError, message and stackTrace properties, the latter of which is only populated when running in Debug mode, and this object is returned for all errors: Client side, transport and server side errors. Regardless of which type of error you get the same object passed (as well as the XHR instance optionally) which makes for a consistent error retrieval mechanism. Specifying HttpVerbs You can also specify HTTP Verbs that are allowed using the AllowedHttpVerbs option on the CallbackMethod attribute: [CallbackMethod(AllowedHttpVerbs=HttpVerbs.GET | HttpVerbs.POST)] public string HelloWorld(string name) { … } If you're building REST style API's this might be useful to force certain request semantics onto the client calling. For the above if call with a non-allowed HttpVerb the request returns a 405 error response along with a JSON (or XML) error object result. The default behavior is to allow all verbs access (HttpVerbs.All). Passing in object Parameters Up to now the parameters I passed were very simple. But what if you need to send something more complex like an object or an array? Let's look at another example now that passes an object from the client to the server. Keeping with the Stock theme here lets add a method called BuyOrder that lets us buy some shares for a stock. Consider the following service method that receives an StockBuyOrder object as a parameter: [CallbackMethod] public string BuyStock(StockBuyOrder buyOrder) { var server = new StockServer(); var quote = server.GetStockQuote(buyOrder.Symbol); if (quote == null) throw new ApplicationException("Invalid or missing stock symbol."); return string.Format("You're buying {0} shares of {1} ({2}) stock at {3} for a total of {4} on {5}.", buyOrder.Quantity, quote.Company, quote.Symbol, quote.LastPrice.ToString("c"), (quote.LastPrice * buyOrder.Quantity).ToString("c"), buyOrder.BuyOn.ToString("MMM d")); } public class StockBuyOrder { public string Symbol { get; set; } public int Quantity { get; set; } public DateTime BuyOn { get; set; } public StockBuyOrder() { BuyOn = DateTime.Now; } } This is a contrived do-nothing example that simply echoes back what was passed in, but it demonstrates how you can pass complex data to a callback method. On the client side we now have a very simple form that captures the three values on a form: <fieldset> <legend>Post a Stock Buy Order</legend> Enter a symbol: <input type="text" name="txtBuySymbol" id="txtBuySymbol" value="GLD" />   Qty: <input type="text" name="txtBuyQty" id="txtBuyQty" value="10" style="width: 50px" />   Buy on: <input type="text" name="txtBuyOn" id="txtBuyOn" value="<%= DateTime.Now.ToString("d") %>" style="width: 70px;" /> <input type="button" id="btnBuyStock" value="Buy Stock" /> <div id="divStockBuyMessage" class="errordisplay" style="display:none"></div> </fieldset> The completed form and demo then looks something like this: The client side code that picks up the input values and assigns them to object properties and sends the AJAX request looks like this: $("#btnBuyStock").click(function () { // create an object map that matches StockBuyOrder signature var buyOrder = { Symbol: $("#txtBuySymbol").val(), Quantity: $("#txtBuyQty").val() * 1, // number Entered: new Date() } ajaxCallMethod("SampleService.ashx", "BuyStock", [buyOrder], function (result) { $("#divStockBuyMessage").text(result).fadeIn(1000); }, onPageError); }); The code creates an object and attaches the properties that match the server side object passed to the BuyStock method. Each property that you want to update needs to be included and the type must match (ie. string, number, date in this case). Any missing properties will not be set but also not cause any errors. Pass POST data instead of Objects In the last example I collected a bunch of values from form variables and stuffed them into object variables in JavaScript code. While that works, often times this isn't really helping - I end up converting my types on the client and then doing another conversion on the server. If lots of input controls are on a page and you just want to pick up the values on the server via plain POST variables - that can be done too - and it makes sense especially if you're creating and filling the client side object only to push data to the server. Let's add another method to the server that once again lets us buy a stock. But this time let's not accept a parameter but rather send POST data to the server. Here's the server method receiving POST data: [CallbackMethod] public string BuyStockPost() { StockBuyOrder buyOrder = new StockBuyOrder(); buyOrder.Symbol = Request.Form["txtBuySymbol"]; ; int qty; int.TryParse(Request.Form["txtBuyQuantity"], out qty); buyOrder.Quantity = qty; DateTime time; DateTime.TryParse(Request.Form["txtBuyBuyOn"], out time); buyOrder.BuyOn = time; // Or easier way yet //FormVariableBinder.Unbind(buyOrder,null,"txtBuy"); var server = new StockServer(); var quote = server.GetStockQuote(buyOrder.Symbol); if (quote == null) throw new ApplicationException("Invalid or missing stock symbol."); return string.Format("You're buying {0} shares of {1} ({2}) stock at {3} for a total of {4} on {5}.", buyOrder.Quantity, quote.Company, quote.Symbol, quote.LastPrice.ToString("c"), (quote.LastPrice * buyOrder.Quantity).ToString("c"), buyOrder.BuyOn.ToString("MMM d")); } Clearly we've made this server method take more code than it did with the object parameter. We've basically moved the parameter assignment logic from the client to the server. As a result the client code to call this method is now a bit shorter since there's no client side shuffling of values from the controls to an object. $("#btnBuyStockPost").click(function () { ajaxCallMethod("SampleService.ashx", "BuyStockPost", [], // Note: No parameters - function (result) { $("#divStockBuyMessage").text(result).fadeIn(1000); }, onPageError, // Force all page Form Variables to be posted { postbackMode: "Post" }); }); The client simply calls the BuyStockQuote method and pushes all the form variables from the page up to the server which parses them instead. The feature that makes this work is one of the options you can pass to the ajaxCallMethod() function: { postbackMode: "Post" }); which directs the function to include form variable POST data when making the service call. Other options include PostNoViewState (for WebForms to strip out WebForms crap vars), PostParametersOnly (default), None. If you pass parameters those are always posted to the server except when None is set. The above code can be simplified a bit by using the FormVariableBinder helper, which can unbind form variables directly into an object: FormVariableBinder.Unbind(buyOrder,null,"txtBuy"); which replaces the manual Request.Form[] reading code. It receives the object to unbind into, a string of properties to skip, and an optional prefix which is stripped off form variables to match property names. The component is similar to the MVC model binder but it's independent of MVC. Returning non-JSON Data CallbackHandler also supports returning non-JSON/XML data via special return types. You can return raw non-JSON encoded strings like this: [CallbackMethod(ReturnAsRawString=true,ContentType="text/plain")] public string HelloWorldNoJSON(string name) { return "Hello " + name + ". Time is: " + DateTime.Now.ToString(); } Calling this method results in just a plain string - no JSON encoding with quotes around the result. This can be useful if your server handling code needs to return a string or HTML result that doesn't fit well for a page or other UI component. Any string output can be returned. You can also return binary data. Stream, byte[] and Bitmap/Image results are automatically streamed back to the client. Notice that you should set the ContentType of the request either on the CallbackMethod attribute or using Response.ContentType. This ensures the Web Server knows how to display your binary response. Using a stream response makes it possible to return any of data. Streamed data can be pretty handy to return bitmap data from a method. The following is a method that returns a stock history graph for a particular stock over a provided number of years: [CallbackMethod(ContentType="image/png",RouteUrl="stocks/history/graph/{symbol}/{years}")] public Stream GetStockHistoryGraph(string symbol, int years = 2,int width = 500, int height=350) { if (width == 0) width = 500; if (height == 0) height = 350; StockServer server = new StockServer(); return server.GetStockHistoryGraph(symbol,"Stock History for " + symbol,width,height,years); } I can now hook this up into the JavaScript code when I get a stock quote. At the end of the process I can assign the URL to the service that returns the image into the src property and so force the image to display. Here's the changed code: $("#btnStockQuote").click(function () { var symbol = $("#txtSymbol").val(); ajaxCallMethod("SampleService.ashx", "GetStockQuote", [symbol], function (quote) { $("#divStockDisplay").fadeIn(1000); $("#stockCompany").text(quote.Company + " (" + quote.Symbol + ")"); $("#stockLastPrice").text(quote.LastPrice); $("#stockQuoteTime").text(quote.LastQuoteTime.formatDate("MMM dd, hh:mmt")); // display a stock chart $("#imgStockHistory").attr("src", "stocks/history/graph/" + symbol + "/2"); },onPageError); }); The resulting output then looks like this: The charting code uses the new ASP.NET 4.0 Chart components via code to display a bar chart of the 2 year stock data as part of the StockServer class which you can find in the sample download. The ability to return arbitrary data from a service is useful as you can see - in this case the chart is clearly associated with the service and it's nice that the graph generation can happen off a handler rather than through a page. Images are common resources, but output can also be PDF reports, zip files for downloads etc. which is becoming increasingly more common to be returned from REST endpoints and other applications. Why reinvent? Obviously the examples I've shown here are pretty basic in terms of functionality. But I hope they demonstrate the core features of AJAX callbacks that you need to work through in most applications which is simple: return data, send back data and potentially retrieve data in various formats. While there are other solutions when it comes down to making AJAX callbacks and servicing REST like requests, I like the flexibility my home grown solution provides. Simply put it's still the easiest solution that I've found that addresses my common use cases: AJAX JSON RPC style callbacks Url based access XML and JSON Output from single method endpoint XML and JSON POST support, querystring input, routing parameter mapping UrlEncoded POST data support on callbacks Ability to return stream/raw string data Essentially ability to return ANYTHING from Service and pass anything All these features are available in various solutions but not together in one place. I've been using this code base for over 4 years now in a number of projects both for myself and commercial work and it's served me extremely well. Besides the AJAX functionality CallbackHandler provides, it's also an easy way to create any kind of output endpoint I need to create. Need to create a few simple routines that spit back some data, but don't want to create a Page or View or full blown handler for it? Create a CallbackHandler and add a method or multiple methods and you have your generic endpoints. It's a quick and easy way to add small code pieces that are pretty efficient as they're running through a pretty small handler implementation. I can have this up and running in a couple of minutes literally without any setup and returning just about any kind of data. Resources Download the Sample NuGet: Westwind Web and AJAX Utilities (Westwind.Web) ajaxCallMethod() Documentation Using the AjaxMethodCallback WebForms Control West Wind Web Toolkit Home Page West Wind Web Toolkit Source Code © Rick Strahl, West Wind Technologies, 2005-2011Posted in ASP.NET jQuery AJAX Tweet (function() { var po = document.createElement('script'); po.type = 'text/javascript'; po.async = true; po.src = 'https://apis.google.com/js/plusone.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s); })();

Read the article

Azure Mobile Services: what files does it consist of?

- by svdoever

Azure Mobile Services is a platform that provides a small set of functionality consisting of authentication, custom data tables, custom API’s, scheduling scripts and push notifications to be used as the back-end of a mobile application or if you want, any application or web site. As described in my previous post Azure Mobile Services: lessons learned the documentation on what can be used in the custom scripts is a bit minimalistic. The list below of all files the complete Azure Mobile Services platform consists of ca shed some light on what is available in the platform. In following posts I will provide more detailed information on what we can conclude from this list of files. Below are the available files as available in the Azure Mobile Services platform. The bold files are files that describe your data model, api scripts, scheduler scripts and table scripts. Those are the files you configure/construct to provide the “configuration”/implementation of you mobile service. The files are located in a folder like C:\DWASFiles\Sites\youreservice\VirtualDirectory0\site\wwwroot. One file is missing in the list below and that is the event log file C:\DWASFiles\Sites\youreservice\VirtualDirectory0\site\LogFiles\eventlog.xml where your messages written with for example console.log() and exception catched by the system are written. NOTA BENE: the Azure Mobile Services system is a system that is under full development, new releases may change the list of files. ./app.js ./App_Data/config/datamodel.json ./App_Data/config/scripts/api/youreapi.js ./App_Data/config/scripts/api/youreapi.json ./App_Data/config/scripts/scheduler/placeholder ./App_Data/config/scripts/scheduler/youresheduler.js ./App_Data/config/scripts/shared/placeholder ./App_Data/config/scripts/table/placeholder ./App_Data/config/scripts/table/yourtable.insert.js ./App_Data/config/scripts/table/yourtable.update.js ./App_Data/config/scripts/table/yourtable.delete.js ./App_Data/config/scripts/table/yourtable.read.js ./node_modules/apn/index.js ./node_modules/apn/lib/connection.js ./node_modules/apn/lib/device.js ./node_modules/apn/lib/errors.js ./node_modules/apn/lib/feedback.js ./node_modules/apn/lib/notification.js ./node_modules/apn/lib/util.js ./node_modules/apn/node_modules/q/package.json ./node_modules/apn/node_modules/q/q.js ./node_modules/apn/package.json ./node_modules/azure/lib/azure.js ./node_modules/azure/lib/cli/blobUtils.js ./node_modules/azure/lib/cli/cacheUtils.js ./node_modules/azure/lib/cli/callbackAggregator.js ./node_modules/azure/lib/cli/cert.js ./node_modules/azure/lib/cli/channel.js ./node_modules/azure/lib/cli/cli.js ./node_modules/azure/lib/cli/commands/account.js ./node_modules/azure/lib/cli/commands/config.js ./node_modules/azure/lib/cli/commands/deployment.js ./node_modules/azure/lib/cli/commands/deployment_.js ./node_modules/azure/lib/cli/commands/help.js ./node_modules/azure/lib/cli/commands/log.js ./node_modules/azure/lib/cli/commands/log_.js ./node_modules/azure/lib/cli/commands/repository.js ./node_modules/azure/lib/cli/commands/repository_.js ./node_modules/azure/lib/cli/commands/service.js ./node_modules/azure/lib/cli/commands/site.js ./node_modules/azure/lib/cli/commands/site_.js ./node_modules/azure/lib/cli/commands/vm.js ./node_modules/azure/lib/cli/common.js ./node_modules/azure/lib/cli/constants.js ./node_modules/azure/lib/cli/generate-psm1-utils.js ./node_modules/azure/lib/cli/generate-psm1.js ./node_modules/azure/lib/cli/iaas/blobserviceex.js ./node_modules/azure/lib/cli/iaas/deleteImage.js ./node_modules/azure/lib/cli/iaas/image.js ./node_modules/azure/lib/cli/iaas/upload/blobInfo.js ./node_modules/azure/lib/cli/iaas/upload/bufferStream.js ./node_modules/azure/lib/cli/iaas/upload/intSet.js ./node_modules/azure/lib/cli/iaas/upload/jobTracker.js ./node_modules/azure/lib/cli/iaas/upload/pageBlob.js ./node_modules/azure/lib/cli/iaas/upload/streamMerger.js ./node_modules/azure/lib/cli/iaas/upload/uploadVMImage.js ./node_modules/azure/lib/cli/iaas/upload/vhdTools.js ./node_modules/azure/lib/cli/keyFiles.js ./node_modules/azure/lib/cli/patch-winston.js ./node_modules/azure/lib/cli/templates/node/iisnode.yml ./node_modules/azure/lib/cli/utils.js ./node_modules/azure/lib/diagnostics/logger.js ./node_modules/azure/lib/http/webresource.js ./node_modules/azure/lib/serviceruntime/fileinputchannel.js ./node_modules/azure/lib/serviceruntime/goalstatedeserializer.js ./node_modules/azure/lib/serviceruntime/namedpipeinputchannel.js ./node_modules/azure/lib/serviceruntime/namedpipeoutputchannel.js ./node_modules/azure/lib/serviceruntime/protocol1runtimeclient.js ./node_modules/azure/lib/serviceruntime/protocol1runtimecurrentstateclient.js ./node_modules/azure/lib/serviceruntime/protocol1runtimegoalstateclient.js ./node_modules/azure/lib/serviceruntime/roleenvironment.js ./node_modules/azure/lib/serviceruntime/runtimekernel.js ./node_modules/azure/lib/serviceruntime/runtimeversionmanager.js ./node_modules/azure/lib/serviceruntime/runtimeversionprotocolclient.js ./node_modules/azure/lib/serviceruntime/xmlcurrentstateserializer.js ./node_modules/azure/lib/serviceruntime/xmlgoalstatedeserializer.js ./node_modules/azure/lib/serviceruntime/xmlroleenvironmentdatadeserializer.js ./node_modules/azure/lib/services/blob/blobservice.js ./node_modules/azure/lib/services/blob/hmacsha256sign.js ./node_modules/azure/lib/services/blob/models/blobresult.js ./node_modules/azure/lib/services/blob/models/blocklistresult.js ./node_modules/azure/lib/services/blob/models/containeraclresult.js ./node_modules/azure/lib/services/blob/models/containerresult.js ./node_modules/azure/lib/services/blob/models/leaseresult.js ./node_modules/azure/lib/services/blob/models/listblobsresultcontinuation.js ./node_modules/azure/lib/services/blob/models/listcontainersresultcontinuation.js ./node_modules/azure/lib/services/blob/models/servicepropertiesresult.js ./node_modules/azure/lib/services/blob/sharedaccesssignature.js ./node_modules/azure/lib/services/blob/sharedkey.js ./node_modules/azure/lib/services/blob/sharedkeylite.js ./node_modules/azure/lib/services/core/connectionstringparser.js ./node_modules/azure/lib/services/core/exponentialretrypolicyfilter.js ./node_modules/azure/lib/services/core/linearretrypolicyfilter.js ./node_modules/azure/lib/services/core/servicebusserviceclient.js ./node_modules/azure/lib/services/core/servicebussettings.js ./node_modules/azure/lib/services/core/serviceclient.js ./node_modules/azure/lib/services/core/servicemanagementclient.js ./node_modules/azure/lib/services/core/servicemanagementsettings.js ./node_modules/azure/lib/services/core/servicesettings.js ./node_modules/azure/lib/services/core/storageserviceclient.js ./node_modules/azure/lib/services/core/storageservicesettings.js ./node_modules/azure/lib/services/queue/models/listqueuesresultcontinuation.js ./node_modules/azure/lib/services/queue/models/queuemessageresult.js ./node_modules/azure/lib/services/queue/models/queueresult.js ./node_modules/azure/lib/services/queue/models/servicepropertiesresult.js ./node_modules/azure/lib/services/queue/queueservice.js ./node_modules/azure/lib/services/serviceBus/models/acstokenresult.js ./node_modules/azure/lib/services/serviceBus/models/queuemessageresult.js ./node_modules/azure/lib/services/serviceBus/models/queueresult.js ./node_modules/azure/lib/services/serviceBus/models/ruleresult.js ./node_modules/azure/lib/services/serviceBus/models/subscriptionresult.js ./node_modules/azure/lib/services/serviceBus/models/topicresult.js ./node_modules/azure/lib/services/serviceBus/servicebusservice.js ./node_modules/azure/lib/services/serviceBus/wrap.js ./node_modules/azure/lib/services/serviceBus/wrapservice.js ./node_modules/azure/lib/services/serviceBus/wraptokenmanager.js ./node_modules/azure/lib/services/serviceManagement/models/roleparser.js ./node_modules/azure/lib/services/serviceManagement/models/roleschema.json ./node_modules/azure/lib/services/serviceManagement/models/servicemanagementserialize.js ./node_modules/azure/lib/services/serviceManagement/servicemanagementservice.js ./node_modules/azure/lib/services/table/batchserviceclient.js ./node_modules/azure/lib/services/table/models/entityresult.js ./node_modules/azure/lib/services/table/models/queryentitiesresultcontinuation.js ./node_modules/azure/lib/services/table/models/querytablesresultcontinuation.js ./node_modules/azure/lib/services/table/models/servicepropertiesresult.js ./node_modules/azure/lib/services/table/models/tableresult.js ./node_modules/azure/lib/services/table/sharedkeylitetable.js ./node_modules/azure/lib/services/table/sharedkeytable.js ./node_modules/azure/lib/services/table/tablequery.js ./node_modules/azure/lib/services/table/tableservice.js ./node_modules/azure/lib/util/atomhandler.js ./node_modules/azure/lib/util/certificates/der.js ./node_modules/azure/lib/util/certificates/pkcs.js ./node_modules/azure/lib/util/constants.js ./node_modules/azure/lib/util/iso8061date.js ./node_modules/azure/lib/util/js2xml.js ./node_modules/azure/lib/util/rfc1123date.js ./node_modules/azure/lib/util/util.js ./node_modules/azure/lib/util/validate.js ./node_modules/azure/LICENSE.txt ./node_modules/azure/node_modules/async/index.js ./node_modules/azure/node_modules/async/lib/async.js ./node_modules/azure/node_modules/async/LICENSE ./node_modules/azure/node_modules/async/package.json ./node_modules/azure/node_modules/azure/lib/azure.js ./node_modules/azure/node_modules/azure/lib/diagnostics/logger.js ./node_modules/azure/node_modules/azure/lib/http/webresource.js ./node_modules/azure/node_modules/azure/lib/serviceruntime/fileinputchannel.js ./node_modules/azure/node_modules/azure/lib/serviceruntime/goalstatedeserializer.js ./node_modules/azure/node_modules/azure/lib/serviceruntime/namedpipeinputchannel.js ./node_modules/azure/node_modules/azure/lib/serviceruntime/namedpipeoutputchannel.js ./node_modules/azure/node_modules/azure/lib/serviceruntime/protocol1runtimeclient.js ./node_modules/azure/node_modules/azure/lib/serviceruntime/protocol1runtimecurrentstateclient.js ./node_modules/azure/node_modules/azure/lib/serviceruntime/protocol1runtimegoalstateclient.js ./node_modules/azure/node_modules/azure/lib/serviceruntime/roleenvironment.js ./node_modules/azure/node_modules/azure/lib/serviceruntime/runtimekernel.js ./node_modules/azure/node_modules/azure/lib/serviceruntime/runtimeversionmanager.js ./node_modules/azure/node_modules/azure/lib/serviceruntime/runtimeversionprotocolclient.js ./node_modules/azure/node_modules/azure/lib/serviceruntime/xmlcurrentstateserializer.js ./node_modules/azure/node_modules/azure/lib/serviceruntime/xmlgoalstatedeserializer.js ./node_modules/azure/node_modules/azure/lib/serviceruntime/xmlroleenvironmentdatadeserializer.js ./node_modules/azure/node_modules/azure/lib/services/blob/blobservice.js ./node_modules/azure/node_modules/azure/lib/services/blob/internal/sharedaccesssignature.js ./node_modules/azure/node_modules/azure/lib/services/blob/internal/sharedkey.js ./node_modules/azure/node_modules/azure/lib/services/blob/internal/sharedkeylite.js ./node_modules/azure/node_modules/azure/lib/services/blob/models/blobresult.js ./node_modules/azure/node_modules/azure/lib/services/blob/models/blocklistresult.js ./node_modules/azure/node_modules/azure/lib/services/blob/models/containeraclresult.js ./node_modules/azure/node_modules/azure/lib/services/blob/models/containerresult.js ./node_modules/azure/node_modules/azure/lib/services/blob/models/leaseresult.js ./node_modules/azure/node_modules/azure/lib/services/blob/models/listblobsresultcontinuation.js ./node_modules/azure/node_modules/azure/lib/services/blob/models/listcontainersresultcontinuation.js ./node_modules/azure/node_modules/azure/lib/services/blob/models/servicepropertiesresult.js ./node_modules/azure/node_modules/azure/lib/services/core/connectionstringparser.js ./node_modules/azure/node_modules/azure/lib/services/core/exponentialretrypolicyfilter.js ./node_modules/azure/node_modules/azure/lib/services/core/hmacsha256sign.js ./node_modules/azure/node_modules/azure/lib/services/core/linearretrypolicyfilter.js ./node_modules/azure/node_modules/azure/lib/services/core/servicebusserviceclient.js ./node_modules/azure/node_modules/azure/lib/services/core/servicebussettings.js ./node_modules/azure/node_modules/azure/lib/services/core/serviceclient.js ./node_modules/azure/node_modules/azure/lib/services/core/servicemanagementclient.js ./node_modules/azure/node_modules/azure/lib/services/core/servicemanagementsettings.js ./node_modules/azure/node_modules/azure/lib/services/core/servicesettings.js ./node_modules/azure/node_modules/azure/lib/services/core/sqlserviceclient.js ./node_modules/azure/node_modules/azure/lib/services/core/storageserviceclient.js ./node_modules/azure/node_modules/azure/lib/services/core/storageservicesettings.js ./node_modules/azure/node_modules/azure/lib/services/queue/models/listqueuesresultcontinuation.js ./node_modules/azure/node_modules/azure/lib/services/queue/models/queuemessageresult.js ./node_modules/azure/node_modules/azure/lib/services/queue/models/queueresult.js ./node_modules/azure/node_modules/azure/lib/services/queue/models/servicepropertiesresult.js ./node_modules/azure/node_modules/azure/lib/services/queue/queueservice.js ./node_modules/azure/node_modules/azure/lib/services/serviceBus/apnsservice.js ./node_modules/azure/node_modules/azure/lib/services/serviceBus/gcmservice.js ./node_modules/azure/node_modules/azure/lib/services/serviceBus/internal/sharedaccesssignature.js ./node_modules/azure/node_modules/azure/lib/services/serviceBus/internal/wrap.js ./node_modules/azure/node_modules/azure/lib/services/serviceBus/internal/wraptokenmanager.js ./node_modules/azure/node_modules/azure/lib/services/serviceBus/models/acstokenresult.js ./node_modules/azure/node_modules/azure/lib/services/serviceBus/models/notificationhubresult.js ./node_modules/azure/node_modules/azure/lib/services/serviceBus/models/queuemessageresult.js ./node_modules/azure/node_modules/azure/lib/services/serviceBus/models/queueresult.js ./node_modules/azure/node_modules/azure/lib/services/serviceBus/models/registrationresult.js ./node_modules/azure/node_modules/azure/lib/services/serviceBus/models/resourceresult.js ./node_modules/azure/node_modules/azure/lib/services/serviceBus/models/ruleresult.js ./node_modules/azure/node_modules/azure/lib/services/serviceBus/models/subscriptionresult.js ./node_modules/azure/node_modules/azure/lib/services/serviceBus/models/topicresult.js ./node_modules/azure/node_modules/azure/lib/services/serviceBus/notificationhubservice.js ./node_modules/azure/node_modules/azure/lib/services/serviceBus/servicebusservice.js ./node_modules/azure/node_modules/azure/lib/services/serviceBus/servicebusservicebase.js ./node_modules/azure/node_modules/azure/lib/services/serviceBus/wnsservice.js ./node_modules/azure/node_modules/azure/lib/services/serviceBus/wrapservice.js ./node_modules/azure/node_modules/azure/lib/services/serviceManagement/hdinsightservice.js ./node_modules/azure/node_modules/azure/lib/services/serviceManagement/models/roleparser.js ./node_modules/azure/node_modules/azure/lib/services/serviceManagement/models/roleschema.json ./node_modules/azure/node_modules/azure/lib/services/serviceManagement/models/servicemanagementserialize.js ./node_modules/azure/node_modules/azure/lib/services/serviceManagement/servicebusmanagementservice.js ./node_modules/azure/node_modules/azure/lib/services/serviceManagement/servicemanagementservice.js ./node_modules/azure/node_modules/azure/lib/services/serviceManagement/sqlmanagementservice.js ./node_modules/azure/node_modules/azure/lib/services/sqlAzure/models/databaseresult.js ./node_modules/azure/node_modules/azure/lib/services/sqlAzure/sqlserveracs.js ./node_modules/azure/node_modules/azure/lib/services/sqlAzure/sqlservice.js ./node_modules/azure/node_modules/azure/lib/services/table/batchserviceclient.js ./node_modules/azure/node_modules/azure/lib/services/table/internal/sharedkeylitetable.js ./node_modules/azure/node_modules/azure/lib/services/table/internal/sharedkeytable.js ./node_modules/azure/node_modules/azure/lib/services/table/models/entityresult.js ./node_modules/azure/node_modules/azure/lib/services/table/models/listresult.js ./node_modules/azure/node_modules/azure/lib/services/table/models/queryentitiesresultcontinuation.js ./node_modules/azure/node_modules/azure/lib/services/table/models/querytablesresultcontinuation.js ./node_modules/azure/node_modules/azure/lib/services/table/models/servicepropertiesresult.js ./node_modules/azure/node_modules/azure/lib/services/table/models/tableresult.js ./node_modules/azure/node_modules/azure/lib/services/table/tablequery.js ./node_modules/azure/node_modules/azure/lib/services/table/tableservice.js ./node_modules/azure/node_modules/azure/lib/util/atomhandler.js ./node_modules/azure/node_modules/azure/lib/util/constants.js ./node_modules/azure/node_modules/azure/lib/util/date.js ./node_modules/azure/node_modules/azure/lib/util/edmtype.js ./node_modules/azure/node_modules/azure/lib/util/iso8061date.js ./node_modules/azure/node_modules/azure/lib/util/js2xml.js ./node_modules/azure/node_modules/azure/lib/util/odatahandler.js ./node_modules/azure/node_modules/azure/lib/util/rfc1123date.js ./node_modules/azure/node_modules/azure/lib/util/util.js ./node_modules/azure/node_modules/azure/lib/util/validate.js ./node_modules/azure/node_modules/azure/LICENSE.txt ./node_modules/azure/node_modules/azure/node_modules/wns/lib/wns.js ./node_modules/azure/node_modules/azure/node_modules/wns/LICENSE.txt ./node_modules/azure/node_modules/azure/node_modules/wns/package.json ./node_modules/azure/node_modules/azure/node_modules/xml2js/lib/xml2js.js ./node_modules/azure/node_modules/azure/node_modules/xml2js/LICENSE ./node_modules/azure/node_modules/azure/node_modules/xml2js/node_modules/sax/lib/sax.js ./node_modules/azure/node_modules/azure/node_modules/xml2js/node_modules/sax/LICENSE ./node_modules/azure/node_modules/azure/node_modules/xml2js/node_modules/sax/package.json ./node_modules/azure/node_modules/azure/node_modules/xml2js/package.json ./node_modules/azure/node_modules/azure/package.json ./node_modules/azure/node_modules/colors/colors.js ./node_modules/azure/node_modules/colors/MIT-LICENSE.txt ./node_modules/azure/node_modules/colors/package.json ./node_modules/azure/node_modules/commander/index.js ./node_modules/azure/node_modules/commander/lib/commander.js ./node_modules/azure/node_modules/commander/node_modules/keypress/index.js ./node_modules/azure/node_modules/commander/node_modules/keypress/package.json ./node_modules/azure/node_modules/commander/package.json ./node_modules/azure/node_modules/dateformat/lib/dateformat.js ./node_modules/azure/node_modules/dateformat/package.json ./node_modules/azure/node_modules/easy-table/lib/table.js ./node_modules/azure/node_modules/easy-table/package.json ./node_modules/azure/node_modules/eyes/lib/eyes.js ./node_modules/azure/node_modules/eyes/LICENSE ./node_modules/azure/node_modules/eyes/package.json ./node_modules/azure/node_modules/log/index.js ./node_modules/azure/node_modules/log/lib/log.js ./node_modules/azure/node_modules/log/package.json ./node_modules/azure/node_modules/mime/LICENSE ./node_modules/azure/node_modules/mime/mime.js ./node_modules/azure/node_modules/mime/package.json ./node_modules/azure/node_modules/mime/types/mime.types ./node_modules/azure/node_modules/mime/types/node.types ./node_modules/azure/node_modules/node-uuid/LICENSE.md ./node_modules/azure/node_modules/node-uuid/package.json ./node_modules/azure/node_modules/node-uuid/uuid.js ./node_modules/azure/node_modules/qs/component.json ./node_modules/azure/node_modules/qs/index.js ./node_modules/azure/node_modules/qs/lib/head.js ./node_modules/azure/node_modules/qs/lib/querystring.js ./node_modules/azure/node_modules/qs/lib/tail.js ./node_modules/azure/node_modules/qs/package.json ./node_modules/azure/node_modules/qs/querystring.js ./node_modules/azure/node_modules/request/aws.js ./node_modules/azure/node_modules/request/forever.js ./node_modules/azure/node_modules/request/LICENSE ./node_modules/azure/node_modules/request/main.js ./node_modules/azure/node_modules/request/node_modules/form-data/lib/form_data.js ./node_modules/azure/node_modules/request/node_modules/form-data/node_modules/async/index.js ./node_modules/azure/node_modules/request/node_modules/form-data/node_modules/async/lib/async.js ./node_modules/azure/node_modules/request/node_modules/form-data/node_modules/async/LICENSE ./node_modules/azure/node_modules/request/node_modules/form-data/node_modules/async/package.json ./node_modules/azure/node_modules/request/node_modules/form-data/node_modules/combined-stream/lib/combined_stream.js ./node_modules/azure/node_modules/request/node_modules/form-data/node_modules/combined-stream/License ./node_modules/azure/node_modules/request/node_modules/form-data/node_modules/combined-stream/node_modules/delayed-stream/lib/delayed_stream.js ./node_modules/azure/node_modules/request/node_modules/form-data/node_modules/combined-stream/node_modules/delayed-stream/License ./node_modules/azure/node_modules/request/node_modules/form-data/node_modules/combined-stream/node_modules/delayed-stream/package.json ./node_modules/azure/node_modules/request/node_modules/form-data/node_modules/combined-stream/package.json ./node_modules/azure/node_modules/request/node_modules/form-data/package.json ./node_modules/azure/node_modules/request/node_modules/mime/LICENSE ./node_modules/azure/node_modules/request/node_modules/mime/mime.js ./node_modules/azure/node_modules/request/node_modules/mime/package.json ./node_modules/azure/node_modules/request/node_modules/mime/types/mime.types ./node_modules/azure/node_modules/request/node_modules/mime/types/node.types ./node_modules/azure/node_modules/request/oauth.js ./node_modules/azure/node_modules/request/package.json ./node_modules/azure/node_modules/request/tunnel.js ./node_modules/azure/node_modules/request/uuid.js ./node_modules/azure/node_modules/request/vendor/cookie/index.js ./node_modules/azure/node_modules/request/vendor/cookie/jar.js ./node_modules/azure/node_modules/sax/lib/sax.js ./node_modules/azure/node_modules/sax/LICENSE ./node_modules/azure/node_modules/sax/package.json ./node_modules/azure/node_modules/streamline/AUTHORS ./node_modules/azure/node_modules/streamline/deps/narcissus/lib/decompiler.js ./node_modules/azure/node_modules/streamline/deps/narcissus/lib/definitions.js ./node_modules/azure/node_modules/streamline/deps/narcissus/lib/jsbrowser.js ./node_modules/azure/node_modules/streamline/deps/narcissus/lib/jsdecomp.js ./node_modules/azure/node_modules/streamline/deps/narcissus/lib/jsdefs.js ./node_modules/azure/node_modules/streamline/deps/narcissus/lib/jsexec.js ./node_modules/azure/node_modules/streamline/deps/narcissus/lib/jslex.js ./node_modules/azure/node_modules/streamline/deps/narcissus/lib/jsparse.js ./node_modules/azure/node_modules/streamline/deps/narcissus/lib/lexer.js ./node_modules/azure/node_modules/streamline/deps/narcissus/lib/parser.js ./node_modules/azure/node_modules/streamline/deps/narcissus/LICENSE ./node_modules/azure/node_modules/streamline/deps/narcissus/main.js ./node_modules/azure/node_modules/streamline/deps/narcissus/package.json ./node_modules/azure/node_modules/streamline/deps/narcissus/xfail/narcissus-failures.txt ./node_modules/azure/node_modules/streamline/deps/narcissus/xfail/narcissus-slow.txt ./node_modules/azure/node_modules/streamline/lib/callbacks/format.js ./node_modules/azure/node_modules/streamline/lib/callbacks/require-stub.js ./node_modules/azure/node_modules/streamline/lib/callbacks/runtime.js ./node_modules/azure/node_modules/streamline/lib/callbacks/transform.js ./node_modules/azure/node_modules/streamline/lib/compile.js ./node_modules/azure/node_modules/streamline/lib/compiler/command.js ./node_modules/azure/node_modules/streamline/lib/compiler/compile--fibers.js ./node_modules/azure/node_modules/streamline/lib/compiler/compile.js ./node_modules/azure/node_modules/streamline/lib/compiler/compile_.js ./node_modules/azure/node_modules/streamline/lib/compiler/index.js ./node_modules/azure/node_modules/streamline/lib/compiler/register.js ./node_modules/azure/node_modules/streamline/lib/fibers/runtime.js ./node_modules/azure/node_modules/streamline/lib/fibers/transform.js ./node_modules/azure/node_modules/streamline/lib/fibers/walker.js ./node_modules/azure/node_modules/streamline/lib/globals.js ./node_modules/azure/node_modules/streamline/lib/index.js ./node_modules/azure/node_modules/streamline/lib/register.js ./node_modules/azure/node_modules/streamline/lib/require/client/require.js ./node_modules/azure/node_modules/streamline/lib/require/server/depend.js ./node_modules/azure/node_modules/streamline/lib/require/server/require.js ./node_modules/azure/node_modules/streamline/lib/streams/client/streams--fibers.js ./node_modules/azure/node_modules/streamline/lib/streams/client/streams.js ./node_modules/azure/node_modules/streamline/lib/streams/client/streams_.js ./node_modules/azure/node_modules/streamline/lib/streams/jsonRequest.js ./node_modules/azure/node_modules/streamline/lib/streams/readers.js ./node_modules/azure/node_modules/streamline/lib/streams/server/httpHelper.js ./node_modules/azure/node_modules/streamline/lib/streams/server/streams.js ./node_modules/azure/node_modules/streamline/lib/streams/streams.js ./node_modules/azure/node_modules/streamline/lib/tools/docTool.js ./node_modules/azure/node_modules/streamline/lib/transform.js ./node_modules/azure/node_modules/streamline/lib/util/flows--fibers.js ./node_modules/azure/node_modules/streamline/lib/util/flows.js ./node_modules/azure/node_modules/streamline/lib/util/flows_.js ./node_modules/azure/node_modules/streamline/lib/util/future.js ./node_modules/azure/node_modules/streamline/lib/util/index.js ./node_modules/azure/node_modules/streamline/lib/util/url.js ./node_modules/azure/node_modules/streamline/lib/util/uuid.js ./node_modules/azure/node_modules/streamline/module.js ./node_modules/azure/node_modules/streamline/package.json ./node_modules/azure/node_modules/tunnel/index.js ./node_modules/azure/node_modules/tunnel/lib/tunnel.js ./node_modules/azure/node_modules/tunnel/package.json ./node_modules/azure/node_modules/underscore/index.js ./node_modules/azure/node_modules/underscore/LICENSE ./node_modules/azure/node_modules/underscore/package.json ./node_modules/azure/node_modules/underscore/underscore.js ./node_modules/azure/node_modules/underscore.string/lib/underscore.string.js ./node_modules/azure/node_modules/underscore.string/package.json ./node_modules/azure/node_modules/validator/index.js ./node_modules/azure/node_modules/validator/lib/defaultError.js ./node_modules/azure/node_modules/validator/lib/entities.js ./node_modules/azure/node_modules/validator/lib/filter.js ./node_modules/azure/node_modules/validator/lib/index.js ./node_modules/azure/node_modules/validator/lib/validator.js ./node_modules/azure/node_modules/validator/lib/validators.js ./node_modules/azure/node_modules/validator/lib/xss.js ./node_modules/azure/node_modules/validator/LICENSE ./node_modules/azure/node_modules/validator/package.json ./node_modules/azure/node_modules/validator/validator.js ./node_modules/azure/node_modules/winston/lib/winston/common.js ./node_modules/azure/node_modules/winston/lib/winston/config/cli-config.js ./node_modules/azure/node_modules/winston/lib/winston/config/npm-config.js ./node_modules/azure/node_modules/winston/lib/winston/config/syslog-config.js ./node_modules/azure/node_modules/winston/lib/winston/config.js ./node_modules/azure/node_modules/winston/lib/winston/container.js ./node_modules/azure/node_modules/winston/lib/winston/exception.js ./node_modules/azure/node_modules/winston/lib/winston/logger.js ./node_modules/azure/node_modules/winston/lib/winston/transports/console.js ./node_modules/azure/node_modules/winston/lib/winston/transports/file.js ./node_modules/azure/node_modules/winston/lib/winston/transports/http.js ./node_modules/azure/node_modules/winston/lib/winston/transports/transport.js ./node_modules/azure/node_modules/winston/lib/winston/transports/webhook.js ./node_modules/azure/node_modules/winston/lib/winston/transports.js ./node_modules/azure/node_modules/winston/lib/winston.js ./node_modules/azure/node_modules/winston/LICENSE ./node_modules/azure/node_modules/winston/node_modules/cycle/cycle.js ./node_modules/azure/node_modules/winston/node_modules/cycle/package.json ./node_modules/azure/node_modules/winston/node_modules/pkginfo/lib/pkginfo.js ./node_modules/azure/node_modules/winston/node_modules/pkginfo/package.json ./node_modules/azure/node_modules/winston/node_modules/request/aws.js ./node_modules/azure/node_modules/winston/node_modules/request/aws2.js ./node_modules/azure/node_modules/winston/node_modules/request/forever.js ./node_modules/azure/node_modules/winston/node_modules/request/LICENSE ./node_modules/azure/node_modules/winston/node_modules/request/main.js ./node_modules/azure/node_modules/winston/node_modules/request/mimetypes.js ./node_modules/azure/node_modules/winston/node_modules/request/oauth.js ./node_modules/azure/node_modules/winston/node_modules/request/package.json ./node_modules/azure/node_modules/winston/node_modules/request/tunnel.js ./node_modules/azure/node_modules/winston/node_modules/request/uuid.js ./node_modules/azure/node_modules/winston/node_modules/request/vendor/cookie/index.js ./node_modules/azure/node_modules/winston/node_modules/request/vendor/cookie/jar.js ./node_modules/azure/node_modules/winston/node_modules/stack-trace/lib/stack-trace.js ./node_modules/azure/node_modules/winston/node_modules/stack-trace/License ./node_modules/azure/node_modules/winston/node_modules/stack-trace/package.json ./node_modules/azure/node_modules/winston/package.json ./node_modules/azure/node_modules/xml2js/lib/xml2js.js ./node_modules/azure/node_modules/xml2js/LICENSE ./node_modules/azure/node_modules/xml2js/package.json ./node_modules/azure/node_modules/xmlbuilder/lib/index.js ./node_modules/azure/node_modules/xmlbuilder/lib/XMLBuilder.js ./node_modules/azure/node_modules/xmlbuilder/lib/XMLFragment.js ./node_modules/azure/node_modules/xmlbuilder/package.json ./node_modules/azure/package.json ./node_modules/dpush/lib/dpush.js ./node_modules/dpush/LICENSE.txt ./node_modules/dpush/package.json ./node_modules/express/.npmignore ./node_modules/express/.travis.yml ./node_modules/express/bin/express ./node_modules/express/History.md ./node_modules/express/index.js ./node_modules/express/lib/application.js ./node_modules/express/lib/express.js ./node_modules/express/lib/middleware.js ./node_modules/express/lib/request.js ./node_modules/express/lib/response.js ./node_modules/express/lib/router/index.js ./node_modules/express/lib/router/route.js ./node_modules/express/lib/utils.js ./node_modules/express/lib/view.js ./node_modules/express/LICENSE ./node_modules/express/Makefile ./node_modules/express/node_modules/buffer-crc32/.npmignore ./node_modules/express/node_modules/buffer-crc32/.travis.yml ./node_modules/express/node_modules/buffer-crc32/index.js ./node_modules/express/node_modules/buffer-crc32/package.json ./node_modules/express/node_modules/buffer-crc32/README.md ./node_modules/express/node_modules/buffer-crc32/tests/crc.test.js ./node_modules/express/node_modules/commander/.npmignore ./node_modules/express/node_modules/commander/.travis.yml ./node_modules/express/node_modules/commander/History.md ./node_modules/express/node_modules/commander/index.js ./node_modules/express/node_modules/commander/lib/commander.js ./node_modules/express/node_modules/commander/Makefile ./node_modules/express/node_modules/commander/package.json ./node_modules/express/node_modules/commander/Readme.md ./node_modules/express/node_modules/connect/.npmignore ./node_modules/express/node_modules/connect/.travis.yml ./node_modules/express/node_modules/connect/index.js ./node_modules/express/node_modules/connect/lib/cache.js ./node_modules/express/node_modules/connect/lib/connect.js ./node_modules/express/node_modules/connect/lib/index.js ./node_modules/express/node_modules/connect/lib/middleware/basicAuth.js ./node_modules/express/node_modules/connect/lib/middleware/bodyParser.js ./node_modules/express/node_modules/connect/lib/middleware/compress.js ./node_modules/express/node_modules/connect/lib/middleware/cookieParser.js ./node_modules/express/node_modules/connect/lib/middleware/cookieSession.js ./node_modules/express/node_modules/connect/lib/middleware/csrf.js ./node_modules/express/node_modules/connect/lib/middleware/directory.js ./node_modules/express/node_modules/connect/lib/middleware/errorHandler.js ./node_modules/express/node_modules/connect/lib/middleware/favicon.js ./node_modules/express/node_modules/connect/lib/middleware/json.js ./node_modules/express/node_modules/connect/lib/middleware/limit.js ./node_modules/express/node_modules/connect/lib/middleware/logger.js ./node_modules/express/node_modules/connect/lib/middleware/methodOverride.js ./node_modules/express/node_modules/connect/lib/middleware/multipart.js ./node_modules/express/node_modules/connect/lib/middleware/query.js ./node_modules/express/node_modules/connect/lib/middleware/responseTime.js ./node_modules/express/node_modules/connect/lib/middleware/session/cookie.js ./node_modules/express/node_modules/connect/lib/middleware/session/memory.js ./node_modules/express/node_modules/connect/lib/middleware/session/session.js ./node_modules/express/node_modules/connect/lib/middleware/session/store.js ./node_modules/express/node_modules/connect/lib/middleware/session.js ./node_modules/express/node_modules/connect/lib/middleware/static.js ./node_modules/express/node_modules/connect/lib/middleware/staticCache.js ./node_modules/express/node_modules/connect/lib/middleware/timeout.js ./node_modules/express/node_modules/connect/lib/middleware/urlencoded.js ./node_modules/express/node_modules/connect/lib/middleware/vhost.js ./node_modules/express/node_modules/connect/lib/patch.js ./node_modules/express/node_modules/connect/lib/proto.js ./node_modules/express/node_modules/connect/lib/public/directory.html ./node_modules/express/node_modules/connect/lib/public/error.html ./node_modules/express/node_modules/connect/lib/public/favicon.ico ./node_modules/express/node_modules/connect/lib/public/icons/page.png ./node_modules/express/node_modules/connect/lib/public/icons/page_add.png ./node_modules/express/node_modules/connect/lib/public/icons/page_attach.png ./node_modules/express/node_modules/connect/lib/public/icons/page_code.png ./node_modules/express/node_modules/connect/lib/public/icons/page_copy.png ./node_modules/express/node_modules/connect/lib/public/icons/page_delete.png ./node_modules/express/node_modules/connect/lib/public/icons/page_edit.png ./node_modules/express/node_modules/connect/lib/public/icons/page_error.png ./node_modules/express/node_modules/connect/lib/public/icons/page_excel.png ./node_modules/express/node_modules/connect/lib/public/icons/page_find.png ./node_modules/express/node_modules/connect/lib/public/icons/page_gear.png ./node_modules/express/node_modules/connect/lib/public/icons/page_go.png ./node_modules/express/node_modules/connect/lib/public/icons/page_green.png ./node_modules/express/node_modules/connect/lib/public/icons/page_key.png ./node_modules/express/node_modules/connect/lib/public/icons/page_lightning.png ./node_modules/express/node_modules/connect/lib/public/icons/page_link.png ./node_modules/express/node_modules/connect/lib/public/icons/page_paintbrush.png ./node_modules/express/node_modules/connect/lib/public/icons/page_paste.png ./node_modules/express/node_modules/connect/lib/public/icons/page_red.png ./node_modules/express/node_modules/connect/lib/public/icons/page_refresh.png ./node_modules/express/node_modules/connect/lib/public/icons/page_save.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_acrobat.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_actionscript.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_add.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_c.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_camera.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_cd.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_code.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_code_red.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_coldfusion.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_compressed.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_copy.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_cplusplus.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_csharp.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_cup.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_database.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_delete.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_dvd.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_edit.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_error.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_excel.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_find.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_flash.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_freehand.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_gear.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_get.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_go.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_h.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_horizontal.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_key.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_lightning.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_link.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_magnify.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_medal.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_office.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_paint.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_paintbrush.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_paste.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_php.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_picture.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_powerpoint.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_put.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_ruby.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_stack.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_star.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_swoosh.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_text.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_text_width.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_tux.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_vector.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_visualstudio.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_width.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_word.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_world.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_wrench.png ./node_modules/express/node_modules/connect/lib/public/icons/page_white_zip.png ./node_modules/express/node_modules/connect/lib/public/icons/page_word.png ./node_modules/express/node_modules/connect/lib/public/icons/page_world.png ./node_modules/express/node_modules/connect/lib/public/style.css ./node_modules/express/node_modules/connect/lib/utils.js ./node_modules/express/node_modules/connect/LICENSE ./node_modules/express/node_modules/connect/node_modules/bytes/.npmignore ./node_modules/express/node_modules/connect/node_modules/bytes/component.json ./node_modules/express/node_modules/connect/node_modules/bytes/History.md ./node_modules/express/node_modules/connect/node_modules/bytes/index.js ./node_modules/express/node_modules/connect/node_modules/bytes/Makefile ./node_modules/express/node_modules/connect/node_modules/bytes/package.json ./node_modules/express/node_modules/connect/node_modules/bytes/Readme.md ./node_modules/express/node_modules/connect/node_modules/formidable/.npmignore ./node_modules/express/node_modules/connect/node_modules/formidable/.travis.yml ./node_modules/express/node_modules/connect/node_modules/formidable/benchmark/bench-multipart-parser.js ./node_modules/express/node_modules/connect/node_modules/formidable/example/json.js ./node_modules/express/node_modules/connect/node_modules/formidable/example/post.js ./node_modules/express/node_modules/connect/node_modules/formidable/example/upload.js ./node_modules/express/node_modules/connect/node_modules/formidable/index.js ./node_modules/express/node_modules/connect/node_modules/formidable/lib/file.js ./node_modules/express/node_modules/connect/node_modules/formidable/lib/incoming_form.js ./node_modules/express/node_modules/connect/node_modules/formidable/lib/index.js ./node_modules/express/node_modules/connect/node_modules/formidable/lib/json_parser.js ./node_modules/express/node_modules/connect/node_modules/formidable/lib/multipart_parser.js ./node_modules/express/node_modules/connect/node_modules/formidable/lib/octet_parser.js ./node_modules/express/node_modules/connect/node_modules/formidable/lib/querystring_parser.js ./node_modules/express/node_modules/connect/node_modules/formidable/LICENSE ./node_modules/express/node_modules/connect/node_modules/formidable/package.json ./node_modules/express/node_modules/connect/node_modules/formidable/Readme.md ./node_modules/express/node_modules/connect/node_modules/formidable/test/common.js ./node_modules/express/node_modules/connect/node_modules/formidable/test/fixture/file/beta-sticker-1.png ./node_modules/express/node_modules/connect/node_modules/formidable/test/fixture/file/binaryfile.tar.gz ./node_modules/express/node_modules/connect/node_modules/formidable/test/fixture/file/blank.gif ./node_modules/express/node_modules/connect/node_modules/formidable/test/fixture/file/funkyfilename.txt ./node_modules/express/node_modules/connect/node_modules/formidable/test/fixture/file/menu_separator.png ./node_modules/express/node_modules/connect/node_modules/formidable/test/fixture/file/plain.txt ./node_modules/express/node_modules/connect/node_modules/formidable/test/fixture/http/special-chars-in-filename/info.md ./node_modules/express/node_modules/connect/node_modules/formidable/test/fixture/js/encoding.js ./node_modules/express/node_modules/connect/node_modules/formidable/test/fixture/js/misc.js ./node_modules/express/node_modules/connect/node_modules/formidable/test/fixture/js/no-filename.js ./node_modules/express/node_modules/connect/node_modules/formidable/test/fixture/js/preamble.js ./node_modules/express/node_modules/connect/node_modules/formidable/test/fixture/js/special-chars-in-filename.js ./node_modules/express/node_modules/connect/node_modules/formidable/test/fixture/js/workarounds.js ./node_modules/express/node_modules/connect/node_modules/formidable/test/fixture/multipart.js ./node_modules/express/node_modules/connect/node_modules/formidable/test/integration/test-fixtures.js ./node_modules/express/node_modules/connect/node_modules/formidable/test/integration/test-json.js ./node_modules/express/node_modules/connect/node_modules/formidable/test/integration/test-octet-stream.js ./node_modules/express/node_modules/connect/node_modules/formidable/test/legacy/common.js ./node_modules/express/node_modules/connect/node_modules/formidable/test/legacy/integration/test-multipart-parser.js ./node_modules/express/node_modules/connect/node_modules/formidable/test/legacy/simple/test-file.js ./node_modules/express/node_modules/connect/node_modules/formidable/test/legacy/simple/test-incoming-form.js ./node_modules/express/node_modules/connect/node_modules/formidable/test/legacy/simple/test-multipart-parser.js ./node_modules/express/node_modules/connect/node_modules/formidable/test/legacy/simple/test-querystring-parser.js ./node_modules/express/node_modules/connect/node_modules/formidable/test/legacy/system/test-multi-video-upload.js ./node_modules/express/node_modules/connect/node_modules/formidable/test/run.js ./node_modules/express/node_modules/connect/node_modules/formidable/test/standalone/test-connection-aborted.js ./node_modules/express/node_modules/connect/node_modules/formidable/test/standalone/test-content-transfer-encoding.js ./node_modules/express/node_modules/connect/node_modules/formidable/test/standalone/test-issue-46.js ./node_modules/express/node_modules/connect/node_modules/formidable/test/tools/base64.html ./node_modules/express/node_modules/connect/node_modules/formidable/test/unit/test-file.js ./node_modules/express/node_modules/connect/node_modules/formidable/test/unit/test-incoming-form.js ./node_modules/express/node_modules/connect/node_modules/formidable/tool/record.js ./node_modules/express/node_modules/connect/node_modules/pause/.npmignore ./node_modules/express/node_modules/connect/node_modules/pause/History.md ./node_modules/express/node_modules/connect/node_modules/pause/index.js ./node_modules/express/node_modules/connect/node_modules/pause/Makefile ./node_modules/express/node_modules/connect/node_modules/pause/package.json ./node_modules/express/node_modules/connect/node_modules/pause/Readme.md ./node_modules/express/node_modules/connect/node_modules/qs/.gitmodules ./node_modules/express/node_modules/connect/node_modules/qs/.npmignore ./node_modules/express/node_modules/connect/node_modules/qs/index.js ./node_modules/express/node_modules/connect/node_modules/qs/package.json ./node_modules/express/node_modules/connect/node_modules/qs/Readme.md ./node_modules/express/node_modules/connect/package.json ./node_modules/express/node_modules/connect/test.js ./node_modules/express/node_modules/cookie/.npmignore ./node_modules/express/node_modules/cookie/.travis.yml ./node_modules/express/node_modules/cookie/index.js ./node_modules/express/node_modules/cookie/package.json ./node_modules/express/node_modules/cookie/README.md ./node_modules/express/node_modules/cookie/test/mocha.opts ./node_modules/express/node_modules/cookie/test/parse.js ./node_modules/express/node_modules/cookie/test/serialize.js ./node_modules/express/node_modules/cookie-signature/.npmignore ./node_modules/express/node_modules/cookie-signature/History.md ./node_modules/express/node_modules/cookie-signature/index.js ./node_modules/express/node_modules/cookie-signature/Makefile ./node_modules/express/node_modules/cookie-signature/package.json ./node_modules/express/node_modules/cookie-signature/Readme.md ./node_modules/express/node_modules/debug/.npmignore ./node_modules/express/node_modules/debug/component.json ./node_modules/express/node_modules/debug/debug.js ./node_modules/express/node_modules/debug/example/app.js ./node_modules/express/node_modules/debug/example/browser.html ./node_modules/express/node_modules/debug/example/wildcards.js ./node_modules/express/node_modules/debug/example/worker.js ./node_modules/express/node_modules/debug/History.md ./node_modules/express/node_modules/debug/index.js ./node_modules/express/node_modules/debug/lib/debug.js ./node_modules/express/node_modules/debug/package.json ./node_modules/express/node_modules/debug/Readme.md ./node_modules/express/node_modules/fresh/.npmignore ./node_modules/express/node_modules/fresh/index.js ./node_modules/express/node_modules/fresh/Makefile ./node_modules/express/node_modules/fresh/package.json ./node_modules/express/node_modules/fresh/Readme.md ./node_modules/express/node_modules/methods/index.js ./node_modules/express/node_modules/methods/package.json ./node_modules/express/node_modules/mkdirp/.npmignore ./node_modules/express/node_modules/mkdirp/.travis.yml ./node_modules/express/node_modules/mkdirp/examples/pow.js ./node_modules/express/node_modules/mkdirp/index.js ./node_modules/express/node_modules/mkdirp/LICENSE ./node_modules/express/node_modules/mkdirp/package.json ./node_modules/express/node_modules/mkdirp/README.markdown ./node_modules/express/node_modules/mkdirp/test/chmod.js ./node_modules/express/node_modules/mkdirp/test/clobber.js ./node_modules/express/node_modules/mkdirp/test/mkdirp.js ./node_modules/express/node_modules/mkdirp/test/perm.js ./node_modules/express/node_modules/mkdirp/test/perm_sync.js ./node_modules/express/node_modules/mkdirp/test/race.js ./node_modules/express/node_modules/mkdirp/test/rel.js ./node_modules/express/node_modules/mkdirp/test/return.js ./node_modules/express/node_modules/mkdirp/test/return_sync.js ./node_modules/express/node_modules/mkdirp/test/root.js ./node_modules/express/node_modules/mkdirp/test/sync.js ./node_modules/express/node_modules/mkdirp/test/umask.js ./node_modules/express/node_modules/mkdirp/test/umask_sync.js ./node_modules/express/node_modules/range-parser/.npmignore ./node_modules/express/node_modules/range-parser/History.md ./node_modules/express/node_modules/range-parser/index.js ./node_modules/express/node_modules/range-parser/Makefile ./node_modules/express/node_modules/range-parser/package.json ./node_modules/express/node_modules/range-parser/Readme.md ./node_modules/express/node_modules/send/.npmignore ./node_modules/express/node_modules/send/History.md ./node_modules/express/node_modules/send/index.js ./node_modules/express/node_modules/send/lib/send.js ./node_modules/express/node_modules/send/lib/utils.js ./node_modules/express/node_modules/send/Makefile ./node_modules/express/node_modules/send/node_modules/mime/LICENSE ./node_modules/express/node_modules/send/node_modules/mime/mime.js ./node_modules/express/node_modules/send/node_modules/mime/package.json ./node_modules/express/node_modules/send/node_modules/mime/README.md ./node_modules/express/node_modules/send/node_modules/mime/test.js ./node_modules/express/node_modules/send/node_modules/mime/types/mime.types ./node_modules/express/node_modules/send/node_modules/mime/types/node.types ./node_modules/express/node_modules/send/package.json ./node_modules/express/node_modules/send/Readme.md ./node_modules/express/package.json ./node_modules/express/Readme.md ./node_modules/mpns/lib/mpns.js ./node_modules/mpns/package.json ./node_modules/oauth/index.js ./node_modules/oauth/lib/oauth.js ./node_modules/oauth/lib/oauth2.js ./node_modules/oauth/lib/sha1.js ./node_modules/oauth/lib/_utils.js ./node_modules/oauth/LICENSE ./node_modules/oauth/package.json ./node_modules/pusher/index.js ./node_modules/pusher/lib/pusher.js ./node_modules/pusher/node_modules/request/aws.js ./node_modules/pusher/node_modules/request/aws2.js ./node_modules/pusher/node_modules/request/forever.js ./node_modules/pusher/node_modules/request/LICENSE ./node_modules/pusher/node_modules/request/main.js ./node_modules/pusher/node_modules/request/mimetypes.js ./node_modules/pusher/node_modules/request/oauth.js ./node_modules/pusher/node_modules/request/package.json ./node_modules/pusher/node_modules/request/tunnel.js ./node_modules/pusher/node_modules/request/uuid.js ./node_modules/pusher/node_modules/request/vendor/cookie/index.js ./node_modules/pusher/node_modules/request/vendor/cookie/jar.js ./node_modules/pusher/package.json ./node_modules/request/forever.js ./node_modules/request/LICENSE ./node_modules/request/main.js ./node_modules/request/mimetypes.js ./node_modules/request/oauth.js ./node_modules/request/package.json ./node_modules/request/uuid.js ./node_modules/request/vendor/cookie/index.js ./node_modules/request/vendor/cookie/jar.js ./node_modules/sax/lib/sax.js ./node_modules/sax/LICENSE ./node_modules/sax/package.json ./node_modules/sendgrid/index.js ./node_modules/sendgrid/lib/email.js ./node_modules/sendgrid/lib/file_handler.js ./node_modules/sendgrid/lib/sendgrid.js ./node_modules/sendgrid/lib/smtpapi_headers.js ./node_modules/sendgrid/lib/validation.js ./node_modules/sendgrid/MIT.LICENSE ./node_modules/sendgrid/node_modules/mime/LICENSE ./node_modules/sendgrid/node_modules/mime/mime.js ./node_modules/sendgrid/node_modules/mime/package.json ./node_modules/sendgrid/node_modules/mime/types/mime.types ./node_modules/sendgrid/node_modules/mime/types/node.types ./node_modules/sendgrid/node_modules/nodemailer/lib/engines/sendmail.js ./node_modules/sendgrid/node_modules/nodemailer/lib/engines/ses.js ./node_modules/sendgrid/node_modules/nodemailer/lib/engines/smtp.js ./node_modules/sendgrid/node_modules/nodemailer/lib/engines/stub.js ./node_modules/sendgrid/node_modules/nodemailer/lib/helpers.js ./node_modules/sendgrid/node_modules/nodemailer/lib/nodemailer.js ./node_modules/sendgrid/node_modules/nodemailer/lib/transport.js ./node_modules/sendgrid/node_modules/nodemailer/lib/wellknown.js ./node_modules/sendgrid/node_modules/nodemailer/lib/xoauth.js ./node_modules/sendgrid/node_modules/nodemailer/LICENSE ./node_modules/sendgrid/node_modules/nodemailer/node_modules/mailcomposer/lib/dkim.js ./node_modules/sendgrid/node_modules/nodemailer/node_modules/mailcomposer/lib/mailcomposer.js ./node_modules/sendgrid/node_modules/nodemailer/node_modules/mailcomposer/lib/punycode.js ./node_modules/sendgrid/node_modules/nodemailer/node_modules/mailcomposer/lib/urlfetch.js ./node_modules/sendgrid/node_modules/nodemailer/node_modules/mailcomposer/LICENSE ./node_modules/sendgrid/node_modules/nodemailer/node_modules/mailcomposer/node_modules/mimelib-noiconv/content-types.js ./node_modules/sendgrid/node_modules/nodemailer/node_modules/mailcomposer/node_modules/mimelib-noiconv/index.js ./node_modules/sendgrid/node_modules/nodemailer/node_modules/mailcomposer/node_modules/mimelib-noiconv/LICENSE ./node_modules/sendgrid/node_modules/nodemailer/node_modules/mailcomposer/node_modules/mimelib-noiconv/mime-functions.js ./node_modules/sendgrid/node_modules/nodemailer/node_modules/mailcomposer/node_modules/mimelib-noiconv/package.json ./node_modules/sendgrid/node_modules/nodemailer/node_modules/mailcomposer/package.json ./node_modules/sendgrid/node_modules/nodemailer/node_modules/simplesmtp/index.js ./node_modules/sendgrid/node_modules/nodemailer/node_modules/simplesmtp/lib/client.js ./node_modules/sendgrid/node_modules/nodemailer/node_modules/simplesmtp/lib/pool.js ./node_modules/sendgrid/node_modules/nodemailer/node_modules/simplesmtp/lib/server.js ./node_modules/sendgrid/node_modules/nodemailer/node_modules/simplesmtp/lib/starttls.js ./node_modules/sendgrid/node_modules/nodemailer/node_modules/simplesmtp/LICENSE ./node_modules/sendgrid/node_modules/nodemailer/node_modules/simplesmtp/node_modules/rai/cert/cert.pem ./node_modules/sendgrid/node_modules/nodemailer/node_modules/simplesmtp/node_modules/rai/cert/key.pem ./node_modules/sendgrid/node_modules/nodemailer/node_modules/simplesmtp/node_modules/rai/lib/mockup.js ./node_modules/sendgrid/node_modules/nodemailer/node_modules/simplesmtp/node_modules/rai/lib/rai.js ./node_modules/sendgrid/node_modules/nodemailer/node_modules/simplesmtp/node_modules/rai/lib/starttls.js ./node_modules/sendgrid/node_modules/nodemailer/node_modules/simplesmtp/node_modules/rai/LICENSE ./node_modules/sendgrid/node_modules/nodemailer/node_modules/simplesmtp/node_modules/rai/package.json ./node_modules/sendgrid/node_modules/nodemailer/node_modules/simplesmtp/package.json ./node_modules/sendgrid/node_modules/nodemailer/package.json ./node_modules/sendgrid/node_modules/step/lib/step.js ./node_modules/sendgrid/node_modules/step/package.json ./node_modules/sendgrid/node_modules/underscore/index.js ./node_modules/sendgrid/node_modules/underscore/LICENSE ./node_modules/sendgrid/node_modules/underscore/package.json ./node_modules/sendgrid/node_modules/underscore/underscore.js ./node_modules/sendgrid/package.json ./node_modules/sqlserver/lib/sql.js ./node_modules/sqlserver/lib/sqlserver.native.js ./node_modules/sqlserver/lib/sqlserver.node ./node_modules/sqlserver/package.json ./node_modules/tripwire/lib/native/windows/x86/tripwire.node ./node_modules/tripwire/lib/tripwire.js ./node_modules/tripwire/LICENSE.txt ./node_modules/tripwire/package.json ./node_modules/underscore/LICENSE ./node_modules/underscore/package.json ./node_modules/underscore/underscore.js ./node_modules/underscore.string/lib/underscore.string.js ./node_modules/underscore.string/package.json ./node_modules/wns/lib/wns.js ./node_modules/wns/LICENSE.txt ./node_modules/wns/package.json ./node_modules/xml2js/lib/xml2js.js ./node_modules/xml2js/LICENSE ./node_modules/xml2js/node_modules/sax/lib/sax.js ./node_modules/xml2js/node_modules/sax/LICENSE ./node_modules/xml2js/node_modules/sax/package.json ./node_modules/xml2js/package.json ./node_modules/xmlbuilder/lib/index.js ./node_modules/xmlbuilder/lib/XMLBuilder.js ./node_modules/xmlbuilder/lib/XMLFragment.js ./node_modules/xmlbuilder/package.json ./runtime/core.js ./runtime/filehelpers.js ./runtime/jsonwebtoken.js ./runtime/logger.js ./runtime/logwriter.js ./runtime/metrics.js ./runtime/query/expressions.js ./runtime/query/expressionvisitor.js ./runtime/query/queryparser.js ./runtime/request/authentication/facebook.js ./runtime/request/authentication/google.js ./runtime/request/authentication/microsoftaccount.js ./runtime/request/authentication/twitter.js ./runtime/request/dataoperation.js ./runtime/request/datapipeline.js ./runtime/request/html/corshelper.js ./runtime/request/html/crossdomainhandler.js ./runtime/request/html/templates/crossdomainbridge.html ./runtime/request/html/templates/loginviaiframe.html ./runtime/request/html/templates/loginviaiframereceiver.html ./runtime/request/html/templates/loginviapostmessage.html ./runtime/request/html/templating.js ./runtime/request/loginhandler.js ./runtime/request/middleware/allowHandler.js ./runtime/request/middleware/authenticate.js ./runtime/request/middleware/authorize.js ./runtime/request/middleware/bodyParser.js ./runtime/request/middleware/errorHandler.js ./runtime/request/middleware/requestLimit.js ./runtime/request/request.js ./runtime/request/requesthandler.js ./runtime/request/schedulerhandler.js ./runtime/request/statushandler.js ./runtime/request/tablehandler.js ./runtime/resources.js ./runtime/script/apibuilder.js ./runtime/script/metadata.js ./runtime/script/push/notify-apns.js ./runtime/script/push/notify-gcm.js ./runtime/script/push/notify-mpns.js ./runtime/script/push/notify-wns.js ./runtime/script/push/notify.js ./runtime/script/scriptcache.js ./runtime/script/scripterror.js ./runtime/script/scriptloader.js ./runtime/script/scriptmanager.js ./runtime/script/scriptstate.js ./runtime/script/sqladapter.js ./runtime/script/table.js ./runtime/server.js ./runtime/statuscodes.js ./runtime/storage/sqlbooleanizer.js ./runtime/storage/sqlformatter.js ./runtime/storage/sqlhelpers.js ./runtime/storage/storage.js ./runtime/Zumo.Node.js ./static/client/MobileServices.Web-1.0.0.js ./static/client/MobileServices.Web-1.0.0.min.js ./static/default.htm ./static/robots.txt ./Web.config

Read the article

Service Discovery in WCF 4.0 – Part 1

- by Shaun

When designing a service oriented architecture (SOA) system, there will be a lot of services with many service contracts, endpoints and behaviors. Besides the client calling the service, in a large distributed system a service may invoke other services. In this case, one service might need to know the endpoints it invokes. This might not be a problem in a small system. But when you have more than 10 services this might be a problem. For example in my current product, there are around 10 services, such as the user authentication service, UI integration service, location service, license service, device monitor service, event monitor service, schedule job service, accounting service, player management service, etc.. Benefit of Discovery Service Since almost all my services need to invoke at least one other service. This would be a difficult task to make sure all services endpoints are configured correctly in every service. And furthermore, it would be a nightmare when a service changed its endpoint at runtime. Hence, we need a discovery service to remove the dependency (configuration dependency). A discovery service plays as a service dictionary which stores the relationship between the contracts and the endpoints for every service. By using the discovery service, when service X wants to invoke service Y, it just need to ask the discovery service where is service Y, then the discovery service will return all proper endpoints of service Y, then service X can use the endpoint to send the request to service Y. And when some services changed their endpoint address, all need to do is to update its records in the discovery service then all others will know its new endpoint. In WCF 4.0 Discovery it supports both managed proxy discovery mode and ad-hoc discovery mode. In ad-hoc mode there is no standalone discovery service. When a client wanted to invoke a service, it will broadcast an message (normally in UDP protocol) to the entire network with the service match criteria. All services which enabled the discovery behavior will receive this message and only those matched services will send their endpoint back to the client. The managed proxy discovery service works as I described above. In this post I will only cover the managed proxy mode, where there’s a discovery service. For more information about the ad-hoc mode please refer to the MSDN. Service Announcement and Probe The main functionality of discovery service should be return the proper endpoint addresses back to the service who is looking for. In most cases the consume service (as a client) will send the contract which it wanted to request to the discovery service. And then the discovery service will find the endpoint and respond. Sometimes the contract and endpoint are not enough. It also contains versioning, extensions attributes. This post I will only cover the case includes contract and endpoint. When a client (or sometimes a service who need to invoke another service) need to connect to a target service, it will firstly request the discovery service through the “Probe” method with the criteria. Basically the criteria contains the contract type name of the target service. Then the discovery service will search its endpoint repository by the criteria. The repository might be a database, a distributed cache or a flat XML file. If it matches, the discovery service will grab the endpoint information (it’s called discovery endpoint metadata in WCF) and send back. And this is called “Probe”. Finally the client received the discovery endpoint metadata and will use the endpoint to connect to the target service. Besides the probe, discovery service should take the responsible to know there is a new service available when it goes online, as well as stopped when it goes offline. This feature is named “Announcement”. When a service started and stopped, it will announce to the discovery service. So the basic functionality of a discovery service should includes: 1, An endpoint which receive the service online message, and add the service endpoint information in the discovery repository. 2, An endpoint which receive the service offline message, and remove the service endpoint information from the discovery repository. 3, An endpoint which receive the client probe message, and return the matches service endpoints, and return the discovery endpoint metadata. WCF 4.0 discovery service just covers all these features in it's infrastructure classes. Discovery Service in WCF 4.0 WCF 4.0 introduced a new assembly named System.ServiceModel.Discovery which has all necessary classes and interfaces to build a WS-Discovery compliant discovery service. It supports ad-hoc and managed proxy modes. For the case mentioned in this post, what we need to build is a standalone discovery service, which is the managed proxy discovery service mode. To build a managed discovery service in WCF 4.0 just create a new class inherits from the abstract class System.ServiceModel.Discovery.DiscoveryProxy. This class implemented and abstracted the procedures of service announcement and probe. And it exposes 8 abstract methods where we can implement our own endpoint register, unregister and find logic. These 8 methods are asynchronized, which means all invokes to the discovery service are asynchronously, for better service capability and performance. 1, OnBeginOnlineAnnouncement, OnEndOnlineAnnouncement: Invoked when a service sent the online announcement message. We need to add the endpoint information to the repository in this method. 2, OnBeginOfflineAnnouncement, OnEndOfflineAnnouncement: Invoked when a service sent the offline announcement message. We need to remove the endpoint information from the repository in this method. 3, OnBeginFind, OnEndFind: Invoked when a client sent the probe message that want to find the service endpoint information. We need to look for the proper endpoints by matching the client’s criteria through the repository in this method. 4, OnBeginResolve, OnEndResolve: Invoked then a client sent the resolve message. Different from the find method, when using resolve method the discovery service will return the exactly one service endpoint metadata to the client. In our example we will NOT implement this method. Let’s create our own discovery service, inherit the base System.ServiceModel.Discovery.DiscoveryProxy. We also need to specify the service behavior in this class. Since the build-in discovery service host class only support the singleton mode, we must set its instance context mode to single. 1: using System; 2: using System.Collections.Generic; 3: using System.Linq; 4: using System.Text; 5: using System.ServiceModel.Discovery; 6: using System.ServiceModel; 7: 8: namespace Phare.Service 9: { 10: [ServiceBehavior(InstanceContextMode = InstanceContextMode.Single, ConcurrencyMode = ConcurrencyMode.Multiple)] 11: public class ManagedProxyDiscoveryService : DiscoveryProxy 12: { 13: protected override IAsyncResult OnBeginFind(FindRequestContext findRequestContext, AsyncCallback callback, object state) 14: { 15: throw new NotImplementedException(); 16: } 17: 18: protected override IAsyncResult OnBeginOfflineAnnouncement(DiscoveryMessageSequence messageSequence, EndpointDiscoveryMetadata endpointDiscoveryMetadata, AsyncCallback callback, object state) 19: { 20: throw new NotImplementedException(); 21: } 22: 23: protected override IAsyncResult OnBeginOnlineAnnouncement(DiscoveryMessageSequence messageSequence, EndpointDiscoveryMetadata endpointDiscoveryMetadata, AsyncCallback callback, object state) 24: { 25: throw new NotImplementedException(); 26: } 27: 28: protected override IAsyncResult OnBeginResolve(ResolveCriteria resolveCriteria, AsyncCallback callback, object state) 29: { 30: throw new NotImplementedException(); 31: } 32: 33: protected override void OnEndFind(IAsyncResult result) 34: { 35: throw new NotImplementedException(); 36: } 37: 38: protected override void OnEndOfflineAnnouncement(IAsyncResult result) 39: { 40: throw new NotImplementedException(); 41: } 42: 43: protected override void OnEndOnlineAnnouncement(IAsyncResult result) 44: { 45: throw new NotImplementedException(); 46: } 47: 48: protected override EndpointDiscoveryMetadata OnEndResolve(IAsyncResult result) 49: { 50: throw new NotImplementedException(); 51: } 52: } 53: } Then let’s implement the online, offline and find methods one by one. WCF discovery service gives us full flexibility to implement the endpoint add, remove and find logic. For the demo purpose we will use an internal dictionary to store the services’ endpoint metadata. In the next post we will see how to serialize and store these information in database. Define a concurrent dictionary inside the service class since our it will be used in the multiple threads scenario. 1: [ServiceBehavior(InstanceContextMode = InstanceContextMode.Single, ConcurrencyMode = ConcurrencyMode.Multiple)] 2: public class ManagedProxyDiscoveryService : DiscoveryProxy 3: { 4: private ConcurrentDictionary<EndpointAddress, EndpointDiscoveryMetadata> _services; 5: 6: public ManagedProxyDiscoveryService() 7: { 8: _services = new ConcurrentDictionary<EndpointAddress, EndpointDiscoveryMetadata>(); 9: } 10: } Then we can simply implement the logic of service online and offline. 1: protected override IAsyncResult OnBeginOnlineAnnouncement(DiscoveryMessageSequence messageSequence, EndpointDiscoveryMetadata endpointDiscoveryMetadata, AsyncCallback callback, object state) 2: { 3: _services.AddOrUpdate(endpointDiscoveryMetadata.Address, endpointDiscoveryMetadata, (key, value) => endpointDiscoveryMetadata); 4: return new OnOnlineAnnouncementAsyncResult(callback, state); 5: } 6: 7: protected override void OnEndOnlineAnnouncement(IAsyncResult result) 8: { 9: OnOnlineAnnouncementAsyncResult.End(result); 10: } 11: 12: protected override IAsyncResult OnBeginOfflineAnnouncement(DiscoveryMessageSequence messageSequence, EndpointDiscoveryMetadata endpointDiscoveryMetadata, AsyncCallback callback, object state) 13: { 14: EndpointDiscoveryMetadata endpoint = null; 15: _services.TryRemove(endpointDiscoveryMetadata.Address, out endpoint); 16: return new OnOfflineAnnouncementAsyncResult(callback, state); 17: } 18: 19: protected override void OnEndOfflineAnnouncement(IAsyncResult result) 20: { 21: OnOfflineAnnouncementAsyncResult.End(result); 22: } Regards the find method, the parameter FindRequestContext.Criteria has a method named IsMatch, which can be use for us to evaluate which service metadata is satisfied with the criteria. So the implementation of find method would be like this. 1: protected override IAsyncResult OnBeginFind(FindRequestContext findRequestContext, AsyncCallback callback, object state) 2: { 3: _services.Where(s => findRequestContext.Criteria.IsMatch(s.Value)) 4: .Select(s => s.Value) 5: .All(meta => 6: { 7: findRequestContext.AddMatchingEndpoint(meta); 8: return true; 9: }); 10: return new OnFindAsyncResult(callback, state); 11: } 12: 13: protected override void OnEndFind(IAsyncResult result) 14: { 15: OnFindAsyncResult.End(result); 16: } As you can see, we checked all endpoints metadata in repository by invoking the IsMatch method. Then add all proper endpoints metadata into the parameter. Finally since all these methods are asynchronized we need some AsyncResult classes as well. Below are the base class and the inherited classes used in previous methods. 1: using System; 2: using System.Collections.Generic; 3: using System.Linq; 4: using System.Text; 5: using System.Threading; 6: 7: namespace Phare.Service 8: { 9: abstract internal class AsyncResult : IAsyncResult 10: { 11: AsyncCallback callback; 12: bool completedSynchronously; 13: bool endCalled; 14: Exception exception; 15: bool isCompleted; 16: ManualResetEvent manualResetEvent; 17: object state; 18: object thisLock; 19: 20: protected AsyncResult(AsyncCallback callback, object state) 21: { 22: this.callback = callback; 23: this.state = state; 24: this.thisLock = new object(); 25: } 26: 27: public object AsyncState 28: { 29: get 30: { 31: return state; 32: } 33: } 34: 35: public WaitHandle AsyncWaitHandle 36: { 37: get 38: { 39: if (manualResetEvent != null) 40: { 41: return manualResetEvent; 42: } 43: lock (ThisLock) 44: { 45: if (manualResetEvent == null) 46: { 47: manualResetEvent = new ManualResetEvent(isCompleted); 48: } 49: } 50: return manualResetEvent; 51: } 52: } 53: 54: public bool CompletedSynchronously 55: { 56: get 57: { 58: return completedSynchronously; 59: } 60: } 61: 62: public bool IsCompleted 63: { 64: get 65: { 66: return isCompleted; 67: } 68: } 69: 70: object ThisLock 71: { 72: get 73: { 74: return this.thisLock; 75: } 76: } 77: 78: protected static TAsyncResult End<TAsyncResult>(IAsyncResult result) 79: where TAsyncResult : AsyncResult 80: { 81: if (result == null) 82: { 83: throw new ArgumentNullException("result"); 84: } 85: 86: TAsyncResult asyncResult = result as TAsyncResult; 87: 88: if (asyncResult == null) 89: { 90: throw new ArgumentException("Invalid async result.", "result"); 91: } 92: 93: if (asyncResult.endCalled) 94: { 95: throw new InvalidOperationException("Async object already ended."); 96: } 97: 98: asyncResult.endCalled = true; 99: 100: if (!asyncResult.isCompleted) 101: { 102: asyncResult.AsyncWaitHandle.WaitOne(); 103: } 104: 105: if (asyncResult.manualResetEvent != null) 106: { 107: asyncResult.manualResetEvent.Close(); 108: } 109: 110: if (asyncResult.exception != null) 111: { 112: throw asyncResult.exception; 113: } 114: 115: return asyncResult; 116: } 117: 118: protected void Complete(bool completedSynchronously) 119: { 120: if (isCompleted) 121: { 122: throw new InvalidOperationException("This async result is already completed."); 123: } 124: 125: this.completedSynchronously = completedSynchronously; 126: 127: if (completedSynchronously) 128: { 129: this.isCompleted = true; 130: } 131: else 132: { 133: lock (ThisLock) 134: { 135: this.isCompleted = true; 136: if (this.manualResetEvent != null) 137: { 138: this.manualResetEvent.Set(); 139: } 140: } 141: } 142: 143: if (callback != null) 144: { 145: callback(this); 146: } 147: } 148: 149: protected void Complete(bool completedSynchronously, Exception exception) 150: { 151: this.exception = exception; 152: Complete(completedSynchronously); 153: } 154: } 155: } 1: using System; 2: using System.Collections.Generic; 3: using System.Linq; 4: using System.Text; 5: using System.ServiceModel.Discovery; 6: using Phare.Service; 7: 8: namespace Phare.Service 9: { 10: internal sealed class OnOnlineAnnouncementAsyncResult : AsyncResult 11: { 12: public OnOnlineAnnouncementAsyncResult(AsyncCallback callback, object state) 13: : base(callback, state) 14: { 15: this.Complete(true); 16: } 17: 18: public static void End(IAsyncResult result) 19: { 20: AsyncResult.End<OnOnlineAnnouncementAsyncResult>(result); 21: } 22: 23: } 24: 25: sealed class OnOfflineAnnouncementAsyncResult : AsyncResult 26: { 27: public OnOfflineAnnouncementAsyncResult(AsyncCallback callback, object state) 28: : base(callback, state) 29: { 30: this.Complete(true); 31: } 32: 33: public static void End(IAsyncResult result) 34: { 35: AsyncResult.End<OnOfflineAnnouncementAsyncResult>(result); 36: } 37: } 38: 39: sealed class OnFindAsyncResult : AsyncResult 40: { 41: public OnFindAsyncResult(AsyncCallback callback, object state) 42: : base(callback, state) 43: { 44: this.Complete(true); 45: } 46: 47: public static void End(IAsyncResult result) 48: { 49: AsyncResult.End<OnFindAsyncResult>(result); 50: } 51: } 52: 53: sealed class OnResolveAsyncResult : AsyncResult 54: { 55: EndpointDiscoveryMetadata matchingEndpoint; 56: 57: public OnResolveAsyncResult(EndpointDiscoveryMetadata matchingEndpoint, AsyncCallback callback, object state) 58: : base(callback, state) 59: { 60: this.matchingEndpoint = matchingEndpoint; 61: this.Complete(true); 62: } 63: 64: public static EndpointDiscoveryMetadata End(IAsyncResult result) 65: { 66: OnResolveAsyncResult thisPtr = AsyncResult.End<OnResolveAsyncResult>(result); 67: return thisPtr.matchingEndpoint; 68: } 69: } 70: } Now we have finished the discovery service. The next step is to host it. The discovery service is a standard WCF service. So we can use ServiceHost on a console application, windows service, or in IIS as usual. The following code is how to host the discovery service we had just created in a console application. 1: static void Main(string[] args) 2: { 3: using (var host = new ServiceHost(new ManagedProxyDiscoveryService())) 4: { 5: host.Opened += (sender, e) => 6: { 7: host.Description.Endpoints.All((ep) => 8: { 9: Console.WriteLine(ep.ListenUri); 10: return true; 11: }); 12: }; 13: 14: try 15: { 16: // retrieve the announcement, probe endpoint and binding from configuration 17: var announcementEndpointAddress = new EndpointAddress(ConfigurationManager.AppSettings["announcementEndpointAddress"]); 18: var probeEndpointAddress = new EndpointAddress(ConfigurationManager.AppSettings["probeEndpointAddress"]); 19: var binding = Activator.CreateInstance(Type.GetType(ConfigurationManager.AppSettings["bindingType"], true, true)) as Binding; 20: var announcementEndpoint = new AnnouncementEndpoint(binding, announcementEndpointAddress); 21: var probeEndpoint = new DiscoveryEndpoint(binding, probeEndpointAddress); 22: probeEndpoint.IsSystemEndpoint = false; 23: // append the service endpoint for announcement and probe 24: host.AddServiceEndpoint(announcementEndpoint); 25: host.AddServiceEndpoint(probeEndpoint); 26: 27: host.Open(); 28: 29: Console.WriteLine("Press any key to exit."); 30: Console.ReadKey(); 31: } 32: catch (Exception ex) 33: { 34: Console.WriteLine(ex.ToString()); 35: } 36: } 37: 38: Console.WriteLine("Done."); 39: Console.ReadKey(); 40: } What we need to notice is that, the discovery service needs two endpoints for announcement and probe. In this example I just retrieve them from the configuration file. I also specified the binding of these two endpoints in configuration file as well. 1: <?xml version="1.0"?> 2: <configuration> 3: <startup> 4: <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0"/> 5: </startup> 6: <appSettings> 7: <add key="announcementEndpointAddress" value="net.tcp://localhost:10010/announcement"/> 8: <add key="probeEndpointAddress" value="net.tcp://localhost:10011/probe"/> 9: <add key="bindingType" value="System.ServiceModel.NetTcpBinding, System.ServiceModel, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089"/> 10: </appSettings> 11: </configuration> And this is the console screen when I ran my discovery service. As you can see there are two endpoints listening for announcement message and probe message. Discoverable Service and Client Next, let’s create a WCF service that is discoverable, which means it can be found by the discovery service. To do so, we need to let the service send the online announcement message to the discovery service, as well as offline message before it shutdown. Just create a simple service which can make the incoming string to upper. The service contract and implementation would be like this. 1: [ServiceContract] 2: public interface IStringService 3: { 4: [OperationContract] 5: string ToUpper(string content); 6: } 1: public class StringService : IStringService 2: { 3: public string ToUpper(string content) 4: { 5: return content.ToUpper(); 6: } 7: } Then host this service in the console application. In order to make the discovery service easy to be tested the service address will be changed each time it’s started. 1: static void Main(string[] args) 2: { 3: var baseAddress = new Uri(string.Format("net.tcp://localhost:11001/stringservice/{0}/", Guid.NewGuid().ToString())); 4: 5: using (var host = new ServiceHost(typeof(StringService), baseAddress)) 6: { 7: host.Opened += (sender, e) => 8: { 9: Console.WriteLine("Service opened at {0}", host.Description.Endpoints.First().ListenUri); 10: }; 11: 12: host.AddServiceEndpoint(typeof(IStringService), new NetTcpBinding(), string.Empty); 13: 14: host.Open(); 15: 16: Console.WriteLine("Press any key to exit."); 17: Console.ReadKey(); 18: } 19: } Currently this service is NOT discoverable. We need to add a special service behavior so that it could send the online and offline message to the discovery service announcement endpoint when the host is opened and closed. WCF 4.0 introduced a service behavior named ServiceDiscoveryBehavior. When we specified the announcement endpoint address and appended it to the service behaviors this service will be discoverable. 1: var announcementAddress = new EndpointAddress(ConfigurationManager.AppSettings["announcementEndpointAddress"]); 2: var announcementBinding = Activator.CreateInstance(Type.GetType(ConfigurationManager.AppSettings["bindingType"], true, true)) as Binding; 3: var announcementEndpoint = new AnnouncementEndpoint(announcementBinding, announcementAddress); 4: var discoveryBehavior = new ServiceDiscoveryBehavior(); 5: discoveryBehavior.AnnouncementEndpoints.Add(announcementEndpoint); 6: host.Description.Behaviors.Add(discoveryBehavior); The ServiceDiscoveryBehavior utilizes the service extension and channel dispatcher to implement the online and offline announcement logic. In short, it injected the channel open and close procedure and send the online and offline message to the announcement endpoint. On client side, when we have the discovery service, a client can invoke a service without knowing its endpoint. WCF discovery assembly provides a class named DiscoveryClient, which can be used to find the proper service endpoint by passing the criteria. In the code below I initialized the DiscoveryClient, specified the discovery service probe endpoint address. Then I created the find criteria by specifying the service contract I wanted to use and invoke the Find method. This will send the probe message to the discovery service and it will find the endpoints back to me. The discovery service will return all endpoints that matches the find criteria, which means in the result of the find method there might be more than one endpoints. In this example I just returned the first matched one back. In the next post I will show how to extend our discovery service to make it work like a service load balancer. 1: static EndpointAddress FindServiceEndpoint() 2: { 3: var probeEndpointAddress = new EndpointAddress(ConfigurationManager.AppSettings["probeEndpointAddress"]); 4: var probeBinding = Activator.CreateInstance(Type.GetType(ConfigurationManager.AppSettings["bindingType"], true, true)) as Binding; 5: var discoveryEndpoint = new DiscoveryEndpoint(probeBinding, probeEndpointAddress); 6: 7: EndpointAddress address = null; 8: FindResponse result = null; 9: using (var discoveryClient = new DiscoveryClient(discoveryEndpoint)) 10: { 11: result = discoveryClient.Find(new FindCriteria(typeof(IStringService))); 12: } 13: 14: if (result != null && result.Endpoints.Any()) 15: { 16: var endpointMetadata = result.Endpoints.First(); 17: address = endpointMetadata.Address; 18: } 19: return address; 20: } Once we probed the discovery service we will receive the endpoint. So in the client code we can created the channel factory from the endpoint and binding, and invoke to the service. When creating the client side channel factory we need to make sure that the client side binding should be the same as the service side. WCF discovery service can be used to find the endpoint for a service contract, but the binding is NOT included. This is because the binding was not in the WS-Discovery specification. In the next post I will demonstrate how to add the binding information into the discovery service. At that moment the client don’t need to create the binding by itself. Instead it will use the binding received from the discovery service. 1: static void Main(string[] args) 2: { 3: Console.WriteLine("Say something..."); 4: var content = Console.ReadLine(); 5: while (!string.IsNullOrWhiteSpace(content)) 6: { 7: Console.WriteLine("Finding the service endpoint..."); 8: var address = FindServiceEndpoint(); 9: if (address == null) 10: { 11: Console.WriteLine("There is no endpoint matches the criteria."); 12: } 13: else 14: { 15: Console.WriteLine("Found the endpoint {0}", address.Uri); 16: 17: var factory = new ChannelFactory<IStringService>(new NetTcpBinding(), address); 18: factory.Opened += (sender, e) => 19: { 20: Console.WriteLine("Connecting to {0}.", factory.Endpoint.ListenUri); 21: }; 22: var proxy = factory.CreateChannel(); 23: using (proxy as IDisposable) 24: { 25: Console.WriteLine("ToUpper: {0} => {1}", content, proxy.ToUpper(content)); 26: } 27: } 28: 29: Console.WriteLine("Say something..."); 30: content = Console.ReadLine(); 31: } 32: } Similarly, the discovery service probe endpoint and binding were defined in the configuration file. 1: <?xml version="1.0"?> 2: <configuration> 3: <startup> 4: <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0"/> 5: </startup> 6: <appSettings> 7: <add key="announcementEndpointAddress" value="net.tcp://localhost:10010/announcement"/> 8: <add key="probeEndpointAddress" value="net.tcp://localhost:10011/probe"/> 9: <add key="bindingType" value="System.ServiceModel.NetTcpBinding, System.ServiceModel, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089"/> 10: </appSettings> 11: </configuration> OK, now let’s have a test. Firstly start the discovery service, and then start our discoverable service. When it started it will announced to the discovery service and registered its endpoint into the repository, which is the local dictionary. And then start the client and type something. As you can see the client asked the discovery service for the endpoint and then establish the connection to the discoverable service. And more interesting, do NOT close the client console but terminate the discoverable service but press the enter key. This will make the service send the offline message to the discovery service. Then start the discoverable service again. Since we made it use a different address each time it started, currently it should be hosted on another address. If we enter something in the client we could see that it asked the discovery service and retrieve the new endpoint, and connect the the service. Summary In this post I discussed the benefit of using the discovery service and the procedures of service announcement and probe. I also demonstrated how to leverage the WCF Discovery feature in WCF 4.0 to build a simple managed discovery service. For test purpose, in this example I used the in memory dictionary as the discovery endpoint metadata repository. And when finding I also just return the first matched endpoint back. I also hard coded the bindings between the discoverable service and the client. In next post I will show you how to solve the problem mentioned above, as well as some additional feature for production usage. You can download the code here. Hope this helps, Shaun All documents and related graphics, codes are provided "AS IS" without warranty of any kind. Copyright © Shaun Ziyan Xu. This work is licensed under the Creative Commons License.

Read the article

Slow NFS and GFS2 performance

- by Tiago

Recently I've designed and configured a 4 node cluster for a webapp that does lots of file handling. The cluster have been broken down into 2 main roles, webserver and storage. Each role is replicated to a second server using drbd in active/passive mode. The webserver does a NFS mount of the data directory of the storage server and the latter also has a webserver running to serve files to browser clients. In the storage servers I've created a GFS2 FS to hold the data which is wired to drbd. I've chose GFS2 mainly because the announced performance and also because the volume size which has to be pretty high. Since we entered production I've been facing two problems that I think are deeply connected. First of all, the NFS mount on the webservers keeps hanging for a minute or so and then resumes normal operations. By analyzing the logs I've found out that NFS stops answering for a while and outputs the following log lines: Oct 15 18:15:42 <server hostname> kernel: nfs: server active.storage.vlan not responding, still trying Oct 15 18:15:44 <server hostname> kernel: nfs: server active.storage.vlan not responding, still trying Oct 15 18:15:46 <server hostname> kernel: nfs: server active.storage.vlan not responding, still trying Oct 15 18:15:47 <server hostname> kernel: nfs: server active.storage.vlan not responding, still trying Oct 15 18:15:47 <server hostname> kernel: nfs: server active.storage.vlan not responding, still trying Oct 15 18:15:47 <server hostname> kernel: nfs: server active.storage.vlan not responding, still trying Oct 15 18:15:48 <server hostname> kernel: nfs: server active.storage.vlan not responding, still trying Oct 15 18:15:48 <server hostname> kernel: nfs: server active.storage.vlan not responding, still trying Oct 15 18:15:51 <server hostname> kernel: nfs: server active.storage.vlan not responding, still trying Oct 15 18:15:52 <server hostname> kernel: nfs: server active.storage.vlan not responding, still trying Oct 15 18:15:52 <server hostname> kernel: nfs: server active.storage.vlan not responding, still trying Oct 15 18:15:55 <server hostname> kernel: nfs: server active.storage.vlan not responding, still trying Oct 15 18:15:55 <server hostname> kernel: nfs: server active.storage.vlan not responding, still trying Oct 15 18:15:58 <server hostname> kernel: nfs: server active.storage.vlan OK Oct 15 18:15:59 <server hostname> kernel: nfs: server active.storage.vlan OK Oct 15 18:15:59 <server hostname> kernel: nfs: server active.storage.vlan OK Oct 15 18:15:59 <server hostname> kernel: nfs: server active.storage.vlan OK Oct 15 18:15:59 <server hostname> kernel: nfs: server active.storage.vlan OK Oct 15 18:15:59 <server hostname> kernel: nfs: server active.storage.vlan OK Oct 15 18:15:59 <server hostname> kernel: nfs: server active.storage.vlan OK Oct 15 18:15:59 <server hostname> kernel: nfs: server active.storage.vlan OK Oct 15 18:15:59 <server hostname> kernel: nfs: server active.storage.vlan OK Oct 15 18:15:59 <server hostname> kernel: nfs: server active.storage.vlan OK Oct 15 18:15:59 <server hostname> kernel: nfs: server active.storage.vlan OK Oct 15 18:15:59 <server hostname> kernel: nfs: server active.storage.vlan OK Oct 15 18:15:59 <server hostname> kernel: nfs: server active.storage.vlan OK In this case, the hang lasted for 16 seconds but sometimes it takes 1 or 2 minutes to resume normal operations. My first guess was this was happening due to heavy load of the NFS mount and that by increasing RPCNFSDCOUNT to a higher value, this would become stable. I've increased it several times and apparently, after a while, the logs started appearing less times. The value is now on 32. After further investigating the issue, I've came across a different hang, despite the NFS messages still appear in the logs. Sometimes, the GFS2 FS simply hangs which causes both the NFS and the storage webserver to serve files. Both stay hang for a while and then they resume normal operations. This hangs leaves no trace on client side (also leaves no NFS ... not responding messages) and, on the storage side, the log system appears to be empty, even though the rsyslogd is running. The nodes connect themselves through a 10Gbps non-dedicated connection but I don't think this is an issue because the GFS2 hang is confirmed but connecting directly to the active storage server. I've been trying to solve this for a while now and I've tried different NFS configuration options, before I've found out the GFS2 FS is also hanging. The NFS mount is exported as such: /srv/data/ <ip_address>(rw,async,no_root_squash,no_all_squash,fsid=25) And the NFS client mounts with: mount -o "async,hard,intr,wsize=8192,rsize=8192" active.storage.vlan:/srv/data /srv/data After some tests, these were the configurations that yielded more performance to the cluster. I am desperate to find a solution for this as the cluster is already in production mode and I need to fix this so that this hangs won't happen in the future and I don't really know for sure what and how I should be benchmarking. What I can tell is that this is happening due to heavy loads as I have tested the cluster earlier and this problems weren't happening at all. Please tell me if you need me to provide configuration details of the cluster, and which do you want me to post. As last resort I can migrate the files to a different FS but I need some solid pointers on whether this will solve this problems as the volume size is extremely large at this point. The servers are being hosted by a third-party enterprise and I don't have physical access to them. Best regards. EDIT 1: The servers are physical servers and their specs are: Webservers: Intel Bi Xeon E5606 2x4 2.13GHz 24GB DDR3 Intel SSD 320 2 x 120GB Raid 1 Storage: Intel i5 3550 3.3GHz 16GB DDR3 12 x 2TB SATA Initially there was a VRack setup between the servers but we've upgraded one of the storage servers to have more RAM and it wasn't inside the VRack. They connect through a shared 10Gbps connection between them. Please note that it is the same connection that is used for public access. They use a single IP (using IP Failover) to connect between them and to allow for a graceful failover. NFS is therefore over a public connection and not under any private network (it was before the upgrade, were the problem still existed). The firewall was configured and tested thoroughly but I disabled it for a while to see if the problem still occurred, and it did. From my knowledge the hosting provider isn't blocking or limiting the connection between either the servers and the public domain (at least under a given bandwidth consumption threshold that hasn't been reached yet). Hope this helps figuring out the problem. EDIT 2: Relevant software versions: CentOS 2.6.32-279.9.1.el6.x86_64 nfs-utils-1.2.3-26.el6.x86_64 nfs-utils-lib-1.1.5-4.el6.x86_64 gfs2-utils-3.0.12.1-32.el6_3.1.x86_64 kmod-drbd84-8.4.2-1.el6_3.elrepo.x86_64 drbd84-utils-8.4.2-1.el6.elrepo.x86_64 DRBD configuration on storage servers: #/etc/drbd.d/storage.res resource storage { protocol C; on <server1 fqdn> { device /dev/drbd0; disk /dev/vg_storage/LV_replicated; address <server1 ip>:7788; meta-disk internal; } on <server2 fqdn> { device /dev/drbd0; disk /dev/vg_storage/LV_replicated; address <server2 ip>:7788; meta-disk internal; } } NFS Configuration in storage servers: #/etc/sysconfig/nfs RPCNFSDCOUNT=32 STATD_PORT=10002 STATD_OUTGOING_PORT=10003 MOUNTD_PORT=10004 RQUOTAD_PORT=10005 LOCKD_UDPPORT=30001 LOCKD_TCPPORT=30001 (can there be any conflict in using the same port for both LOCKD_UDPPORT and LOCKD_TCPPORT?) GFS2 configuration: # gfs2_tool gettune <mountpoint> incore_log_blocks = 1024 log_flush_secs = 60 quota_warn_period = 10 quota_quantum = 60 max_readahead = 262144 complain_secs = 10 statfs_slow = 0 quota_simul_sync = 64 statfs_quantum = 30 quota_scale = 1.0000 (1, 1) new_files_jdata = 0 Storage network environment: eth0 Link encap:Ethernet HWaddr <mac address> inet addr:<ip address> Bcast:<bcast address> Mask:<ip mask> inet6 addr: <ip address> Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:957025127 errors:0 dropped:0 overruns:0 frame:0 TX packets:1473338731 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:2630984979622 (2.3 TiB) TX bytes:1648430431523 (1.4 TiB) eth0:0 Link encap:Ethernet HWaddr <mac address> inet addr:<ip failover address> Bcast:<bcast address> Mask:<ip mask> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 The IP addresses are statically assigned with the given network configurations: DEVICE="eth0" BOOTPROTO="static" HWADDR=<mac address> ONBOOT="yes" TYPE="Ethernet" IPADDR=<ip address> NETMASK=<net mask> and DEVICE="eth0:0" BOOTPROTO="static" HWADDR=<mac address> IPADDR=<ip failover> NETMASK=<net mask> ONBOOT="yes" BROADCAST=<bcast address> Hosts file to allow for a graceful NFS failover in conjunction with NFS option fsid=25 set on both storage servers: #/etc/hosts <storage ip failover address> active.storage.vlan <webserver ip failover address> active.service.vlan As you can see, packet errors are down to 0. I've also ran ping for a long time without any packet loss. MTU size is the normal 1500. As there is no VLan by now, this is the MTU used to communicate between servers. The webservers' network environment is similar. One thing I forgot to mention is that the storage servers handle ~200GB of new files each day through the NFS connection, which is a key point for me to think this is some kind of heavy load problem with either NFS or GFS2. If you need further configuration details please tell me. EDIT 3: Earlier today we had a major filesystem crash on the storage server. I couldn't get the details of the crash right away because the server stop responding. After the reboot, I noticed the filesystem was extremely slow, and I was not being able to serve a single file through either NFS or httpd, perhaps due to cache warming or so. Nevertheless, I've been monitoring the server closely and the following error came up in dmesg. The source of the problem is clearly GFS, which is waiting for a lock and ends up starving after a while. INFO: task nfsd:3029 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. nfsd D 0000000000000000 0 3029 2 0x00000080 ffff8803814f79e0 0000000000000046 0000000000000000 ffffffff8109213f ffff880434c5e148 ffff880624508d88 ffff8803814f7960 ffffffffa037253f ffff8803815c1098 ffff8803814f7fd8 000000000000fb88 ffff8803815c1098 Call Trace: [<ffffffff8109213f>] ? wake_up_bit+0x2f/0x40 [<ffffffffa037253f>] ? gfs2_holder_wake+0x1f/0x30 [gfs2] [<ffffffff814ff42e>] __mutex_lock_slowpath+0x13e/0x180 [<ffffffff814ff2cb>] mutex_lock+0x2b/0x50 [<ffffffffa0379f21>] gfs2_log_reserve+0x51/0x190 [gfs2] [<ffffffffa0390da2>] gfs2_trans_begin+0x112/0x1d0 [gfs2] [<ffffffffa0369b05>] ? gfs2_dir_check+0x35/0xe0 [gfs2] [<ffffffffa0377943>] gfs2_createi+0x1a3/0xaa0 [gfs2] [<ffffffff8121aab1>] ? avc_has_perm+0x71/0x90 [<ffffffffa0383d1e>] gfs2_create+0x7e/0x1a0 [gfs2] [<ffffffffa037783f>] ? gfs2_createi+0x9f/0xaa0 [gfs2] [<ffffffff81188cf4>] vfs_create+0xb4/0xe0 [<ffffffffa04217d6>] nfsd_create_v3+0x366/0x4c0 [nfsd] [<ffffffffa0429703>] nfsd3_proc_create+0x123/0x1b0 [nfsd] [<ffffffffa041a43e>] nfsd_dispatch+0xfe/0x240 [nfsd] [<ffffffffa025a5d4>] svc_process_common+0x344/0x640 [sunrpc] [<ffffffff810602a0>] ? default_wake_function+0x0/0x20 [<ffffffffa025ac10>] svc_process+0x110/0x160 [sunrpc] [<ffffffffa041ab62>] nfsd+0xc2/0x160 [nfsd] [<ffffffffa041aaa0>] ? nfsd+0x0/0x160 [nfsd] [<ffffffff81091de6>] kthread+0x96/0xa0 [<ffffffff8100c14a>] child_rip+0xa/0x20 [<ffffffff81091d50>] ? kthread+0x0/0xa0 [<ffffffff8100c140>] ? child_rip+0x0/0x20

Read the article

An Xml Serializable PropertyBag Dictionary Class for .NET

- by Rick Strahl

I don't know about you but I frequently need property bags in my applications to store and possibly cache arbitrary data. Dictionary<T,V> works well for this although I always seem to be hunting for a more specific generic type that provides a string key based dictionary. There's string dictionary, but it only works with strings. There's Hashset<T> but it uses the actual values as keys. In most key value pair situations for me string is key value to work off. Dictionary<T,V> works well enough, but there are some issues with serialization of dictionaries in .NET. The .NET framework doesn't do well serializing IDictionary objects out of the box. The XmlSerializer doesn't support serialization of IDictionary via it's default serialization, and while the DataContractSerializer does support IDictionary serialization it produces some pretty atrocious XML. What doesn't work? First off Dictionary serialization with the Xml Serializer doesn't work so the following fails: [TestMethod] public void DictionaryXmlSerializerTest() { var bag = new Dictionary<string, object>(); bag.Add("key", "Value"); bag.Add("Key2", 100.10M); bag.Add("Key3", Guid.NewGuid()); bag.Add("Key4", DateTime.Now); bag.Add("Key5", true); bag.Add("Key7", new byte[3] { 42, 45, 66 }); TestContext.WriteLine(this.ToXml(bag)); } public string ToXml(object obj) { if (obj == null) return null; StringWriter sw = new StringWriter(); XmlSerializer ser = new XmlSerializer(obj.GetType()); ser.Serialize(sw, obj); return sw.ToString(); } The error you get with this is: System.NotSupportedException: The type System.Collections.Generic.Dictionary`2[[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.Object, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]] is not supported because it implements IDictionary. Got it! BTW, the same is true with binary serialization. Running the same code above against the DataContractSerializer does work: [TestMethod] public void DictionaryDataContextSerializerTest() { var bag = new Dictionary<string, object>(); bag.Add("key", "Value"); bag.Add("Key2", 100.10M); bag.Add("Key3", Guid.NewGuid()); bag.Add("Key4", DateTime.Now); bag.Add("Key5", true); bag.Add("Key7", new byte[3] { 42, 45, 66 }); TestContext.WriteLine(this.ToXmlDcs(bag)); } public string ToXmlDcs(object value, bool throwExceptions = false) { var ser = new DataContractSerializer(value.GetType(), null, int.MaxValue, true, false, null); MemoryStream ms = new MemoryStream(); ser.WriteObject(ms, value); return Encoding.UTF8.GetString(ms.ToArray(), 0, (int)ms.Length); } This DOES work but produces some pretty heinous XML (formatted with line breaks and indentation here): <ArrayOfKeyValueOfstringanyType xmlns="http://schemas.microsoft.com/2003/10/Serialization/Arrays" xmlns:i="http://www.w3.org/2001/XMLSchema-instance"> <KeyValueOfstringanyType> <Key>key</Key> <Value i:type="a:string" xmlns:a="http://www.w3.org/2001/XMLSchema">Value</Value> </KeyValueOfstringanyType> <KeyValueOfstringanyType> <Key>Key2</Key> <Value i:type="a:decimal" xmlns:a="http://www.w3.org/2001/XMLSchema">100.10</Value> </KeyValueOfstringanyType> <KeyValueOfstringanyType> <Key>Key3</Key> <Value i:type="a:guid" xmlns:a="http://schemas.microsoft.com/2003/10/Serialization/">2cd46d2a-a636-4af4-979b-e834d39b6d37</Value> </KeyValueOfstringanyType> <KeyValueOfstringanyType> <Key>Key4</Key> <Value i:type="a:dateTime" xmlns:a="http://www.w3.org/2001/XMLSchema">2011-09-19T17:17:05.4406999-07:00</Value> </KeyValueOfstringanyType> <KeyValueOfstringanyType> <Key>Key5</Key> <Value i:type="a:boolean" xmlns:a="http://www.w3.org/2001/XMLSchema">true</Value> </KeyValueOfstringanyType> <KeyValueOfstringanyType> <Key>Key7</Key> <Value i:type="a:base64Binary" xmlns:a="http://www.w3.org/2001/XMLSchema">Ki1C</Value> </KeyValueOfstringanyType> </ArrayOfKeyValueOfstringanyType> Ouch! That seriously hurts the eye! :-) Worse though it's extremely verbose with all those repetitive namespace declarations. It's good to know that it works in a pinch, but for a human readable/editable solution or something lightweight to store in a database it's not quite ideal. Why should I care? As a little background, in one of my applications I have a need for a flexible property bag that is used on a free form database field on an otherwise static entity. Basically what I have is a standard database record to which arbitrary properties can be added in an XML based string field. I intend to expose those arbitrary properties as a collection from field data stored in XML. The concept is pretty simple: When loading write the data to the collection, when the data is saved serialize the data into an XML string and store it into the database. When reading the data pick up the XML and if the collection on the entity is accessed automatically deserialize the XML into the Dictionary. (I'll talk more about this in another post). While the DataContext Serializer would work, it's verbosity is problematic both for size of the generated XML strings and the fact that users can manually edit this XML based property data in an advanced mode. A clean(er) layout certainly would be preferable and more user friendly. Custom XMLSerialization with a PropertyBag Class So… after a bunch of experimentation with different serialization formats I decided to create a custom PropertyBag class that provides for a serializable Dictionary. It's basically a custom Dictionary<TType,TValue> implementation with the keys always set as string keys. The result are PropertyBag<TValue> and PropertyBag (which defaults to the object type for values). The PropertyBag<TType> and PropertyBag classes provide these features: Subclassed from Dictionary<T,V> Implements IXmlSerializable with a cleanish XML format ToXml() and FromXml() methods to export and import to and from XML strings Static CreateFromXml() method to create an instance It's simple enough as it's merely a Dictionary<string,object> subclass but that supports serialization to a - what I think at least - cleaner XML format. The class is super simple to use: [TestMethod] public void PropertyBagTwoWayObjectSerializationTest() { var bag = new PropertyBag(); bag.Add("key", "Value"); bag.Add("Key2", 100.10M); bag.Add("Key3", Guid.NewGuid()); bag.Add("Key4", DateTime.Now); bag.Add("Key5", true); bag.Add("Key7", new byte[3] { 42,45,66 } ); bag.Add("Key8", null); bag.Add("Key9", new ComplexObject() { Name = "Rick", Entered = DateTime.Now, Count = 10 }); string xml = bag.ToXml(); TestContext.WriteLine(bag.ToXml()); bag.Clear(); bag.FromXml(xml); Assert.IsTrue(bag["key"] as string == "Value"); Assert.IsInstanceOfType( bag["Key3"], typeof(Guid)); Assert.IsNull(bag["Key8"]); //Assert.IsNull(bag["Key10"]); Assert.IsInstanceOfType(bag["Key9"], typeof(ComplexObject)); } This uses the PropertyBag class which uses a PropertyBag<string,object> - which means it returns untyped values of type object. I suspect for me this will be the most common scenario as I'd want to store arbitrary values in the PropertyBag rather than one specific type. The same code with a strongly typed PropertyBag<decimal> looks like this: [TestMethod] public void PropertyBagTwoWayValueTypeSerializationTest() { var bag = new PropertyBag<decimal>(); bag.Add("key", 10M); bag.Add("Key1", 100.10M); bag.Add("Key2", 200.10M); bag.Add("Key3", 300.10M); string xml = bag.ToXml(); TestContext.WriteLine(bag.ToXml()); bag.Clear(); bag.FromXml(xml); Assert.IsTrue(bag.Get("Key1") == 100.10M); Assert.IsTrue(bag.Get("Key3") == 300.10M); } and produces typed results of type decimal. The types can be either value or reference types the combination of which actually proved to be a little more tricky than anticipated due to null and specific string value checks required - getting the generic typing right required use of default(T) and Convert.ChangeType() to trick the compiler into playing nice. Of course the whole raison d'etre for this class is the XML serialization. You can see in the code above that we're doing a .ToXml() and .FromXml() to serialize to and from string. The XML produced for the first example looks like this: <?xml version="1.0" encoding="utf-8"?> <properties> <item> <key>key</key> <value>Value</value> </item> <item> <key>Key2</key> <value type="decimal">100.10</value> </item> <item> <key>Key3</key> <value type="___System.Guid"> <guid>f7a92032-0c6d-4e9d-9950-b15ff7cd207d</guid> </value> </item> <item> <key>Key4</key> <value type="datetime">2011-09-26T17:45:58.5789578-10:00</value> </item> <item> <key>Key5</key> <value type="boolean">true</value> </item> <item> <key>Key7</key> <value type="base64Binary">Ki1C</value> </item> <item> <key>Key8</key> <value type="nil" /> </item> <item> <key>Key9</key> <value type="___Westwind.Tools.Tests.PropertyBagTest+ComplexObject"> <ComplexObject> <Name>Rick</Name> <Entered>2011-09-26T17:45:58.5789578-10:00</Entered> <Count>10</Count> </ComplexObject> </value> </item> </properties> The format is a bit cleaner than the DataContractSerializer. Each item is serialized into <key> <value> pairs. If the value is a string no type information is written. Since string tends to be the most common type this saves space and serialization processing. All other types are attributed. Simple types are mapped to XML types so things like decimal, datetime, boolean and base64Binary are encoded using their Xml type values. All other types are embedded with a hokey format that describes the .NET type preceded by a three underscores and then are encoded using the XmlSerializer. You can see this best above in the ComplexObject encoding. For custom types this isn't pretty either, but it's more concise than the DCS and it works as long as you're serializing back and forth between .NET clients at least. The XML generated from the second example that uses PropertyBag<decimal> looks like this: <?xml version="1.0" encoding="utf-8"?> <properties> <item> <key>key</key> <value type="decimal">10</value> </item> <item> <key>Key1</key> <value type="decimal">100.10</value> </item> <item> <key>Key2</key> <value type="decimal">200.10</value> </item> <item> <key>Key3</key> <value type="decimal">300.10</value> </item> </properties> How does it work As I mentioned there's nothing fancy about this solution - it's little more than a subclass of Dictionary<T,V> that implements custom Xml Serialization and a couple of helper methods that facilitate getting the XML in and out of the class more easily. But it's proven very handy for a number of projects for me where dynamic data storage is required. Here's the code: /// <summary> /// Creates a serializable string/object dictionary that is XML serializable /// Encodes keys as element names and values as simple values with a type /// attribute that contains an XML type name. Complex names encode the type /// name with type='___namespace.classname' format followed by a standard xml /// serialized format. The latter serialization can be slow so it's not recommended /// to pass complex types if performance is critical. /// </summary> [XmlRoot("properties")] public class PropertyBag : PropertyBag<object> { /// <summary> /// Creates an instance of a propertybag from an Xml string /// </summary> /// <param name="xml">Serialize</param> /// <returns></returns> public static PropertyBag CreateFromXml(string xml) { var bag = new PropertyBag(); bag.FromXml(xml); return bag; } } /// <summary> /// Creates a serializable string for generic types that is XML serializable. /// /// Encodes keys as element names and values as simple values with a type /// attribute that contains an XML type name. Complex names encode the type /// name with type='___namespace.classname' format followed by a standard xml /// serialized format. The latter serialization can be slow so it's not recommended /// to pass complex types if performance is critical. /// </summary> /// <typeparam name="TValue">Must be a reference type. For value types use type object</typeparam> [XmlRoot("properties")] public class PropertyBag<TValue> : Dictionary<string, TValue>, IXmlSerializable { /// <summary> /// Not implemented - this means no schema information is passed /// so this won't work with ASMX/WCF services. /// </summary> /// <returns></returns> public System.Xml.Schema.XmlSchema GetSchema() { return null; } /// <summary> /// Serializes the dictionary to XML. Keys are /// serialized to element names and values as /// element values. An xml type attribute is embedded /// for each serialized element - a .NET type /// element is embedded for each complex type and /// prefixed with three underscores. /// </summary> /// <param name="writer"></param> public void WriteXml(System.Xml.XmlWriter writer) { foreach (string key in this.Keys) { TValue value = this[key]; Type type = null; if (value != null) type = value.GetType(); writer.WriteStartElement("item"); writer.WriteStartElement("key"); writer.WriteString(key as string); writer.WriteEndElement(); writer.WriteStartElement("value"); string xmlType = XmlUtils.MapTypeToXmlType(type); bool isCustom = false; // Type information attribute if not string if (value == null) { writer.WriteAttributeString("type", "nil"); } else if (!string.IsNullOrEmpty(xmlType)) { if (xmlType != "string") { writer.WriteStartAttribute("type"); writer.WriteString(xmlType); writer.WriteEndAttribute(); } } else { isCustom = true; xmlType = "___" + value.GetType().FullName; writer.WriteStartAttribute("type"); writer.WriteString(xmlType); writer.WriteEndAttribute(); } // Actual deserialization if (!isCustom) { if (value != null) writer.WriteValue(value); } else { XmlSerializer ser = new XmlSerializer(value.GetType()); ser.Serialize(writer, value); } writer.WriteEndElement(); // value writer.WriteEndElement(); // item } } /// <summary> /// Reads the custom serialized format /// </summary> /// <param name="reader"></param> public void ReadXml(System.Xml.XmlReader reader) { this.Clear(); while (reader.Read()) { if (reader.NodeType == XmlNodeType.Element && reader.Name == "key") { string xmlType = null; string name = reader.ReadElementContentAsString(); // item element reader.ReadToNextSibling("value"); if (reader.MoveToNextAttribute()) xmlType = reader.Value; reader.MoveToContent(); TValue value; if (xmlType == "nil") value = default(TValue); // null else if (string.IsNullOrEmpty(xmlType)) { // value is a string or object and we can assign TValue to value string strval = reader.ReadElementContentAsString(); value = (TValue) Convert.ChangeType(strval, typeof(TValue)); } else if (xmlType.StartsWith("___")) { while (reader.Read() && reader.NodeType != XmlNodeType.Element) { } Type type = ReflectionUtils.GetTypeFromName(xmlType.Substring(3)); //value = reader.ReadElementContentAs(type,null); XmlSerializer ser = new XmlSerializer(type); value = (TValue)ser.Deserialize(reader); } else value = (TValue)reader.ReadElementContentAs(XmlUtils.MapXmlTypeToType(xmlType), null); this.Add(name, value); } } } /// <summary> /// Serializes this dictionary to an XML string /// </summary> /// <returns>XML String or Null if it fails</returns> public string ToXml() { string xml = null; SerializationUtils.SerializeObject(this, out xml); return xml; } /// <summary> /// Deserializes from an XML string /// </summary> /// <param name="xml"></param> /// <returns>true or false</returns> public bool FromXml(string xml) { this.Clear(); // if xml string is empty we return an empty dictionary if (string.IsNullOrEmpty(xml)) return true; var result = SerializationUtils.DeSerializeObject(xml, this.GetType()) as PropertyBag<TValue>; if (result != null) { foreach (var item in result) { this.Add(item.Key, item.Value); } } else // null is a failure return false; return true; } /// <summary> /// Creates an instance of a propertybag from an Xml string /// </summary> /// <param name="xml"></param> /// <returns></returns> public static PropertyBag<TValue> CreateFromXml(string xml) { var bag = new PropertyBag<TValue>(); bag.FromXml(xml); return bag; } } } The code uses a couple of small helper classes SerializationUtils and XmlUtils for mapping Xml types to and from .NET, both of which are from the WestWind,Utilities project (which is the same project where PropertyBag lives) from the West Wind Web Toolkit. The code implements ReadXml and WriteXml for the IXmlSerializable implementation using old school XmlReaders and XmlWriters (because it's pretty simple stuff - no need for XLinq here). Then there are two helper methods .ToXml() and .FromXml() that basically allow your code to easily convert between XML and a PropertyBag object. In my code that's what I use to actually to persist to and from the entity XML property during .Load() and .Save() operations. It's sweet to be able to have a string key dictionary and then be able to turn around with 1 line of code to persist the whole thing to XML and back. Hopefully some of you will find this class as useful as I've found it. It's a simple solution to a common requirement in my applications and I've used the hell out of it in the short time since I created it. Resources You can find the complete code for the two classes plus the helpers in the Subversion repository for Westwind.Utilities. You can grab the source files from there or download the whole project. You can also grab the full Westwind.Utilities assembly from NuGet and add it to your project if that's easier for you. PropertyBag Source Code SerializationUtils and XmlUtils Westwind.Utilities Assembly on NuGet (add from Visual Studio) © Rick Strahl, West Wind Technologies, 2005-2011Posted in .NET CSharp Tweet (function() { var po = document.createElement('script'); po.type = 'text/javascript'; po.async = true; po.src = 'https://apis.google.com/js/plusone.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s); })();

Read the article

unexplainable packet drops with 5 ethernet NICs and low traffic on Ubuntu

- by jon

I'm stuck on problem where my machine started to drops packets with no sign of ANY system load or high interrupt usage after an upgrade to Ubuntu 12.04. My server is a network monitoring sensor, running Ubuntu LTS 12.04, it passively collects packets from 5 interfaces doing network intrusion type stuff. Before the upgrade I managed to collect 200+GB of packets a day while writing them to disk with around 0% packet loss depending on the day with the help of CPU affinity and NIC IRQ to CPU bindings. Now I lose a great deal of packets with none of my applications running and at very low PPS rate which a modern workstation NIC would have no trouble with. Specs: x64 Xeon 4 cores 3.2 Ghz 16 GB RAM NICs: 5 Intel Pro NICs using the e1000 driver (NAPI). [1] eth0 and eth1 are integrated NICs (in the motherboard) There are 2 other PCI-X network cards, each with 2 Ethernet ports. 3 of the interfaces are running at Gigabit Ethernet, the others are not because they're attached to hubs. Specs: [2] http://support.dell.com/support/edocs/systems/pe2850/en/ug/t1390aa.htm uptime 17:36:00 up 1:43, 2 users, load average: 0.00, 0.01, 0.05 # uname -a Linux nms 3.2.0-29-generic #46-Ubuntu SMP Fri Jul 27 17:03:23 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux I also have the CPU governor set to performance mode and irqbalance off. The problem still occurs with them on. # lspci -t -vv -[0000:00]-+-00.0 Intel Corporation E7520 Memory Controller Hub +-02.0-[01-03]--+-00.0-[02]----0e.0 Dell PowerEdge Expandable RAID controller 4 | \-00.2-[03]-- +-04.0-[04]-- +-05.0-[05-07]--+-00.0-[06]----07.0 Intel Corporation 82541GI Gigabit Ethernet Controller | \-00.2-[07]----08.0 Intel Corporation 82541GI Gigabit Ethernet Controller +-06.0-[08-0a]--+-00.0-[09]--+-04.0 Intel Corporation 82546EB Gigabit Ethernet Controller (Copper) | | \-04.1 Intel Corporation 82546EB Gigabit Ethernet Controller (Copper) | \-00.2-[0a]--+-02.0 Digium, Inc. Wildcard TE210P/TE212P dual-span T1/E1/J1 card 3.3V | +-03.0 Intel Corporation 82546EB Gigabit Ethernet Controller (Copper) | \-03.1 Intel Corporation 82546EB Gigabit Ethernet Controller (Copper) +-1d.0 Intel Corporation 82801EB/ER (ICH5/ICH5R) USB UHCI Controller #1 +-1d.1 Intel Corporation 82801EB/ER (ICH5/ICH5R) USB UHCI Controller #2 +-1d.2 Intel Corporation 82801EB/ER (ICH5/ICH5R) USB UHCI Controller #3 +-1d.7 Intel Corporation 82801EB/ER (ICH5/ICH5R) USB2 EHCI Controller +-1e.0-[0b]----0d.0 Advanced Micro Devices [AMD] nee ATI RV100 QY [Radeon 7000/VE] +-1f.0 Intel Corporation 82801EB/ER (ICH5/ICH5R) LPC Interface Bridge \-1f.1 Intel Corporation 82801EB/ER (ICH5/ICH5R) IDE Controller I believe the NIC nor the NIC drivers are dropping the packets because ethtool reports 0 under rx_missed_errors and rx_no_buffer_count for each interface. On the old system, if it couldn't keep up this is where the drops would be. I drop packets on multiple interfaces just about every second, usually in small increments of 2-4. I tried all these sysctl values, I'm currently using the uncommented ones. # cat /etc/sysctl.conf # high net.core.netdev_max_backlog = 3000000 net.core.rmem_max = 16000000 net.core.rmem_default = 8000000 # defaults #net.core.netdev_max_backlog = 1000 #net.core.rmem_max = 131071 #net.core.rmem_default = 163480 # moderate #net.core.netdev_max_backlog = 10000 #net.core.rmem_max = 33554432 #net.core.rmem_default = 33554432 Here's an example of an interface stats report with ethtool. They are all the same, nothing is out of the ordinary ( I think ), so I'm only going to show one: ethtool -S eth2 NIC statistics: rx_packets: 7498 tx_packets: 0 rx_bytes: 2722585 tx_bytes: 0 rx_broadcast: 327 tx_broadcast: 0 rx_multicast: 1504 tx_multicast: 0 rx_errors: 0 tx_errors: 0 tx_dropped: 0 multicast: 1504 collisions: 0 rx_length_errors: 0 rx_over_errors: 0 rx_crc_errors: 0 rx_frame_errors: 0 rx_no_buffer_count: 0 rx_missed_errors: 0 tx_aborted_errors: 0 tx_carrier_errors: 0 tx_fifo_errors: 0 tx_heartbeat_errors: 0 tx_window_errors: 0 tx_abort_late_coll: 0 tx_deferred_ok: 0 tx_single_coll_ok: 0 tx_multi_coll_ok: 0 tx_timeout_count: 0 tx_restart_queue: 0 rx_long_length_errors: 0 rx_short_length_errors: 0 rx_align_errors: 0 tx_tcp_seg_good: 0 tx_tcp_seg_failed: 0 rx_flow_control_xon: 0 rx_flow_control_xoff: 0 tx_flow_control_xon: 0 tx_flow_control_xoff: 0 rx_long_byte_count: 2722585 rx_csum_offload_good: 0 rx_csum_offload_errors: 0 alloc_rx_buff_failed: 0 tx_smbus: 0 rx_smbus: 0 dropped_smbus: 01 # ifconfig eth0 Link encap:Ethernet HWaddr 00:11:43:e0:e2:8c UP BROADCAST RUNNING NOARP PROMISC ALLMULTI MULTICAST MTU:1500 Metric:1 RX packets:373348 errors:16 dropped:95 overruns:0 frame:16 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:356830572 (356.8 MB) TX bytes:0 (0.0 B) eth1 Link encap:Ethernet HWaddr 00:11:43:e0:e2:8d UP BROADCAST RUNNING NOARP PROMISC ALLMULTI MULTICAST MTU:1500 Metric:1 RX packets:13616 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:8690528 (8.6 MB) TX bytes:0 (0.0 B) eth2 Link encap:Ethernet HWaddr 00:04:23:e1:77:6a UP BROADCAST RUNNING NOARP PROMISC ALLMULTI MULTICAST MTU:1500 Metric:1 RX packets:7750 errors:0 dropped:471 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:2780935 (2.7 MB) TX bytes:0 (0.0 B) eth3 Link encap:Ethernet HWaddr 00:04:23:e1:77:6b UP BROADCAST RUNNING NOARP PROMISC ALLMULTI MULTICAST MTU:1500 Metric:1 RX packets:5112 errors:0 dropped:206 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:639472 (639.4 KB) TX bytes:0 (0.0 B) eth4 Link encap:Ethernet HWaddr 00:04:23:b6:35:6c UP BROADCAST RUNNING NOARP PROMISC ALLMULTI MULTICAST MTU:1500 Metric:1 RX packets:961467 errors:0 dropped:935 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:958561305 (958.5 MB) TX bytes:0 (0.0 B) eth5 Link encap:Ethernet HWaddr 00:04:23:b6:35:6d inet addr:192.168.1.6 Bcast:192.168.1.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:4264 errors:0 dropped:16 overruns:0 frame:0 TX packets:699 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:572228 (572.2 KB) TX bytes:124456 (124.4 KB) I tried the defaults, then started to play around with settings. I wasn't using any flow control and I increased the RxDescriptor count to 4096 before the upgrade as well without any problems. # cat /etc/modprobe.d/e1000.conf options e1000 XsumRX=0,0,0,0,0 RxDescriptors=4096,4096,4096,4096,4096 FlowControl=0,0,0,0,0 debug=16 Here's my network configuration file, I turned off checksumming and various offloading mechanisms along with setting CPU affinity with heavy use interfaces getting an entire CPU and light use interfaces sharing a CPU. I used these settings prior to the upgrade without problems. # cat /etc/network/interfaces # The loopback network interface auto lo iface lo inet loopback # The primary network interface auto eth0 iface eth0 inet manual pre-up /sbin/ethtool -G eth0 rx 4096 tx 0 pre-up /sbin/ethtool -K eth0 gro off gso off rx off pre-up /sbin/ethtool -A eth0 rx off autoneg off up ifconfig eth0 0.0.0.0 -arp promisc mtu 1500 allmulti txqueuelen 0 up post-up echo "4" > /proc/irq/48/smp_affinity down ifconfig eth0 down post-down /sbin/ethtool -G eth0 rx 256 tx 256 post-down /sbin/ethtool -K eth0 gro on gso on rx on post-down /sbin/ethtool -A eth0 rx on autoneg on auto eth1 iface eth1 inet manual pre-up /sbin/ethtool -G eth1 rx 4096 tx 0 pre-up /sbin/ethtool -K eth1 gro off gso off rx off pre-up /sbin/ethtool -A eth1 rx off autoneg off up ifconfig eth1 0.0.0.0 -arp promisc mtu 1500 allmulti txqueuelen 0 up post-up echo "4" > /proc/irq/49/smp_affinity down ifconfig eth1 down post-down /sbin/ethtool -G eth1 rx 256 tx 256 post-down /sbin/ethtool -K eth1 gro on gso on rx on post-down /sbin/ethtool -A eth1 rx on autoneg on auto eth2 iface eth2 inet manual pre-up /sbin/ethtool -G eth2 rx 4096 tx 0 pre-up /sbin/ethtool -K eth2 gro off gso off rx off pre-up /sbin/ethtool -A eth2 rx off autoneg off up ifconfig eth2 0.0.0.0 -arp promisc mtu 1500 allmulti txqueuelen 0 up post-up echo "1" > /proc/irq/82/smp_affinity down ifconfig eth2 down post-down /sbin/ethtool -G eth2 rx 256 tx 256 post-down /sbin/ethtool -K eth2 gro on gso on rx on post-down /sbin/ethtool -A eth2 rx on autoneg on auto eth3 iface eth3 inet manual pre-up /sbin/ethtool -G eth3 rx 4096 tx 0 pre-up /sbin/ethtool -K eth3 gro off gso off rx off pre-up /sbin/ethtool -A eth3 rx off autoneg off up ifconfig eth3 0.0.0.0 -arp promisc mtu 1500 allmulti txqueuelen 0 up post-up echo "2" > /proc/irq/83/smp_affinity down ifconfig eth3 down post-down /sbin/ethtool -G eth3 rx 256 tx 256 post-down /sbin/ethtool -K eth3 gro on gso on rx on post-down /sbin/ethtool -A eth3 rx on autoneg on auto eth4 iface eth4 inet manual pre-up /sbin/ethtool -G eth4 rx 4096 tx 0 pre-up /sbin/ethtool -K eth4 gro off gso off rx off pre-up /sbin/ethtool -A eth4 rx off autoneg off up ifconfig eth4 0.0.0.0 -arp promisc mtu 1500 allmulti txqueuelen 0 up post-up echo "4" > /proc/irq/77/smp_affinity down ifconfig eth4 down post-down /sbin/ethtool -G eth4 rx 256 tx 256 post-down /sbin/ethtool -K eth4 gro on gso on rx on post-down /sbin/ethtool -A eth4 rx on autoneg on auto eth5 iface eth5 inet static pre-up /etc/fw.conf address 192.168.1.1 netmask 255.255.255.0 broadcast 192.168.1.255 gateway 192.168.1.1 dns-nameservers 192.168.1.2 192.168.1.3 up ifconfig eth5 up post-up echo "8" > /proc/irq/77/smp_affinity down ifconfig eth5 down Here's a few examples of packet drops, i ran one after another, probabling totaling 3 or 4 seconds. You can see increases in the drops from the 1st and 3rd. This was a non-busy time, very little traffic. # awk '{ print $1,$5 }' /proc/net/dev Inter-| face drop eth3: 225 lo: 0 eth2: 505 eth1: 0 eth5: 17 eth0: 105 eth4: 1034 # awk '{ print $1,$5 }' /proc/net/dev Inter-| face drop eth3: 225 lo: 0 eth2: 507 eth1: 0 eth5: 17 eth0: 105 eth4: 1034 # awk '{ print $1,$5 }' /proc/net/dev Inter-| face drop eth3: 227 lo: 0 eth2: 512 eth1: 0 eth5: 17 eth0: 105 eth4: 1039 I tried the pci=noacpi options. With and without, it's the same. This is what my interrupt stats looked like before the upgrade, after, with ACPI on PCI it showed multiple NICs bound to an interrupt and shared with other devices such as USB drives which I didn't like so I think i'm going to keep it with ACPI off as it's easier to designate sole purpose interrupts. Is there any advantage I would have using the default i.e. ACPI w/ PCI. ? # cat /etc/default/grub | grep CMD_LINE GRUB_CMDLINE_LINUX_DEFAULT="ipv6.disable=1 noacpi pci=noacpi" GRUB_CMDLINE_LINUX="" # cat /proc/interrupts CPU0 CPU1 CPU2 CPU3 0: 45 0 0 16 IO-APIC-edge timer 1: 1 0 0 7936 IO-APIC-edge i8042 2: 0 0 0 0 XT-PIC-XT-PIC cascade 6: 0 0 0 3 IO-APIC-edge floppy 8: 0 0 0 1 IO-APIC-edge rtc0 9: 0 0 0 0 IO-APIC-edge acpi 12: 0 0 0 1809 IO-APIC-edge i8042 14: 1 0 0 4498 IO-APIC-edge ata_piix 15: 0 0 0 0 IO-APIC-edge ata_piix 16: 0 0 0 0 IO-APIC-fasteoi uhci_hcd:usb2 18: 0 0 0 1350 IO-APIC-fasteoi uhci_hcd:usb4, radeon 19: 0 0 0 0 IO-APIC-fasteoi uhci_hcd:usb3 23: 0 0 0 4099 IO-APIC-fasteoi ehci_hcd:usb1 38: 0 0 0 61963 IO-APIC-fasteoi megaraid 48: 0 0 1002319 4 IO-APIC-fasteoi eth0 49: 0 0 38772 3 IO-APIC-fasteoi eth1 77: 0 0 130076 432159 IO-APIC-fasteoi eth4 78: 0 0 0 23917 IO-APIC-fasteoi eth5 82: 1329033 0 0 4 IO-APIC-fasteoi eth2 83: 0 4886525 0 6 IO-APIC-fasteoi eth3 NMI: 5 6 4 5 Non-maskable interrupts LOC: 61409 57076 64257 114764 Local timer interrupts SPU: 0 0 0 0 Spurious interrupts IWI: 0 0 0 0 IRQ work interrupts RES: 17956 25333 13436 14789 Rescheduling interrupts CAL: 22436 607 539 478 Function call interrupts TLB: 1525 1458 4600 4151 TLB shootdowns TRM: 0 0 0 0 Thermal event interrupts THR: 0 0 0 0 Threshold APIC interrupts MCE: 0 0 0 0 Machine check exceptions MCP: 16 16 16 16 Machine check polls ERR: 0 MIS: 0 Here's sample output of vmstat, showing the system. Barebones system right now. root@nms:~# vmstat -S m 1 procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu---- r b swpd free buff cache si so bi bo in cs us sy id wa 0 0 0 14992 192 1029 0 0 56 2 419 29 1 0 99 0 0 0 0 14992 192 1029 0 0 0 0 922 27 0 0 100 0 0 0 0 14991 192 1029 0 0 0 36 763 50 0 0 100 0 0 0 0 14991 192 1029 0 0 0 0 646 35 0 0 100 0 0 0 0 14991 192 1029 0 0 0 0 722 54 0 0 100 0 0 0 0 14991 192 1029 0 0 0 0 793 27 0 0 100 0 ^C Here's dmesg output. I can't figure out why my PCI-X slots are negotiated as PCI. The network cards are all PCI-X with the exception of the integrated NICs that came with the server. In the output below it looks as if eth3 and eth2 negotiated at PCI-X speeds rather than PCI:66Mhz. Wouldn't they all drop to PCI:66Mhz? If your integrated NICs are PCI, as labeled below (eth0,eth1), then wouldn't all devices on your bus speed drop down to that slower bus speed? If not, I still don't know why only one of my NICs ( each has two ethernet ports) is labeled as PCI-X in the output below. Does that mean it is running at PCI-X speeds are is it showing that it's capable? # dmesg | grep e1000 [ 3678.349337] e1000: Intel(R) PRO/1000 Network Driver - version 7.3.21-k8-NAPI [ 3678.349342] e1000: Copyright (c) 1999-2006 Intel Corporation. [ 3678.349394] e1000 0000:06:07.0: PCI->APIC IRQ transform: INT A -> IRQ 48 [ 3678.409725] e1000 0000:06:07.0: Receive Descriptors set to 4096 [ 3678.409730] e1000 0000:06:07.0: Checksum Offload Disabled [ 3678.409734] e1000 0000:06:07.0: Flow Control Disabled [ 3678.586409] e1000 0000:06:07.0: eth0: (PCI:66MHz:32-bit) 00:11:43:e0:e2:8c [ 3678.586419] e1000 0000:06:07.0: eth0: Intel(R) PRO/1000 Network Connection [ 3678.586642] e1000 0000:07:08.0: PCI->APIC IRQ transform: INT A -> IRQ 49 [ 3678.649854] e1000 0000:07:08.0: Receive Descriptors set to 4096 [ 3678.649859] e1000 0000:07:08.0: Checksum Offload Disabled [ 3678.649863] e1000 0000:07:08.0: Flow Control Disabled [ 3678.826436] e1000 0000:07:08.0: eth1: (PCI:66MHz:32-bit) 00:11:43:e0:e2:8d [ 3678.826444] e1000 0000:07:08.0: eth1: Intel(R) PRO/1000 Network Connection [ 3678.826627] e1000 0000:09:04.0: PCI->APIC IRQ transform: INT A -> IRQ 82 [ 3679.093266] e1000 0000:09:04.0: Receive Descriptors set to 4096 [ 3679.093271] e1000 0000:09:04.0: Checksum Offload Disabled [ 3679.093275] e1000 0000:09:04.0: Flow Control Disabled [ 3679.130239] e1000 0000:09:04.0: eth2: (PCI-X:133MHz:64-bit) 00:04:23:e1:77:6a [ 3679.130246] e1000 0000:09:04.0: eth2: Intel(R) PRO/1000 Network Connection [ 3679.130449] e1000 0000:09:04.1: PCI->APIC IRQ transform: INT B -> IRQ 83 [ 3679.397312] e1000 0000:09:04.1: Receive Descriptors set to 4096 [ 3679.397318] e1000 0000:09:04.1: Checksum Offload Disabled [ 3679.397321] e1000 0000:09:04.1: Flow Control Disabled [ 3679.434350] e1000 0000:09:04.1: eth3: (PCI-X:133MHz:64-bit) 00:04:23:e1:77:6b [ 3679.434360] e1000 0000:09:04.1: eth3: Intel(R) PRO/1000 Network Connection [ 3679.434553] e1000 0000:0a:03.0: PCI->APIC IRQ transform: INT A -> IRQ 77 [ 3679.704072] e1000 0000:0a:03.0: Receive Descriptors set to 4096 [ 3679.704077] e1000 0000:0a:03.0: Checksum Offload Disabled [ 3679.704081] e1000 0000:0a:03.0: Flow Control Disabled [ 3679.738364] e1000 0000:0a:03.0: eth4: (PCI:33MHz:64-bit) 00:04:23:b6:35:6c [ 3679.738371] e1000 0000:0a:03.0: eth4: Intel(R) PRO/1000 Network Connection [ 3679.738538] e1000 0000:0a:03.1: PCI->APIC IRQ transform: INT B -> IRQ 78 [ 3680.046060] e1000 0000:0a:03.1: eth5: (PCI:33MHz:64-bit) 00:04:23:b6:35:6d [ 3680.046067] e1000 0000:0a:03.1: eth5: Intel(R) PRO/1000 Network Connection [ 3682.132415] e1000: eth0 NIC Link is Up 100 Mbps Half Duplex, Flow Control: None [ 3682.224423] e1000: eth1 NIC Link is Up 100 Mbps Half Duplex, Flow Control: None [ 3682.316385] e1000: eth2 NIC Link is Up 100 Mbps Half Duplex, Flow Control: None [ 3682.408391] e1000: eth3 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None [ 3682.500396] e1000: eth4 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None [ 3682.708401] e1000: eth5 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX At first I thought it was the NIC drivers but I'm not so sure. I really have no idea where else to look at the moment. Any help is greatly appreciated as I'm struggling with this. If you need more information just ask. Thanks! [1]http://www.cs.fsu.edu/~baker/devices/lxr/http/source/linux/Documentation/networking/e1000.txt?v=2.6.11.8 [2] http://support.dell.com/support/edocs/systems/pe2850/en/ug/t1390aa.htm

Read the article

Is it possible to run dhcpd3 as non-root user in a chroot jail?

- by Lenain

Hi everyone. I would like to run dhcpd3 from a chroot jail on Debian Lenny. At the moment, I can run it as root from my jail. Now I want to do this as non-root user (as "-u blah -t /path/to/jail" Bind option). If I start my process like this : start-stop-daemon --chroot /home/jails/dhcp --chuid dhcp \ --start --pidfile /home/jails/dhcp/var/run/dhcp.pid --exec /usr/sbin/dhcpd3 I get stuck with these errors : Internet Systems Consortium DHCP Server V3.1.1 Copyright 2004-2008 Internet Systems Consortium. All rights reserved. For info, please visit http://www.isc.org/sw/dhcp/ unable to create icmp socket: Operation not permitted Wrote 0 deleted host decls to leases file. Wrote 0 new dynamic host decls to leases file. Wrote 0 leases to leases file. Open a socket for LPF: Operation not permitted strace : brk(0) = 0x911b000 fcntl64(0, F_GETFD) = 0 fcntl64(1, F_GETFD) = 0 fcntl64(2, F_GETFD) = 0 access("/etc/suid-debug", F_OK) = -1 ENOENT (No such file or directory) access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb775d000 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) open("/etc/ld.so.cache", O_RDONLY) = -1 ENOENT (No such file or directory) open("/lib/tls/i686/cmov/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/lib/tls/i686/cmov", 0xbfc2ac84) = -1 ENOENT (No such file or directory) open("/lib/tls/i686/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/lib/tls/i686", 0xbfc2ac84) = -1 ENOENT (No such file or directory) open("/lib/tls/cmov/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/lib/tls/cmov", 0xbfc2ac84) = -1 ENOENT (No such file or directory) open("/lib/tls/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/lib/tls", 0xbfc2ac84) = -1 ENOENT (No such file or directory) open("/lib/i686/cmov/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/lib/i686/cmov", 0xbfc2ac84) = -1 ENOENT (No such file or directory) open("/lib/i686/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/lib/i686", 0xbfc2ac84) = -1 ENOENT (No such file or directory) open("/lib/cmov/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/lib/cmov", 0xbfc2ac84) = -1 ENOENT (No such file or directory) open("/lib/libc.so.6", O_RDONLY) = 3 read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\260e\1\0004\0\0\0t"..., 512) = 512 fstat64(3, {st_mode=S_IFREG|0755, st_size=1294572, ...}) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb775c000 mmap2(NULL, 1300080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb761e000 mmap2(0xb7756000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x138) = 0xb7756000 mmap2(0xb7759000, 9840, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7759000 close(3) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb761d000 set_thread_area({entry_number:-1 - 6, base_addr:0xb761d6b0, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0 mprotect(0xb7756000, 4096, PROT_READ) = 0 open("/dev/null", O_RDWR) = 3 close(3) = 0 brk(0) = 0x911b000 brk(0x913c000) = 0x913c000 socket(PF_FILE, SOCK_DGRAM, 0) = 3 fcntl64(3, F_SETFD, FD_CLOEXEC) = 0 connect(3, {sa_family=AF_FILE, path="/dev/log"...}, 110) = 0 time(NULL) = 1284760816 open("/etc/localtime", O_RDONLY) = 4 fstat64(4, {st_mode=S_IFREG|0644, st_size=2945, ...}) = 0 fstat64(4, {st_mode=S_IFREG|0644, st_size=2945, ...}) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb761c000 read(4, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\f\0\0\0\f\0\0\0\0\0"..., 4096) = 2945 _llseek(4, -28, [2917], SEEK_CUR) = 0 read(4, "\nCET-1CEST,M3.5.0,M10.5.0/3\n"..., 4096) = 28 close(4) = 0 munmap(0xb761c000, 4096) = 0 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2945, ...}) = 0 send(3, "Sep 18 00:00:16 dhcpd: Intern"..., 73, MSG_NOSIGNAL) = 73 write(2, "Internet Systems Consortium DHCP "..., 46Internet Systems Consortium DHCP Server V3.1.1) = 46 write(2, "\n"..., 1 ) = 1 time(NULL) = 1284760816 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2945, ...}) = 0 send(3, "Sep 18 00:00:16 dhcpd: Copyri"..., 75, MSG_NOSIGNAL) = 75 write(2, "Copyright 2004-2008 Internet Syst"..., 48Copyright 2004-2008 Internet Systems Consortium.) = 48 write(2, "\n"..., 1 ) = 1 time(NULL) = 1284760816 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2945, ...}) = 0 send(3, "Sep 18 00:00:16 dhcpd: All ri"..., 47, MSG_NOSIGNAL) = 47 write(2, "All rights reserved."..., 20All rights reserved.) = 20 write(2, "\n"..., 1 ) = 1 time(NULL) = 1284760816 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2945, ...}) = 0 send(3, "Sep 18 00:00:16 dhcpd: For in"..., 77, MSG_NOSIGNAL) = 77 write(2, "For info, please visit http://www"..., 50For info, please visit http://www.isc.org/sw/dhcp/) = 50 write(2, "\n"..., 1 ) = 1 socket(PF_FILE, SOCK_STREAM, 0) = 4 fcntl64(4, F_SETFL, O_RDWR|O_NONBLOCK) = 0 connect(4, {sa_family=AF_FILE, path="/var/run/nscd/socket"...}, 110) = -1 ENOENT (No such file or directory) close(4) = 0 socket(PF_FILE, SOCK_STREAM, 0) = 4 fcntl64(4, F_SETFL, O_RDWR|O_NONBLOCK) = 0 connect(4, {sa_family=AF_FILE, path="/var/run/nscd/socket"...}, 110) = -1 ENOENT (No such file or directory) close(4) = 0 open("/etc/nsswitch.conf", O_RDONLY) = 4 fstat64(4, {st_mode=S_IFREG|0644, st_size=475, ...}) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb761c000 read(4, "# /etc/nsswitch.conf\n#\n# Example "..., 4096) = 475 read(4, ""..., 4096) = 0 close(4) = 0 munmap(0xb761c000, 4096) = 0 open("/lib/libnss_db.so.2", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/lib/tls/i686/cmov/libnss_db.so.2", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/usr/lib/tls/i686/cmov", 0xbfc2ad5c) = -1 ENOENT (No such file or directory) open("/usr/lib/tls/i686/libnss_db.so.2", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/usr/lib/tls/i686", 0xbfc2ad5c) = -1 ENOENT (No such file or directory) open("/usr/lib/tls/cmov/libnss_db.so.2", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/usr/lib/tls/cmov", 0xbfc2ad5c) = -1 ENOENT (No such file or directory) open("/usr/lib/tls/libnss_db.so.2", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/usr/lib/tls", 0xbfc2ad5c) = -1 ENOENT (No such file or directory) open("/usr/lib/i686/cmov/libnss_db.so.2", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/usr/lib/i686/cmov", 0xbfc2ad5c) = -1 ENOENT (No such file or directory) open("/usr/lib/i686/libnss_db.so.2", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/usr/lib/i686", 0xbfc2ad5c) = -1 ENOENT (No such file or directory) open("/usr/lib/cmov/libnss_db.so.2", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/usr/lib/cmov", 0xbfc2ad5c) = -1 ENOENT (No such file or directory) open("/usr/lib/libnss_db.so.2", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/usr/lib", 0xbfc2ad5c) = -1 ENOENT (No such file or directory) open("/lib/i486-linux-gnu/tls/i686/cmov/libnss_db.so.2", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/lib/i486-linux-gnu/tls/i686/cmov", 0xbfc2ad5c) = -1 ENOENT (No such file or directory) open("/lib/i486-linux-gnu/tls/i686/libnss_db.so.2", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/lib/i486-linux-gnu/tls/i686", 0xbfc2ad5c) = -1 ENOENT (No such file or directory) open("/lib/i486-linux-gnu/tls/cmov/libnss_db.so.2", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/lib/i486-linux-gnu/tls/cmov", 0xbfc2ad5c) = -1 ENOENT (No such file or directory) open("/lib/i486-linux-gnu/tls/libnss_db.so.2", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/lib/i486-linux-gnu/tls", 0xbfc2ad5c) = -1 ENOENT (No such file or directory) open("/lib/i486-linux-gnu/i686/cmov/libnss_db.so.2", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/lib/i486-linux-gnu/i686/cmov", 0xbfc2ad5c) = -1 ENOENT (No such file or directory) open("/lib/i486-linux-gnu/i686/libnss_db.so.2", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/lib/i486-linux-gnu/i686", 0xbfc2ad5c) = -1 ENOENT (No such file or directory) open("/lib/i486-linux-gnu/cmov/libnss_db.so.2", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/lib/i486-linux-gnu/cmov", 0xbfc2ad5c) = -1 ENOENT (No such file or directory) open("/lib/i486-linux-gnu/libnss_db.so.2", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/lib/i486-linux-gnu", 0xbfc2ad5c) = -1 ENOENT (No such file or directory) open("/usr/lib/i486-linux-gnu/tls/i686/cmov/libnss_db.so.2", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/usr/lib/i486-linux-gnu/tls/i686/cmov", 0xbfc2ad5c) = -1 ENOENT (No such file or directory) open("/usr/lib/i486-linux-gnu/tls/i686/libnss_db.so.2", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/usr/lib/i486-linux-gnu/tls/i686", 0xbfc2ad5c) = -1 ENOENT (No such file or directory) open("/usr/lib/i486-linux-gnu/tls/cmov/libnss_db.so.2", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/usr/lib/i486-linux-gnu/tls/cmov", 0xbfc2ad5c) = -1 ENOENT (No such file or directory) open("/usr/lib/i486-linux-gnu/tls/libnss_db.so.2", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/usr/lib/i486-linux-gnu/tls", 0xbfc2ad5c) = -1 ENOENT (No such file or directory) open("/usr/lib/i486-linux-gnu/i686/cmov/libnss_db.so.2", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/usr/lib/i486-linux-gnu/i686/cmov", 0xbfc2ad5c) = -1 ENOENT (No such file or directory) open("/usr/lib/i486-linux-gnu/i686/libnss_db.so.2", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/usr/lib/i486-linux-gnu/i686", 0xbfc2ad5c) = -1 ENOENT (No such file or directory) open("/usr/lib/i486-linux-gnu/cmov/libnss_db.so.2", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/usr/lib/i486-linux-gnu/cmov", 0xbfc2ad5c) = -1 ENOENT (No such file or directory) open("/usr/lib/i486-linux-gnu/libnss_db.so.2", O_RDONLY) = -1 ENOENT (No such file or directory) stat64("/usr/lib/i486-linux-gnu", 0xbfc2ad5c) = -1 ENOENT (No such file or directory) open("/lib/libnss_files.so.2", O_RDONLY) = 4 read(4, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\320\30\0\0004\0\0\0\250"..., 512) = 512 fstat64(4, {st_mode=S_IFREG|0644, st_size=38408, ...}) = 0 mmap2(NULL, 41624, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0xb7612000 mmap2(0xb761b000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x8) = 0xb761b000 close(4) = 0 open("/etc/services", O_RDONLY|O_CLOEXEC) = 4 fcntl64(4, F_GETFD) = 0x1 (flags FD_CLOEXEC) fstat64(4, {st_mode=S_IFREG|0644, st_size=18480, ...}) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7611000 read(4, "# Network services, Internet styl"..., 4096) = 4096 read(4, "9/tcp\t\t\t\t# Quick Mail Transfer Pr"..., 4096) = 4096 read(4, "note\t1352/tcp\tlotusnotes\t# Lotus "..., 4096) = 4096 read(4, "tion\nafs3-kaserver\t7004/udp\nafs3-"..., 4096) = 4096 read(4, "backup\t2989/tcp\t\t\t# Afmbackup sys"..., 4096) = 2096 read(4, ""..., 4096) = 0 close(4) = 0 munmap(0xb7611000, 4096) = 0 time(NULL) = 1284760816 open("/etc/protocols", O_RDONLY|O_CLOEXEC) = 4 fstat64(4, {st_mode=S_IFREG|0644, st_size=2626, ...}) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7611000 read(4, "# Internet (IP) protocols\n#\n# Upd"..., 4096) = 2626 close(4) = 0 munmap(0xb7611000, 4096) = 0 socket(PF_INET, SOCK_RAW, IPPROTO_ICMP) = -1 EPERM (Operation not permitted) time(NULL) = 1284760816 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2945, ...}) = 0 send(3, "Sep 18 00:00:16 dhcpd: unable"..., 80, MSG_NOSIGNAL) = 80 write(2, "unable to create icmp socket: Ope"..., 53unable to create icmp socket: Operation not permitted) = 53 write(2, "\n"..., 1 ) = 1 open("/etc/dhcp3/dhcpd.conf", O_RDONLY) = 4 lseek(4, 0, SEEK_END) = 1426 lseek(4, 0, SEEK_SET) = 0 read(4, "#----------------------------\n# G"..., 1426) = 1426 close(4) = 0 mmap2(NULL, 401408, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb75b0000 mmap2(NULL, 401408, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb754e000 mmap2(NULL, 401408, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb74ec000 brk(0x916f000) = 0x916f000 close(3) = 0 socket(PF_FILE, SOCK_DGRAM, 0) = 3 fcntl64(3, F_SETFD, FD_CLOEXEC) = 0 connect(3, {sa_family=AF_FILE, path="/dev/log"...}, 110) = 0 time(NULL) = 1284760816 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2945, ...}) = 0 send(3, "Sep 18 00:00:16 dhcpd: Inter"..., 74, MSG_NOSIGNAL) = 74 time(NULL) = 1284760816 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2945, ...}) = 0 send(3, "Sep 18 00:00:16 dhcpd: Copyr"..., 76, MSG_NOSIGNAL) = 76 time(NULL) = 1284760816 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2945, ...}) = 0 send(3, "Sep 18 00:00:16 dhcpd: All r"..., 48, MSG_NOSIGNAL) = 48 time(NULL) = 1284760816 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2945, ...}) = 0 send(3, "Sep 18 00:00:16 dhcpd: For i"..., 78, MSG_NOSIGNAL) = 78 open("/var/lib/dhcp3/dhcpd.leases", O_RDONLY) = 4 lseek(4, 0, SEEK_END) = 126 lseek(4, 0, SEEK_SET) = 0 read(4, "# The format of this file is docu"..., 126) = 126 close(4) = 0 open("/var/lib/dhcp3/dhcpd.leases", O_WRONLY|O_CREAT|O_APPEND, 0666) = 4 fstat64(4, {st_mode=S_IFREG|0644, st_size=126, ...}) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb74eb000 fstat64(4, {st_mode=S_IFREG|0644, st_size=126, ...}) = 0 _llseek(4, 126, [126], SEEK_SET) = 0 time(NULL) = 1284760816 time(NULL) = 1284760816 open("/var/lib/dhcp3/dhcpd.leases.1284760816", O_WRONLY|O_CREAT|O_TRUNC, 0664) = 5 fcntl64(5, F_GETFL) = 0x1 (flags O_WRONLY) fstat64(5, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb74ea000 _llseek(5, 0, [0], SEEK_CUR) = 0 close(4) = 0 munmap(0xb74eb000, 4096) = 0 time(NULL) = 1284760816 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2945, ...}) = 0 send(3, "Sep 18 00:00:16 dhcpd: Wrote"..., 70, MSG_NOSIGNAL) = 70 write(2, "Wrote 0 deleted host decls to lea"..., 42Wrote 0 deleted host decls to leases file.) = 42 write(2, "\n"..., 1 ) = 1 time(NULL) = 1284760816 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2945, ...}) = 0 send(3, "Sep 18 00:00:16 dhcpd: Wrote"..., 74, MSG_NOSIGNAL) = 74 write(2, "Wrote 0 new dynamic host decls to"..., 46Wrote 0 new dynamic host decls to leases file.) = 46 write(2, "\n"..., 1 ) = 1 time(NULL) = 1284760816 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2945, ...}) = 0 send(3, "Sep 18 00:00:16 dhcpd: Wrote"..., 58, MSG_NOSIGNAL) = 58 write(2, "Wrote 0 leases to leases file."..., 30Wrote 0 leases to leases file.) = 30 write(2, "\n"..., 1 ) = 1 write(5, "# The format of this file is docu"..., 126) = 126 fsync(5) = 0 unlink("/var/lib/dhcp3/dhcpd.leases~") = 0 link("/var/lib/dhcp3/dhcpd.leases", "/var/lib/dhcp3/dhcpd.leases~") = 0 rename("/var/lib/dhcp3/dhcpd.leases.1284760816", "/var/lib/dhcp3/dhcpd.leases") = 0 socket(PF_INET, SOCK_DGRAM, IPPROTO_UDP) = 4 ioctl(4, SIOCGIFCONF, {0 - 64, NULL}) = 0 ioctl(4, SIOCGIFCONF, {64, {{"lo", {AF_INET, inet_addr("127.0.0.1")}}, {"eth0", {AF_INET, inet_addr("192.168.0.10")}}}}) = 0 ioctl(4, SIOCGIFFLAGS, {ifr_name="lo", ifr_flags=IFF_UP|IFF_LOOPBACK|IFF_RUNNING}) = 0 ioctl(4, SIOCGIFFLAGS, {ifr_name="eth0", ifr_flags=IFF_UP|IFF_BROADCAST|IFF_RUNNING|IFF_MULTICAST}) = 0 ioctl(4, SIOCGIFHWADDR, {ifr_name="eth0", ifr_hwaddr=00:c0:26:87:55:c0}) = 0 socket(PF_PACKET, SOCK_PACKET, 768) = -1 EPERM (Operation not permitted) time(NULL) = 1284760816 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2945, ...}) = 0 send(3, "Sep 18 00:00:16 dhcpd: Open "..., 74, MSG_NOSIGNAL) = 74 write(2, "Open a socket for LPF: Operation "..., 46Open a socket for LPF: Operation not permitted) = 46 write(2, "\n"..., 1 ) = 1 exit_group(1) = ? I understand that dhcpd wants to create sockets on port 67... but I don't know how to authorize that through the chroot. Any idea?

Read the article

value types in the vm

- by john.rose

value types in the vm p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 14.0px Times} p.p2 {margin: 0.0px 0.0px 14.0px 0.0px; font: 14.0px Times} p.p3 {margin: 0.0px 0.0px 12.0px 0.0px; font: 14.0px Times} p.p4 {margin: 0.0px 0.0px 15.0px 0.0px; font: 14.0px Times} p.p5 {margin: 0.0px 0.0px 0.0px 0.0px; font: 14.0px Courier} p.p6 {margin: 0.0px 0.0px 0.0px 0.0px; font: 14.0px Courier; min-height: 17.0px} p.p7 {margin: 0.0px 0.0px 0.0px 0.0px; font: 14.0px Times; min-height: 18.0px} p.p8 {margin: 0.0px 0.0px 0.0px 36.0px; text-indent: -36.0px; font: 14.0px Times; min-height: 18.0px} p.p9 {margin: 0.0px 0.0px 12.0px 0.0px; font: 14.0px Times; min-height: 18.0px} p.p10 {margin: 0.0px 0.0px 12.0px 0.0px; font: 14.0px Times; color: #000000} li.li1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 14.0px Times} li.li7 {margin: 0.0px 0.0px 0.0px 0.0px; font: 14.0px Times; min-height: 18.0px} span.s1 {font: 14.0px Courier} span.s2 {color: #000000} span.s3 {font: 14.0px Courier; color: #000000} ol.ol1 {list-style-type: decimal} Or, enduring values for a changing world. Introduction A value type is a data type which, generally speaking, is designed for being passed by value in and out of methods, and stored by value in data structures. The only value types which the Java language directly supports are the eight primitive types. Java indirectly and approximately supports value types, if they are implemented in terms of classes. For example, both Integer and String may be viewed as value types, especially if their usage is restricted to avoid operations appropriate to Object. In this note, we propose a definition of value types in terms of a design pattern for Java classes, accompanied by a set of usage restrictions. We also sketch the relation of such value types to tuple types (which are a JVM-level notion), and point out JVM optimizations that can apply to value types. This note is a thought experiment to extend the JVM’s performance model in support of value types. The demonstration has two phases. Initially the extension can simply use design patterns, within the current bytecode architecture, and in today’s Java language. But if the performance model is to be realized in practice, it will probably require new JVM bytecode features, changes to the Java language, or both. We will look at a few possibilities for these new features. An Axiom of Value In the context of the JVM, a value type is a data type equipped with construction, assignment, and equality operations, and a set of typed components, such that, whenever two variables of the value type produce equal corresponding values for their components, the values of the two variables cannot be distinguished by any JVM operation. Here are some corollaries: A value type is immutable, since otherwise a copy could be constructed and the original could be modified in one of its components, allowing the copies to be distinguished. Changing the component of a value type requires construction of a new value. The equals and hashCode operations are strictly component-wise. If a value type is represented by a JVM reference, that reference cannot be successfully synchronized on, and cannot be usefully compared for reference equality. A value type can be viewed in terms of what it doesn’t do. We can say that a value type omits all value-unsafe operations, which could violate the constraints on value types. These operations, which are ordinarily allowed for Java object types, are pointer equality comparison (the acmp instruction), synchronization (the monitor instructions), all the wait and notify methods of class Object, and non-trivial finalize methods. The clone method is also value-unsafe, although for value types it could be treated as the identity function. Finally, and most importantly, any side effect on an object (however visible) also counts as an value-unsafe operation. A value type may have methods, but such methods must not change the components of the value. It is reasonable and useful to define methods like toString, equals, and hashCode on value types, and also methods which are specifically valuable to users of the value type. Representations of Value Value types have two natural representations in the JVM, unboxed and boxed. An unboxed value consists of the components, as simple variables. For example, the complex number x=(1+2i), in rectangular coordinate form, may be represented in unboxed form by the following pair of variables: /*Complex x = Complex.valueOf(1.0, 2.0):*/ double x_re = 1.0, x_im = 2.0; These variables might be locals, parameters, or fields. Their association as components of a single value is not defined to the JVM. Here is a sample computation which computes the norm of the difference between two complex numbers: double distance(/*Complex x:*/ double x_re, double x_im, /*Complex y:*/ double y_re, double y_im) { /*Complex z = x.minus(y):*/ double z_re = x_re - y_re, z_im = x_im - y_im; /*return z.abs():*/ return Math.sqrt(z_re*z_re + z_im*z_im); } A boxed representation groups component values under a single object reference. The reference is to a ‘wrapper class’ that carries the component values in its fields. (A primitive type can naturally be equated with a trivial value type with just one component of that type. In that view, the wrapper class Integer can serve as a boxed representation of value type int.) The unboxed representation of complex numbers is practical for many uses, but it fails to cover several major use cases: return values, array elements, and generic APIs. The two components of a complex number cannot be directly returned from a Java function, since Java does not support multiple return values. The same story applies to array elements: Java has no ’array of structs’ feature. (Double-length arrays are a possible workaround for complex numbers, but not for value types with heterogeneous components.) By generic APIs I mean both those which use generic types, like Arrays.asList and those which have special case support for primitive types, like String.valueOf and PrintStream.println. Those APIs do not support unboxed values, and offer some problems to boxed values. Any ’real’ JVM type should have a story for returns, arrays, and API interoperability. The basic problem here is that value types fall between primitive types and object types. Value types are clearly more complex than primitive types, and object types are slightly too complicated. Objects are a little bit dangerous to use as value carriers, since object references can be compared for pointer equality, and can be synchronized on. Also, as many Java programmers have observed, there is often a performance cost to using wrapper objects, even on modern JVMs. Even so, wrapper classes are a good starting point for talking about value types. If there were a set of structural rules and restrictions which would prevent value-unsafe operations on value types, wrapper classes would provide a good notation for defining value types. This note attempts to define such rules and restrictions. Let’s Start Coding Now it is time to look at some real code. Here is a definition, written in Java, of a complex number value type. @ValueSafe public final class Complex implements java.io.Serializable { // immutable component structure: public final double re, im; private Complex(double re, double im) { this.re = re; this.im = im; } // interoperability methods: public String toString() { return "Complex("+re+","+im+")"; } public List<Double> asList() { return Arrays.asList(re, im); } public boolean equals(Complex c) { return re == c.re && im == c.im; } public boolean equals(@ValueSafe Object x) { return x instanceof Complex && equals((Complex) x); } public int hashCode() { return 31*Double.valueOf(re).hashCode() + Double.valueOf(im).hashCode(); } // factory methods: public static Complex valueOf(double re, double im) { return new Complex(re, im); } public Complex changeRe(double re2) { return valueOf(re2, im); } public Complex changeIm(double im2) { return valueOf(re, im2); } public static Complex cast(@ValueSafe Object x) { return x == null ? ZERO : (Complex) x; } // utility methods and constants: public Complex plus(Complex c) { return new Complex(re+c.re, im+c.im); } public Complex minus(Complex c) { return new Complex(re-c.re, im-c.im); } public double abs() { return Math.sqrt(re*re + im*im); } public static final Complex PI = valueOf(Math.PI, 0.0); public static final Complex ZERO = valueOf(0.0, 0.0); } This is not a minimal definition, because it includes some utility methods and other optional parts. The essential elements are as follows: The class is marked as a value type with an annotation. The class is final, because it does not make sense to create subclasses of value types. The fields of the class are all non-private and final. (I.e., the type is immutable and structurally transparent.) From the supertype Object, all public non-final methods are overridden. The constructor is private. Beyond these bare essentials, we can observe the following features in this example, which are likely to be typical of all value types: One or more factory methods are responsible for value creation, including a component-wise valueOf method. There are utility methods for complex arithmetic and instance creation, such as plus and changeIm. There are static utility constants, such as PI. The type is serializable, using the default mechanisms. There are methods for converting to and from dynamically typed references, such as asList and cast. The Rules In order to use value types properly, the programmer must avoid value-unsafe operations. A helpful Java compiler should issue errors (or at least warnings) for code which provably applies value-unsafe operations, and should issue warnings for code which might be correct but does not provably avoid value-unsafe operations. No such compilers exist today, but to simplify our account here, we will pretend that they do exist. A value-safe type is any class, interface, or type parameter marked with the @ValueSafe annotation, or any subtype of a value-safe type. If a value-safe class is marked final, it is in fact a value type. All other value-safe classes must be abstract. The non-static fields of a value class must be non-public and final, and all its constructors must be private. Under the above rules, a standard interface could be helpful to define value types like Complex. Here is an example: @ValueSafe public interface ValueType extends java.io.Serializable { // All methods listed here must get redefined. // Definitions must be value-safe, which means // they may depend on component values only. List<? extends Object> asList(); int hashCode(); boolean equals(@ValueSafe Object c); String toString(); } //@ValueSafe inherited from supertype: public final class Complex implements ValueType { … The main advantage of such a conventional interface is that (unlike an annotation) it is reified in the runtime type system. It could appear as an element type or parameter bound, for facilities which are designed to work on value types only. More broadly, it might assist the JVM to perform dynamic enforcement of the rules for value types. Besides types, the annotation @ValueSafe can mark fields, parameters, local variables, and methods. (This is redundant when the type is also value-safe, but may be useful when the type is Object or another supertype of a value type.) Working forward from these annotations, an expression E is defined as value-safe if it satisfies one or more of the following: The type of E is a value-safe type. E names a field, parameter, or local variable whose declaration is marked @ValueSafe. E is a call to a method whose declaration is marked @ValueSafe. E is an assignment to a value-safe variable, field reference, or array reference. E is a cast to a value-safe type from a value-safe expression. E is a conditional expression E0 ? E1 : E2, and both E1 and E2 are value-safe. Assignments to value-safe expressions and initializations of value-safe names must take their values from value-safe expressions. A value-safe expression may not be the subject of a value-unsafe operation. In particular, it cannot be synchronized on, nor can it be compared with the “==” operator, not even with a null or with another value-safe type. In a program where all of these rules are followed, no value-type value will be subject to a value-unsafe operation. Thus, the prime axiom of value types will be satisfied, that no two value type will be distinguishable as long as their component values are equal. More Code To illustrate these rules, here are some usage examples for Complex: Complex pi = Complex.valueOf(Math.PI, 0); Complex zero = pi.changeRe(0); //zero = pi; zero.re = 0; ValueType vtype = pi; @SuppressWarnings("value-unsafe") Object obj = pi; @ValueSafe Object obj2 = pi; obj2 = new Object(); // ok List<Complex> clist = new ArrayList<Complex>(); clist.add(pi); // (ok assuming List.add param is @ValueSafe) List<ValueType> vlist = new ArrayList<ValueType>(); vlist.add(pi); // (ok) List<Object> olist = new ArrayList<Object>(); olist.add(pi); // warning: "value-unsafe" boolean z = pi.equals(zero); boolean z1 = (pi == zero); // error: reference comparison on value type boolean z2 = (pi == null); // error: reference comparison on value type boolean z3 = (pi == obj2); // error: reference comparison on value type synchronized (pi) { } // error: synch of value, unpredictable result synchronized (obj2) { } // unpredictable result Complex qq = pi; qq = null; // possible NPE; warning: “null-unsafe" qq = (Complex) obj; // warning: “null-unsafe" qq = Complex.cast(obj); // OK @SuppressWarnings("null-unsafe") Complex empty = null; // possible NPE qq = empty; // possible NPE (null pollution) The Payoffs It follows from this that either the JVM or the java compiler can replace boxed value-type values with unboxed ones, without affecting normal computations. Fields and variables of value types can be split into their unboxed components. Non-static methods on value types can be transformed into static methods which take the components as value parameters. Some common questions arise around this point in any discussion of value types. Why burden the programmer with all these extra rules? Why not detect programs automagically and perform unboxing transparently? The answer is that it is easy to break the rules accidently unless they are agreed to by the programmer and enforced. Automatic unboxing optimizations are tantalizing but (so far) unreachable ideal. In the current state of the art, it is possible exhibit benchmarks in which automatic unboxing provides the desired effects, but it is not possible to provide a JVM with a performance model that assures the programmer when unboxing will occur. This is why I’m writing this note, to enlist help from, and provide assurances to, the programmer. Basically, I’m shooting for a good set of user-supplied “pragmas” to frame the desired optimization. Again, the important thing is that the unboxing must be done reliably, or else programmers will have no reason to work with the extra complexity of the value-safety rules. There must be a reasonably stable performance model, wherein using a value type has approximately the same performance characteristics as writing the unboxed components as separate Java variables. There are some rough corners to the present scheme. Since Java fields and array elements are initialized to null, value-type computations which incorporate uninitialized variables can produce null pointer exceptions. One workaround for this is to require such variables to be null-tested, and the result replaced with a suitable all-zero value of the value type. That is what the “cast” method does above. Generically typed APIs like List<T> will continue to manipulate boxed values always, at least until we figure out how to do reification of generic type instances. Use of such APIs will elicit warnings until their type parameters (and/or relevant members) are annotated or typed as value-safe. Retrofitting List<T> is likely to expose flaws in the present scheme, which we will need to engineer around. Here are a couple of first approaches: public interface java.util.List<@ValueSafe T> extends Collection<T> { … public interface java.util.List<T extends Object|ValueType> extends Collection<T> { … (The second approach would require disjunctive types, in which value-safety is “contagious” from the constituent types.) With more transformations, the return value types of methods can also be unboxed. This may require significant bytecode-level transformations, and would work best in the presence of a bytecode representation for multiple value groups, which I have proposed elsewhere under the title “Tuples in the VM”. But for starters, the JVM can apply this transformation under the covers, to internally compiled methods. This would give a way to express multiple return values and structured return values, which is a significant pain-point for Java programmers, especially those who work with low-level structure types favored by modern vector and graphics processors. The lack of multiple return values has a strong distorting effect on many Java APIs. Even if the JVM fails to unbox a value, there is still potential benefit to the value type. Clustered computing systems something have copy operations (serialization or something similar) which apply implicitly to command operands. When copying JVM objects, it is extremely helpful to know when an object’s identity is important or not. If an object reference is a copied operand, the system may have to create a proxy handle which points back to the original object, so that side effects are visible. Proxies must be managed carefully, and this can be expensive. On the other hand, value types are exactly those types which a JVM can “copy and forget” with no downside. Array types are crucial to bulk data interfaces. (As data sizes and rates increase, bulk data becomes more important than scalar data, so arrays are definitely accompanying us into the future of computing.) Value types are very helpful for adding structure to bulk data, so a successful value type mechanism will make it easier for us to express richer forms of bulk data. Unboxing arrays (i.e., arrays containing unboxed values) will provide better cache and memory density, and more direct data movement within clustered or heterogeneous computing systems. They require the deepest transformations, relative to today’s JVM. There is an impedance mismatch between value-type arrays and Java’s covariant array typing, so compromises will need to be struck with existing Java semantics. It is probably worth the effort, since arrays of unboxed value types are inherently more memory-efficient than standard Java arrays, which rely on dependent pointer chains. It may be sufficient to extend the “value-safe” concept to array declarations, and allow low-level transformations to change value-safe array declarations from the standard boxed form into an unboxed tuple-based form. Such value-safe arrays would not be convertible to Object[] arrays. Certain connection points, such as Arrays.copyOf and System.arraycopy might need additional input/output combinations, to allow smooth conversion between arrays with boxed and unboxed elements. Alternatively, the correct solution may have to wait until we have enough reification of generic types, and enough operator overloading, to enable an overhaul of Java arrays. Implicit Method Definitions The example of class Complex above may be unattractively complex. I believe most or all of the elements of the example class are required by the logic of value types. If this is true, a programmer who writes a value type will have to write lots of error-prone boilerplate code. On the other hand, I think nearly all of the code (except for the domain-specific parts like plus and minus) can be implicitly generated. Java has a rule for implicitly defining a class’s constructor, if no it defines no constructors explicitly. Likewise, there are rules for providing default access modifiers for interface members. Because of the highly regular structure of value types, it might be reasonable to perform similar implicit transformations on value types. Here’s an example of a “highly implicit” definition of a complex number type: public class Complex implements ValueType { // implicitly final public double re, im; // implicitly public final //implicit methods are defined elementwise from te fields: // toString, asList, equals(2), hashCode, valueOf, cast //optionally, explicit methods (plus, abs, etc.) would go here } In other words, with the right defaults, a simple value type definition can be a one-liner. The observant reader will have noticed the similarities (and suitable differences) between the explicit methods above and the corresponding methods for List<T>. Another way to abbreviate such a class would be to make an annotation the primary trigger of the functionality, and to add the interface(s) implicitly: public @ValueType class Complex { … // implicitly final, implements ValueType (But to me it seems better to communicate the “magic” via an interface, even if it is rooted in an annotation.) Implicitly Defined Value Types So far we have been working with nominal value types, which is to say that the sequence of typed components is associated with a name and additional methods that convey the intention of the programmer. A simple ordered pair of floating point numbers can be variously interpreted as (to name a few possibilities) a rectangular or polar complex number or Cartesian point. The name and the methods convey the intended meaning. But what if we need a truly simple ordered pair of floating point numbers, without any further conceptual baggage? Perhaps we are writing a method (like “divideAndRemainder”) which naturally returns a pair of numbers instead of a single number. Wrapping the pair of numbers in a nominal type (like “QuotientAndRemainder”) makes as little sense as wrapping a single return value in a nominal type (like “Quotient”). What we need here are structural value types commonly known as tuples. For the present discussion, let us assign a conventional, JVM-friendly name to tuples, roughly as follows: public class java.lang.tuple.$DD extends java.lang.tuple.Tuple { double $1, $2; } Here the component names are fixed and all the required methods are defined implicitly. The supertype is an abstract class which has suitable shared declarations. The name itself mentions a JVM-style method parameter descriptor, which may be “cracked” to determine the number and types of the component fields. The odd thing about such a tuple type (and structural types in general) is it must be instantiated lazily, in response to linkage requests from one or more classes that need it. The JVM and/or its class loaders must be prepared to spin a tuple type on demand, given a simple name reference, $xyz, where the xyz is cracked into a series of component types. (Specifics of naming and name mangling need some tasteful engineering.) Tuples also seem to demand, even more than nominal types, some support from the language. (This is probably because notations for non-nominal types work best as combinations of punctuation and type names, rather than named constructors like Function3 or Tuple2.) At a minimum, languages with tuples usually (I think) have some sort of simple bracket notation for creating tuples, and a corresponding pattern-matching syntax (or “destructuring bind”) for taking tuples apart, at least when they are parameter lists. Designing such a syntax is no simple thing, because it ought to play well with nominal value types, and also with pre-existing Java features, such as method parameter lists, implicit conversions, generic types, and reflection. That is a task for another day. Other Use Cases Besides complex numbers and simple tuples there are many use cases for value types. Many tuple-like types have natural value-type representations. These include rational numbers, point locations and pixel colors, and various kinds of dates and addresses. Other types have a variable-length ‘tail’ of internal values. The most common example of this is String, which is (mathematically) a sequence of UTF-16 character values. Similarly, bit vectors, multiple-precision numbers, and polynomials are composed of sequences of values. Such types include, in their representation, a reference to a variable-sized data structure (often an array) which (somehow) represents the sequence of values. The value type may also include ’header’ information. Variable-sized values often have a length distribution which favors short lengths. In that case, the design of the value type can make the first few values in the sequence be direct ’header’ fields of the value type. In the common case where the header is enough to represent the whole value, the tail can be a shared null value, or even just a null reference. Note that the tail need not be an immutable object, as long as the header type encapsulates it well enough. This is the case with String, where the tail is a mutable (but never mutated) character array. Field types and their order must be a globally visible part of the API. The structure of the value type must be transparent enough to have a globally consistent unboxed representation, so that all callers and callees agree about the type and order of components that appear as parameters, return types, and array elements. This is a trade-off between efficiency and encapsulation, which is forced on us when we remove an indirection enjoyed by boxed representations. A JVM-only transformation would not care about such visibility, but a bytecode transformation would need to take care that (say) the components of complex numbers would not get swapped after a redefinition of Complex and a partial recompile. Perhaps constant pool references to value types need to declare the field order as assumed by each API user. This brings up the delicate status of private fields in a value type. It must always be possible to load, store, and copy value types as coordinated groups, and the JVM performs those movements by moving individual scalar values between locals and stack. If a component field is not public, what is to prevent hostile code from plucking it out of the tuple using a rogue aload or astore instruction? Nothing but the verifier, so we may need to give it more smarts, so that it treats value types as inseparable groups of stack slots or locals (something like long or double). My initial thought was to make the fields always public, which would make the security problem moot. But public is not always the right answer; consider the case of String, where the underlying mutable character array must be encapsulated to prevent security holes. I believe we can win back both sides of the tradeoff, by training the verifier never to split up the components in an unboxed value. Just as the verifier encapsulates the two halves of a 64-bit primitive, it can encapsulate the the header and body of an unboxed String, so that no code other than that of class String itself can take apart the values. Similar to String, we could build an efficient multi-precision decimal type along these lines: public final class DecimalValue extends ValueType { protected final long header; protected private final BigInteger digits; public DecimalValue valueOf(int value, int scale) { assert(scale >= 0); return new DecimalValue(((long)value << 32) + scale, null); } public DecimalValue valueOf(long value, int scale) { if (value == (int) value) return valueOf((int)value, scale); return new DecimalValue(-scale, new BigInteger(value)); } } Values of this type would be passed between methods as two machine words. Small values (those with a significand which fits into 32 bits) would be represented without any heap data at all, unless the DecimalValue itself were boxed. (Note the tension between encapsulation and unboxing in this case. It would be better if the header and digits fields were private, but depending on where the unboxing information must “leak”, it is probably safer to make a public revelation of the internal structure.) Note that, although an array of Complex can be faked with a double-length array of double, there is no easy way to fake an array of unboxed DecimalValues. (Either an array of boxed values or a transposed pair of homogeneous arrays would be reasonable fallbacks, in a current JVM.) Getting the full benefit of unboxing and arrays will require some new JVM magic. Although the JVM emphasizes portability, system dependent code will benefit from using machine-level types larger than 64 bits. For example, the back end of a linear algebra package might benefit from value types like Float4 which map to stock vector types. This is probably only worthwhile if the unboxing arrays can be packed with such values. More Daydreams A more finely-divided design for dynamic enforcement of value safety could feature separate marker interfaces for each invariant. An empty marker interface Unsynchronizable could cause suitable exceptions for monitor instructions on objects in marked classes. More radically, a Interchangeable marker interface could cause JVM primitives that are sensitive to object identity to raise exceptions; the strangest result would be that the acmp instruction would have to be specified as raising an exception. @ValueSafe public interface ValueType extends java.io.Serializable, Unsynchronizable, Interchangeable { … public class Complex implements ValueType { // inherits Serializable, Unsynchronizable, Interchangeable, @ValueSafe … It seems possible that Integer and the other wrapper types could be retro-fitted as value-safe types. This is a major change, since wrapper objects would be unsynchronizable and their references interchangeable. It is likely that code which violates value-safety for wrapper types exists but is uncommon. It is less plausible to retro-fit String, since the prominent operation String.intern is often used with value-unsafe code. We should also reconsider the distinction between boxed and unboxed values in code. The design presented above obscures that distinction. As another thought experiment, we could imagine making a first class distinction in the type system between boxed and unboxed representations. Since only primitive types are named with a lower-case initial letter, we could define that the capitalized version of a value type name always refers to the boxed representation, while the initial lower-case variant always refers to boxed. For example: complex pi = complex.valueOf(Math.PI, 0); Complex boxPi = pi; // convert to boxed myList.add(boxPi); complex z = myList.get(0); // unbox Such a convention could perhaps absorb the current difference between int and Integer, double and Double. It might also allow the programmer to express a helpful distinction among array types. As said above, array types are crucial to bulk data interfaces, but are limited in the JVM. Extending arrays beyond the present limitations is worth thinking about; for example, the Maxine JVM implementation has a hybrid object/array type. Something like this which can also accommodate value type components seems worthwhile. On the other hand, does it make sense for value types to contain short arrays? And why should random-access arrays be the end of our design process, when bulk data is often sequentially accessed, and it might make sense to have heterogeneous streams of data as the natural “jumbo” data structure. These considerations must wait for another day and another note. More Work It seems to me that a good sequence for introducing such value types would be as follows: Add the value-safety restrictions to an experimental version of javac. Code some sample applications with value types, including Complex and DecimalValue. Create an experimental JVM which internally unboxes value types but does not require new bytecodes to do so. Ensure the feasibility of the performance model for the sample applications. Add tuple-like bytecodes (with or without generic type reification) to a major revision of the JVM, and teach the Java compiler to switch in the new bytecodes without code changes. A staggered roll-out like this would decouple language changes from bytecode changes, which is always a convenient thing. A similar investigation should be applied (concurrently) to array types. In this case, it seems to me that the starting point is in the JVM: Add an experimental unboxing array data structure to a production JVM, perhaps along the lines of Maxine hybrids. No bytecode or language support is required at first; everything can be done with encapsulated unsafe operations and/or method handles. Create an experimental JVM which internally unboxes value types but does not require new bytecodes to do so. Ensure the feasibility of the performance model for the sample applications. Add tuple-like bytecodes (with or without generic type reification) to a major revision of the JVM, and teach the Java compiler to switch in the new bytecodes without code changes. That’s enough musing me for now. Back to work!

Read the article

Java Cloud Service Integration to REST Service

- by Jani Rautiainen

Service (JCS) provides a platform to develop and deploy business applications in the cloud. In Fusion Applications Cloud deployments customers do not have the option to deploy custom applications developed with JDeveloper to ensure the integrity and supportability of the hosted application service. Instead the custom applications can be deployed to the JCS and integrated to the Fusion Application Cloud instance. This series of articles will go through the features of JCS, provide end-to-end examples on how to develop and deploy applications on JCS and how to integrate them with the Fusion Applications instance. In this article a custom application integrating with REST service will be implemented. We will use REST services provided by Taleo as an example; however the same approach will work with any REST service. In this example the data from the REST service is used to populate a dynamic table. Pre-requisites Access to Cloud instance In order to deploy the application access to a JCS instance is needed, a free trial JCS instance can be obtained from Oracle Cloud site. To register you will need a credit card even if the credit card will not be charged. To register simply click "Try it" and choose the "Java" option. The confirmation email will contain the connection details. See this video for example of the registration.Once the request is processed you will be assigned 2 service instances; Java and Database. Applications deployed to the JCS must use Oracle Database Cloud Service as their underlying database. So when JCS instance is created a database instance is associated with it using a JDBC data source.The cloud services can be monitored and managed through the web UI. For details refer to Getting Started with Oracle Cloud. JDeveloper JDeveloper contains Cloud specific features related to e.g. connection and deployment. To use these features download the JDeveloper from JDeveloper download site by clicking the "Download JDeveloper 11.1.1.7.1 for ADF deployment on Oracle Cloud" link, this version of JDeveloper will have the JCS integration features that will be used in this article. For versions that do not include the Cloud integration features the Oracle Java Cloud Service SDK or the JCS Java Console can be used for deployment. For details on installing and configuring the JDeveloper refer to the installation guideFor details on SDK refer to Using the Command-Line Interface to Monitor Oracle Java Cloud Service and Using the Command-Line Interface to Manage Oracle Java Cloud Service. Access to a local database The database associated with the JCS instance cannot be connected to with JDBC. Since creating ADFbc business component requires a JDBC connection we will need access to a local database. 3rd party libraries This example will use some 3rd party libraries for implementing the REST service call and processing the input / output content. Other libraries may also be used, however these are tested to work. Jersey 1.x Jersey library will be used as a client to make the call to the REST service. JCS documentation for supported specifications states: Java API for RESTful Web Services (JAX-RS) 1.1 So Jersey 1.x will be used. Download the single-JAR Jersey bundle; in this example Jersey 1.18 JAR bundle is used. Json-simple Jjson-simple library will be used to process the json objects. Download the JAR file; in this example json-simple-1.1.1.jar is used. Accessing data in Taleo Before implementing the application it is beneficial to familiarize oneself with the data in Taleo. Easiest way to do this is by using a RESTClient on your browser. Once added to the browser you can access the UI: The client can be used to call the REST services to test the URLs and data before adding them into the application. First derive the base URL for the service this can be done with: Method: GET URL: https://tbe.taleo.net/MANAGER/dispatcher/api/v1/serviceUrl/<company name> The response will contain the base URL to be used for the service calls for the company. Next obtain authentication token with: Method: POST URL: https://ch.tbe.taleo.net/CH07/ats/api/v1/login?orgCode=<company>&userName=<user name>&password=<password> The response includes an authentication token that can be used for few hours to authenticate with the service: { "response": { "authToken": "webapi26419680747505890557" }, "status": { "detail": {}, "success": true } } To authenticate the service calls navigate to "Headers -> Custom Header": And add a new request header with: Name: Cookie Value: authToken=webapi26419680747505890557 Once authentication token is defined the tool can be used to invoke REST services; for example: Method: GET URL: https://ch.tbe.taleo.net/CH07/ats/api/v1/object/candidate/search.xml?status=16 This data will be used on the application to be created. For details on the Taleo REST services refer to the Taleo Business Edition REST API Guide. Create Application First Fusion Web Application is created and configured. Start JDeveloper and click "New Application": Application Name: JcsRestDemo Application Package Prefix: oracle.apps.jcs.test Application Template: Fusion Web Application (ADF) Configure Local Cloud Connection Follow the steps documented in the "Java Cloud Service ADF Web Application" article to configure a local database connection needed to create the ADFbc objects. Configure Libraries Add the 3rd party libraries into the class path. Create the following directory and copy the jar files into it: <JDEV_USER_HOME>/JcsRestDemo/lib Select the "Model" project, navigate "Application -> Project Properties -> Libraries and Classpath -> Add JAR / Directory" and add the 2 3rd party libraries: Accessing Data from Taleo To access data from Taleo using the REST service the 3rd party libraries will be used. 2 Java classes are implemented, one representing the Candidate object and another for accessing the Taleo repository Candidate Candidate object is a POJO object used to represent the candidate data obtained from the Taleo repository. The data obtained will be used to populate the ADFbc object used to display the data on the UI. The candidate object contains simply the variables we obtain using the REST services and the getters / setters for them: Navigate "New -> General -> Java -> Java Class", enter "Candidate" as the name and create it in the package "oracle.apps.jcs.test.model". Copy / paste the following as the content: import oracle.jbo.domain.Number; public class Candidate { private Number candId; private String firstName; private String lastName; public Candidate() { super(); } public Candidate(Number candId, String firstName, String lastName) { super(); this.candId = candId; this.firstName = firstName; this.lastName = lastName; } public void setCandId(Number candId) { this.candId = candId; } public Number getCandId() { return candId; } public void setFirstName(String firstName) { this.firstName = firstName; } public String getFirstName() { return firstName; } public void setLastName(String lastName) { this.lastName = lastName; } public String getLastName() { return lastName; } } Taleo Repository Taleo repository class will interact with the Taleo REST services. The logic will query data from Taleo and populate Candidate objects with the data. The Candidate object will then be used to populate the ADFbc object used to display data on the UI. Navigate "New -> General -> Java -> Java Class", enter "TaleoRepository" as the name and create it in the package "oracle.apps.jcs.test.model". Copy / paste the following as the content (for details of the implementation refer to the documentation in the code): import com.sun.jersey.api.client.Client; import com.sun.jersey.api.client.ClientResponse; import com.sun.jersey.api.client.WebResource; import com.sun.jersey.core.util.MultivaluedMapImpl; import java.io.StringReader; import java.util.ArrayList; import java.util.Iterator; import java.util.List; import java.util.Map; import javax.ws.rs.core.MediaType; import javax.ws.rs.core.MultivaluedMap; import oracle.jbo.domain.Number; import org.json.simple.JSONArray; import org.json.simple.JSONObject; import org.json.simple.parser.JSONParser; /** * This class interacts with the Taleo REST services */ public class TaleoRepository { /** * Connection information needed to access the Taleo services */ String _company = null; String _userName = null; String _password = null; /** * Jersey client used to access the REST services */ Client _client = null; /** * Parser for processing the JSON objects used as * input / output for the services */ JSONParser _parser = null; /** * The base url for constructing the REST URLs. This is obtained * from Taleo with a service call */ String _baseUrl = null; /** * Authentication token obtained from Taleo using a service call. * The token can be used to authenticate on subsequent * service calls. The token will expire in 4 hours */ String _authToken = null; /** * Static url that can be used to obtain the url used to construct * service calls for a given company */ private static String _taleoUrl = "https://tbe.taleo.net/MANAGER/dispatcher/api/v1/serviceUrl/"; /** * Default constructor for the repository * Authentication details are passed as parameters and used to generate * authentication token. Note that each service call will * generate its own token. This is done to avoid dealing with the expiry * of the token. Also only 20 tokens are allowed per user simultaneously. * So instead for each call there is login / logout. * * @param company the company for which the service calls are made * @param userName the user name to authenticate with * @param password the password to authenticate with. */ public TaleoRepository(String company, String userName, String password) { super(); _company = company; _userName = userName; _password = password; _client = Client.create(); _parser = new JSONParser(); _baseUrl = getBaseUrl(); } /** * This obtains the base url for a company to be used * to construct the urls for service calls * @return base url for the service calls */ private String getBaseUrl() { String result = null; if (null != _baseUrl) { result = _baseUrl; } else { try { String company = _company; WebResource resource = _client.resource(_taleoUrl + company); ClientResponse response = resource.type(MediaType.APPLICATION_FORM_URLENCODED_TYPE).get(ClientResponse.class); String entity = response.getEntity(String.class); JSONObject jsonObject = (JSONObject)_parser.parse(new StringReader(entity)); JSONObject jsonResponse = (JSONObject)jsonObject.get("response"); result = (String)jsonResponse.get("URL"); } catch (Exception ex) { ex.printStackTrace(); } } return result; } /** * Generates authentication token, that can be used to authenticate on * subsequent service calls. Note that each service call will * generate its own token. This is done to avoid dealing with the expiry * of the token. Also only 20 tokens are allowed per user simultaneously. * So instead for each call there is login / logout. * @return authentication token that can be used to authenticate on * subsequent service calls */ private String login() { String result = null; try { MultivaluedMap<String, String> formData = new MultivaluedMapImpl(); formData.add("orgCode", _company); formData.add("userName", _userName); formData.add("password", _password); WebResource resource = _client.resource(_baseUrl + "login"); ClientResponse response = resource.type(MediaType.APPLICATION_FORM_URLENCODED_TYPE).post(ClientResponse.class, formData); String entity = response.getEntity(String.class); JSONObject jsonObject = (JSONObject)_parser.parse(new StringReader(entity)); JSONObject jsonResponse = (JSONObject)jsonObject.get("response"); result = (String)jsonResponse.get("authToken"); } catch (Exception ex) { throw new RuntimeException("Unable to login ", ex); } if (null == result) throw new RuntimeException("Unable to login "); return result; } /** * Releases a authentication token. Each call to login must be followed * by call to logout after the processing is done. This is required as * the tokens are limited to 20 per user and if not released the tokens * will only expire after 4 hours. * @param authToken */ private void logout(String authToken) { WebResource resource = _client.resource(_baseUrl + "logout"); resource.header("cookie", "authToken=" + authToken).post(ClientResponse.class); } /** * This method is used to obtain a list of candidates using a REST * service call. At this example the query is hard coded to query * based on status. The url constructed to access the service is: * <_baseUrl>/object/candidate/search.xml?status=16 * @return List of candidates obtained with the service call */ public List<Candidate> getCandidates() { List<Candidate> result = new ArrayList<Candidate>(); try { // First login, note that in finally block we must have logout _authToken = "authToken=" + login(); /** * Construct the URL, the resulting url will be: * <_baseUrl>/object/candidate/search.xml?status=16 */ MultivaluedMap<String, String> formData = new MultivaluedMapImpl(); formData.add("status", "16"); JSONArray searchResults = (JSONArray)getTaleoResource("object/candidate/search", "searchResults", formData); /** * Process the results, the resulting JSON object is something like * this (simplified for readability): * * { * "response": * { * "searchResults": * [ * { * "candidate": * { * "candId": 211, * "firstName": "Mary", * "lastName": "Stochi", * logic here will find the candidate object(s), obtain the desired * data from them, construct a Candidate object based on the data * and add it to the results. */ for (Object object : searchResults) { JSONObject temp = (JSONObject)object; JSONObject candidate = (JSONObject)findObject(temp, "candidate"); Long candIdTemp = (Long)candidate.get("candId"); Number candId = (null == candIdTemp ? null : new Number(candIdTemp)); String firstName = (String)candidate.get("firstName"); String lastName = (String)candidate.get("lastName"); result.add(new Candidate(candId, firstName, lastName)); } } catch (Exception ex) { ex.printStackTrace(); } finally { if (null != _authToken) logout(_authToken); } return result; } /** * Convenience method to construct url for the service call, invoke the * service and obtain a resource from the response * @param path the path for the service to be invoked. This is combined * with the base url to construct a url for the service * @param resource the key for the object in the response that will be * obtained * @param parameters any parameters used for the service call. The call * is slightly different depending whether parameters exist or not. * @return the resource from the response for the service call */ private Object getTaleoResource(String path, String resource, MultivaluedMap<String, String> parameters) { Object result = null; try { WebResource webResource = _client.resource(_baseUrl + path); ClientResponse response = null; if (null == parameters) response = webResource.header("cookie", _authToken).get(ClientResponse.class); else response = webResource.queryParams(parameters).header("cookie", _authToken).get(ClientResponse.class); String entity = response.getEntity(String.class); JSONObject jsonObject = (JSONObject)_parser.parse(new StringReader(entity)); result = findObject(jsonObject, resource); } catch (Exception ex) { ex.printStackTrace(); } return result; } /** * Convenience method to recursively find a object with an key * traversing down from a given root object. This will traverse a * JSONObject / JSONArray recursively to find a matching key, if found * the object with the key is returned. * @param root root object which contains the key searched for * @param key the key for the object to search for * @return the object matching the key */ private Object findObject(Object root, String key) { Object result = null; if (root instanceof JSONObject) { JSONObject rootJSON = (JSONObject)root; if (rootJSON.containsKey(key)) { result = rootJSON.get(key); } else { Iterator children = rootJSON.entrySet().iterator(); while (children.hasNext()) { Map.Entry entry = (Map.Entry)children.next(); Object child = entry.getValue(); if (child instanceof JSONObject || child instanceof JSONArray) { result = findObject(child, key); if (null != result) break; } } } } else if (root instanceof JSONArray) { JSONArray rootJSON = (JSONArray)root; for (Object child : rootJSON) { if (child instanceof JSONObject || child instanceof JSONArray) { result = findObject(child, key); if (null != result) break; } } } return result; } } Creating Business Objects While JCS application can be created without a local database, the local database is required when using ADFbc objects even if database objects are not referred. For this example we will create a "Transient" view object that will be programmatically populated based the data obtained from Taleo REST services. Creating ADFbc objects Choose the "Model" project and navigate "New -> Business Tier : ADF Business Components : View Object". On the "Initialize Business Components Project" choose the local database connection created in previous step. On Step 1 enter "JcsRestDemoVO" on the "Name" and choose "Rows populated programmatically, not based on query": On step 2 create the following attributes: CandId Type: Number Updatable: Always Key Attribute: checked Name Type: String Updatable: Always On steps 3 and 4 accept defaults and click "Next". On step 5 check the "Application Module" checkbox and enter "JcsRestDemoAM" as the name: Click "Finish" to generate the objects. Populating the VO To display the data on the UI the "transient VO" is populated programmatically based on the data obtained from the Taleo REST services. Open the "JcsRestDemoVOImpl.java". Copy / paste the following as the content (for details of the implementation refer to the documentation in the code): import java.sql.ResultSet; import java.util.List; import java.util.ListIterator; import oracle.jbo.server.ViewObjectImpl; import oracle.jbo.server.ViewRowImpl; import oracle.jbo.server.ViewRowSetImpl; // --------------------------------------------------------------------- // --- File generated by Oracle ADF Business Components Design Time. // --- Tue Feb 18 09:40:25 PST 2014 // --- Custom code may be added to this class. // --- Warning: Do not modify method signatures of generated methods. // --------------------------------------------------------------------- public class JcsRestDemoVOImpl extends ViewObjectImpl { /** * This is the default constructor (do not remove). */ public JcsRestDemoVOImpl() { } @Override public void executeQuery() { /** * For some reason we need to reset everything, otherwise * 2nd entry to the UI screen may fail with * "java.util.NoSuchElementException" in createRowFromResultSet * call to "candidates.next()". I am not sure why this is happening * as the Iterator is new and "hasNext" is true at the point * of the execution. My theory is that since the iterator object is * exactly the same the VO cache somehow reuses the iterator including * the pointer that has already exhausted the iterable elements on the * previous run. Working around the issue * here by cleaning out everything on the VO every time before query * is executed on the VO. */ getViewDef().setQuery(null); getViewDef().setSelectClause(null); setQuery(null); this.reset(); this.clearCache(); super.executeQuery(); } /** * executeQueryForCollection - overridden for custom java data source support. */ protected void executeQueryForCollection(Object qc, Object[] params, int noUserParams) { /** * Integrate with the Taleo REST services using TaleoRepository class. * A list of candidates matching a hard coded query is obtained. */ TaleoRepository repository = new TaleoRepository(<company>, <username>, <password>); List<Candidate> candidates = repository.getCandidates(); /** * Store iterator for the candidates as user data on the collection. * This will be used in createRowFromResultSet to create rows based on * the custom iterator. */ ListIterator<Candidate> candidatescIterator = candidates.listIterator(); setUserDataForCollection(qc, candidatescIterator); super.executeQueryForCollection(qc, params, noUserParams); } /** * hasNextForCollection - overridden for custom java data source support. */ protected boolean hasNextForCollection(Object qc) { boolean result = false; /** * Determines whether there are candidates for which to create a row */ ListIterator<Candidate> candidates = (ListIterator<Candidate>)getUserDataForCollection(qc); result = candidates.hasNext(); /** * If all candidates to be created indicate that processing is done */ if (!result) { setFetchCompleteForCollection(qc, true); } return result; } /** * createRowFromResultSet - overridden for custom java data source support. */ protected ViewRowImpl createRowFromResultSet(Object qc, ResultSet resultSet) { /** * Obtain the next candidate from the collection and create a row * for it. */ ListIterator<Candidate> candidates = (ListIterator<Candidate>)getUserDataForCollection(qc); ViewRowImpl row = createNewRowForCollection(qc); try { Candidate candidate = candidates.next(); row.setAttribute("CandId", candidate.getCandId()); row.setAttribute("Name", candidate.getFirstName() + " " + candidate.getLastName()); } catch (Exception e) { e.printStackTrace(); } return row; } /** * getQueryHitCount - overridden for custom java data source support. */ public long getQueryHitCount(ViewRowSetImpl viewRowSet) { /** * For this example this is not implemented rather we always return 0. */ return 0; } } Creating UI Choose the "ViewController" project and navigate "New -> Web Tier : JSF : JSF Page". On the "Create JSF Page" enter "JcsRestDemo" as name and ensure that the "Create as XML document (*.jspx)" is checked. Open "JcsRestDemo.jspx" and navigate to "Data Controls -> JcsRestDemoAMDataControl -> JcsRestDemoVO1" and drag & drop the VO to the "<af:form> " as a "ADF Read-only Table": Accept the defaults in "Edit Table Columns". To execute the query navigate to to "Data Controls -> JcsRestDemoAMDataControl -> JcsRestDemoVO1 -> Operations -> Execute" and drag & drop the operation to the "<af:form> " as a "Button": Deploying to JCS Follow the same steps as documented in previous article"Java Cloud Service ADF Web Application". Once deployed the application can be accessed with URL: https://java-[identity domain].java.[data center].oraclecloudapps.com/JcsRestDemo-ViewController-context-root/faces/JcsRestDemo.jspx The UI displays a list of candidates obtained from the Taleo REST Services: Summary In this article we learned how to integrate with REST services using Jersey library in JCS. In future articles various other integration techniques will be covered.

Read the article

Troubleshoot Perl module installation on Mac OS X

- by Daniel Standage

I'm trying to install the Perl module Set::IntervalTree on Mac OS X. I recently installed it today on an Ubuntu box with no problem. I simply started cpan, entered install Set:IntervalTree, and it all worked out. However, the installation failed on Mac OS X--it spits out a huge list of compiler errors (below). How would I troubleshoot this. I don't even know where to begin. cpan[1]> install Set::IntervalTree CPAN: Storable loaded ok (v2.18) Going to read /Users/standage/.cpan/Metadata Database was generated on Fri, 14 Jan 2011 02:58:42 GMT CPAN: YAML loaded ok (v0.72) Going to read /Users/standage/.cpan/build/ ............................................................................DONE Found 1 old build, restored the state of 1 Running install for module 'Set::IntervalTree' Running make for B/BE/BENBOOTH/Set-IntervalTree-0.01.tar.gz CPAN: Digest::SHA loaded ok (v5.45) CPAN: Compress::Zlib loaded ok (v2.008) Checksum for /Users/standage/.cpan/sources/authors/id/B/BE/BENBOOTH/Set-IntervalTree-0.01.tar.gz ok Scanning cache /Users/standage/.cpan/build for sizes ............................................................................DONE x Set-IntervalTree-0.01/ x Set-IntervalTree-0.01/src/ x Set-IntervalTree-0.01/src/Makefile x Set-IntervalTree-0.01/src/interval_tree.h x Set-IntervalTree-0.01/src/test_main.cc x Set-IntervalTree-0.01/lib/ x Set-IntervalTree-0.01/lib/Set/ x Set-IntervalTree-0.01/lib/Set/IntervalTree.pm x Set-IntervalTree-0.01/Changes x Set-IntervalTree-0.01/MANIFEST x Set-IntervalTree-0.01/t/ x Set-IntervalTree-0.01/t/Set-IntervalTree.t x Set-IntervalTree-0.01/typemap x Set-IntervalTree-0.01/perlobject.map x Set-IntervalTree-0.01/IntervalTree.xs x Set-IntervalTree-0.01/Makefile.PL x Set-IntervalTree-0.01/README x Set-IntervalTree-0.01/META.yml CPAN: File::Temp loaded ok (v0.18) CPAN.pm: Going to build B/BE/BENBOOTH/Set-IntervalTree-0.01.tar.gz Checking if your kit is complete... Looks good Writing Makefile for Set::IntervalTree cp lib/Set/IntervalTree.pm blib/lib/Set/IntervalTree.pm AutoSplitting blib/lib/Set/IntervalTree.pm (blib/lib/auto/Set/IntervalTree) /usr/bin/perl /System/Library/Perl/5.10.0/ExtUtils/xsubpp -C++ -typemap /System/Library/Perl/5.10.0/ExtUtils/typemap -typemap perlobject.map -typemap typemap IntervalTree.xs > IntervalTree.xsc && mv IntervalTree.xsc IntervalTree.c g++ -c -Isrc -arch x86_64 -arch i386 -arch ppc -g -pipe -fno-common -DPERL_DARWIN -fno-strict-aliasing -I/usr/local/include -g -O0 -DVERSION=\"0.01\" -DXS_VERSION=\"0.01\" "-I/System/Library/Perl/5.10.0/darwin-thread-multi-2level/CORE" -Isrc IntervalTree.c In file included from /usr/include/c++/4.2.1/bits/basic_ios.h:44, from /usr/include/c++/4.2.1/ios:50, from /usr/include/c++/4.2.1/ostream:45, from /usr/include/c++/4.2.1/iostream:45, from IntervalTree.xs:16: /usr/include/c++/4.2.1/bits/locale_facets.h:4420:40: error: macro "do_open" requires 7 arguments, but only 2 given /usr/include/c++/4.2.1/bits/locale_facets.h:4467:34: error: macro "do_close" requires 2 arguments, but only 1 given /usr/include/c++/4.2.1/bits/locale_facets.h:4486:55: error: macro "do_open" requires 7 arguments, but only 2 given /usr/include/c++/4.2.1/bits/locale_facets.h:4513:23: error: macro "do_close" requires 2 arguments, but only 1 given In file included from /usr/include/c++/4.2.1/bits/locale_facets.h:4599, from /usr/include/c++/4.2.1/bits/basic_ios.h:44, from /usr/include/c++/4.2.1/ios:50, from /usr/include/c++/4.2.1/ostream:45, from /usr/include/c++/4.2.1/iostream:45, from IntervalTree.xs:16: /usr/include/c++/4.2.1/i686-apple-darwin10/x86_64/bits/messages_members.h:58:38: error: macro "do_open" requires 7 arguments, but only 2 given /usr/include/c++/4.2.1/i686-apple-darwin10/x86_64/bits/messages_members.h:67:71: error: macro "do_open" requires 7 arguments, but only 2 given /usr/include/c++/4.2.1/i686-apple-darwin10/x86_64/bits/messages_members.h:78:39: error: macro "do_close" requires 2 arguments, but only 1 given In file included from /usr/include/c++/4.2.1/bits/basic_ios.h:44, from /usr/include/c++/4.2.1/ios:50, from /usr/include/c++/4.2.1/ostream:45, from /usr/include/c++/4.2.1/iostream:45, from IntervalTree.xs:16: /usr/include/c++/4.2.1/bits/locale_facets.h:4486: error: ‘do_open’ declared as a ‘virtual’ field /usr/include/c++/4.2.1/bits/locale_facets.h:4486: error: expected ‘;’ before ‘const’ /usr/include/c++/4.2.1/bits/locale_facets.h:4513: error: variable or field ‘do_close’ declared void /usr/include/c++/4.2.1/bits/locale_facets.h:4513: error: expected ‘;’ before ‘const’ In file included from /usr/include/c++/4.2.1/bits/locale_facets.h:4599, from /usr/include/c++/4.2.1/bits/basic_ios.h:44, from /usr/include/c++/4.2.1/ios:50, from /usr/include/c++/4.2.1/ostream:45, from /usr/include/c++/4.2.1/iostream:45, from IntervalTree.xs:16: /usr/include/c++/4.2.1/i686-apple-darwin10/x86_64/bits/messages_members.h:67: error: expected initializer before ‘const’ /usr/include/c++/4.2.1/i686-apple-darwin10/x86_64/bits/messages_members.h:78: error: expected initializer before ‘const’ In file included from IntervalTree.xs:19: src/interval_tree.h:95: error: type/value mismatch at argument 1 in template parameter list for ‘template<class _Tp, class _Alloc> class std::vector’ src/interval_tree.h:95: error: expected a type, got ‘IntervalTree<T,N>::it_recursion_node’ src/interval_tree.h:95: error: template argument 2 is invalid src/interval_tree.h: In constructor ‘IntervalTree<T, N>::IntervalTree()’: src/interval_tree.h:130: error: expected type-specifier src/interval_tree.h:130: error: expected `;' src/interval_tree.h:135: error: expected type-specifier src/interval_tree.h:135: error: expected `;' src/interval_tree.h:141: error: request for member ‘push_back’ in ‘((IntervalTree<T, N>*)this)->IntervalTree<T, N>::recursionNodeStack’, which is of non-class type ‘int’ src/interval_tree.h: In member function ‘void IntervalTree<T, N>::LeftRotate(IntervalTree<T, N>::Node*)’: src/interval_tree.h:178: error: ‘y’ was not declared in this scope src/interval_tree.h: In member function ‘void IntervalTree<T, N>::RightRotate(IntervalTree<T, N>::Node*)’: src/interval_tree.h:240: error: ‘x’ was not declared in this scope src/interval_tree.h: In member function ‘void IntervalTree<T, N>::TreeInsertHelp(IntervalTree<T, N>::Node*)’: src/interval_tree.h:298: error: ‘x’ was not declared in this scope src/interval_tree.h:299: error: ‘y’ was not declared in this scope src/interval_tree.h: In member function ‘typename IntervalTree<T, N>::Node* IntervalTree<T, N>::insert(const T&, N, N)’: src/interval_tree.h:375: error: ‘y’ was not declared in this scope src/interval_tree.h:376: error: ‘x’ was not declared in this scope src/interval_tree.h:377: error: ‘newNode’ was not declared in this scope src/interval_tree.h:379: error: expected type-specifier src/interval_tree.h:379: error: expected `;' src/interval_tree.h: In member function ‘typename IntervalTree<T, N>::Node* IntervalTree<T, N>::GetSuccessorOf(IntervalTree<T, N>::Node*) const’: src/interval_tree.h:450: error: ‘y’ was not declared in this scope src/interval_tree.h: In member function ‘typename IntervalTree<T, N>::Node* IntervalTree<T, N>::GetPredecessorOf(IntervalTree<T, N>::Node*) const’: src/interval_tree.h:483: error: ‘y’ was not declared in this scope src/interval_tree.h: In destructor ‘IntervalTree<T, N>::~IntervalTree()’: src/interval_tree.h:546: error: ‘x’ was not declared in this scope src/interval_tree.h:547: error: type/value mismatch at argument 1 in template parameter list for ‘template<class _Tp, class _Alloc> class std::vector’ src/interval_tree.h:547: error: expected a type, got ‘(IntervalTree<T,N>::Node * <expression error>)’ src/interval_tree.h:547: error: template argument 2 is invalid src/interval_tree.h:547: error: invalid type in declaration before ‘;’ token src/interval_tree.h:551: error: request for member ‘push_back’ in ‘stuffToFree’, which is of non-class type ‘int’ src/interval_tree.h:554: error: request for member ‘push_back’ in ‘stuffToFree’, which is of non-class type ‘int’ src/interval_tree.h:557: error: request for member ‘empty’ in ‘stuffToFree’, which is of non-class type ‘int’ src/interval_tree.h:558: error: request for member ‘back’ in ‘stuffToFree’, which is of non-class type ‘int’ src/interval_tree.h:559: error: request for member ‘pop_back’ in ‘stuffToFree’, which is of non-class type ‘int’ src/interval_tree.h:561: error: request for member ‘push_back’ in ‘stuffToFree’, which is of non-class type ‘int’ src/interval_tree.h:564: error: request for member ‘push_back’ in ‘stuffToFree’, which is of non-class type ‘int’ src/interval_tree.h: In member function ‘void IntervalTree<T, N>::DeleteFixUp(IntervalTree<T, N>::Node*)’: src/interval_tree.h:613: error: ‘w’ was not declared in this scope src/interval_tree.h:614: error: ‘rootLeft’ was not declared in this scope src/interval_tree.h: In member function ‘T IntervalTree<T, N>::remove(IntervalTree<T, N>::Node*)’: src/interval_tree.h:697: error: ‘y’ was not declared in this scope src/interval_tree.h:698: error: ‘x’ was not declared in this scope src/interval_tree.h: In member function ‘std::vector<T, std::allocator<_CharT> > IntervalTree<T, N>::fetch(N, N)’: src/interval_tree.h:819: error: ‘x’ was not declared in this scope src/interval_tree.h:833: error: invalid types ‘int[size_t]’ for array subscript src/interval_tree.h:836: error: request for member ‘push_back’ in ‘((IntervalTree<T, N>*)this)->IntervalTree<T, N>::recursionNodeStack’, which is of non-class type ‘int’ src/interval_tree.h:837: error: request for member ‘back’ in ‘((IntervalTree<T, N>*)this)->IntervalTree<T, N>::recursionNodeStack’, which is of non-class type ‘int’ src/interval_tree.h:838: error: request for member ‘back’ in ‘((IntervalTree<T, N>*)this)->IntervalTree<T, N>::recursionNodeStack’, which is of non-class type ‘int’ src/interval_tree.h:839: error: request for member ‘back’ in ‘((IntervalTree<T, N>*)this)->IntervalTree<T, N>::recursionNodeStack’, which is of non-class type ‘int’ src/interval_tree.h:840: error: request for member ‘size’ in ‘((IntervalTree<T, N>*)this)->IntervalTree<T, N>::recursionNodeStack’, which is of non-class type ‘int’ src/interval_tree.h:846: error: request for member ‘size’ in ‘((IntervalTree<T, N>*)this)->IntervalTree<T, N>::recursionNodeStack’, which is of non-class type ‘int’ src/interval_tree.h:847: error: expected `;' before ‘back’ src/interval_tree.h:848: error: request for member ‘pop_back’ in ‘((IntervalTree<T, N>*)this)->IntervalTree<T, N>::recursionNodeStack’, which is of non-class type ‘int’ src/interval_tree.h:850: error: ‘back’ was not declared in this scope src/interval_tree.h:853: error: invalid types ‘int[size_t]’ for array subscript IntervalTree.c: In function ‘void boot_Set__IntervalTree(PerlInterpreter*, CV*)’: IntervalTree.c:365: warning: deprecated conversion from string constant to ‘char*’ src/interval_tree.h: In constructor ‘IntervalTree<T, N>::IntervalTree() [with T = std::tr1::shared_ptr<sv>, N = long int]’: IntervalTree.c:67: instantiated from here src/interval_tree.h:130: error: cannot convert ‘int*’ to ‘IntervalTree<std::tr1::shared_ptr<sv>, long int>::Node*’ in assignment src/interval_tree.h:135: error: cannot convert ‘int*’ to ‘IntervalTree<std::tr1::shared_ptr<sv>, long int>::Node*’ in assignment ...blah blah blah... ...blah blah blah... ...blah blah blah... ...blah blah blah... ...blah blah blah... ...blah blah blah... src/interval_tree.h:848: error: request for member ‘pop_back’ in ‘((IntervalTree<T, N>*)this)->IntervalTree<T, N>::recursionNodeStack’, which is of non-class type ‘int’ src/interval_tree.h:850: error: ‘back’ was not declared in this scope src/interval_tree.h:853: error: invalid types ‘int[size_t]’ for array subscript IntervalTree.c: In function ‘void boot_Set__IntervalTree(PerlInterpreter*, CV*)’: IntervalTree.c:365: warning: deprecated conversion from string constant to ‘char*’ src/interval_tree.h: In constructor ‘IntervalTree<T, N>::IntervalTree() [with T = std::tr1::shared_ptr<sv>, N = long int]’: IntervalTree.c:67: instantiated from here src/interval_tree.h:130: error: cannot convert ‘int*’ to ‘IntervalTree<std::tr1::shared_ptr<sv>, long int>::Node*’ in assignment src/interval_tree.h:135: error: cannot convert ‘int*’ to ‘IntervalTree<std::tr1::shared_ptr<sv>, long int>::Node*’ in assignment src/interval_tree.h: In member function ‘typename IntervalTree<T, N>::Node* IntervalTree<T, N>::insert(const T&, N, N) [with T = std::tr1::shared_ptr<sv>, N = long int]’: IntervalTree.xs:57: instantiated from here src/interval_tree.h:375: error: dependent-name ‘IntervalTree<T,N>::Node’ is parsed as a non-type, but instantiation yields a type src/interval_tree.h:375: note: say ‘typename IntervalTree<T,N>::Node’ if a type is meant src/interval_tree.h:376: error: dependent-name ‘IntervalTree<T,N>::Node’ is parsed as a non-type, but instantiation yields a type src/interval_tree.h:376: note: say ‘typename IntervalTree<T,N>::Node’ if a type is meant src/interval_tree.h:377: error: dependent-name ‘IntervalTree<T,N>::Node’ is parsed as a non-type, but instantiation yields a type src/interval_tree.h:377: note: say ‘typename IntervalTree<T,N>::Node’ if a type is meant src/interval_tree.h: In member function ‘std::vector<T, std::allocator<_CharT> > IntervalTree<T, N>::fetch(N, N) [with T = std::tr1::shared_ptr<sv>, N = long int]’: IntervalTree.xs:65: instantiated from here src/interval_tree.h:819: error: dependent-name ‘IntervalTree<T,N>::Node’ is parsed as a non-type, but instantiation yields a type src/interval_tree.h:819: note: say ‘typename IntervalTree<T,N>::Node’ if a type is meant IntervalTree.xs:65: instantiated from here src/interval_tree.h:847: error: dependent-name ‘IntervalTree<T,N>::it_recursion_node’ is parsed as a non-type, but instantiation yields a type src/interval_tree.h:847: note: say ‘typename IntervalTree<T,N>::it_recursion_node’ if a type is meant src/interval_tree.h: In destructor ‘IntervalTree<T, N>::~IntervalTree() [with T = std::tr1::shared_ptr<sv>, N = long int]’: IntervalTree.c:205: instantiated from here src/interval_tree.h:546: error: dependent-name ‘IntervalTree<T,N>::Node’ is parsed as a non-type, but instantiation yields a type src/interval_tree.h:546: note: say ‘typename IntervalTree<T,N>::Node’ if a type is meant src/interval_tree.h: In member function ‘void IntervalTree<T, N>::TreeInsertHelp(IntervalTree<T, N>::Node*) [with T = std::tr1::shared_ptr<sv>, N = long int]’: src/interval_tree.h:380: instantiated from ‘typename IntervalTree<T, N>::Node* IntervalTree<T, N>::insert(const T&, N, N) [with T = std::tr1::shared_ptr<sv>, N = long int]’ IntervalTree.xs:57: instantiated from here src/interval_tree.h:298: error: dependent-name ‘IntervalTree<T,N>::Node’ is parsed as a non-type, but instantiation yields a type src/interval_tree.h:298: note: say ‘typename IntervalTree<T,N>::Node’ if a type is meant src/interval_tree.h:299: error: dependent-name ‘IntervalTree<T,N>::Node’ is parsed as a non-type, but instantiation yields a type src/interval_tree.h:299: note: say ‘typename IntervalTree<T,N>::Node’ if a type is meant src/interval_tree.h: In member function ‘void IntervalTree<T, N>::LeftRotate(IntervalTree<T, N>::Node*) [with T = std::tr1::shared_ptr<sv>, N = long int]’: src/interval_tree.h:395: instantiated from ‘typename IntervalTree<T, N>::Node* IntervalTree<T, N>::insert(const T&, N, N) [with T = std::tr1::shared_ptr<sv>, N = long int]’ IntervalTree.xs:57: instantiated from here src/interval_tree.h:178: error: dependent-name ‘IntervalTree<T,N>::Node’ is parsed as a non-type, but instantiation yields a type src/interval_tree.h:178: note: say ‘typename IntervalTree<T,N>::Node’ if a type is meant src/interval_tree.h: In member function ‘void IntervalTree<T, N>::RightRotate(IntervalTree<T, N>::Node*) [with T = std::tr1::shared_ptr<sv>, N = long int]’: src/interval_tree.h:399: instantiated from ‘typename IntervalTree<T, N>::Node* IntervalTree<T, N>::insert(const T&, N, N) [with T = std::tr1::shared_ptr<sv>, N = long int]’ IntervalTree.xs:57: instantiated from here src/interval_tree.h:240: error: dependent-name ‘IntervalTree<T,N>::Node’ is parsed as a non-type, but instantiation yields a type src/interval_tree.h:240: note: say ‘typename IntervalTree<T,N>::Node’ if a type is meant lipo: can't open input file: /var/tmp//ccLthuaw.out (No such file or directory) make: *** [IntervalTree.o] Error 1 BENBOOTH/Set-IntervalTree-0.01.tar.gz make -- NOT OK Running make test Can't test without successful make Running make install Make had returned bad status, install seems impossible Failed during this command: BENBOOTH/Set-IntervalTree-0.01.tar.gz : make NO

Search Results

Search found 7128 results on 286 pages for 'httpcontext cache'.

Page 284/286 | < Previous Page | 280 281 282 283 284 285 286 | Next Page >

- by algro

- by Abram

- by lucian.jp

- by Kishore

- by user1761316

- by WarDoGG

- by FastAl

- by FastAl

- by user2330772

- by Jacob

- by kjsteuer

- by raindrop18

- by raindrop18

- by nando pandi

- by Prabhakar

- by Rick Strahl

- by svdoever

- by Shaun

- by Tiago

- by Rick Strahl

- by jon

- by Lenain

- by john.rose

- by Jani Rautiainen

- by Daniel Standage

< Previous Page | 280 281 282 283 284 285 286 | Next Page >