Search Results

Search found 8875 results on 355 pages for 'mime types'.

Page 347/355 | < Previous Page | 343 344 345 346 347 348 349 350 351 352 353 354  | Next Page >

  • linux thread synchronization

    - by johnnycrash
    I am new to linux and linux threads. I have spent some time googling to try to understand the differences between all the functions available for thread synchronization. I still have some questions. I have found all of these different types of synchronizations, each with a number of functions for locking, unlocking, testing the lock, etc. gcc atomic operations futexes mutexes spinlocks seqlocks rculocks conditions semaphores My current (but probably flawed) understanding is this: semaphores are process wide, involve the filesystem (virtually I assume), and are probably the slowest. Futexes might be the base locking mechanism used by mutexes, spinlocks, seqlocks, and rculocks. Futexes might be faster than the locking mechanisms that are based on them. Spinlocks dont block and thus avoid context swtiches. However they avoid the context switch at the expense of consuming all the cycles on a CPU until the lock is released (spinning). They should only should be used on multi processor systems for obvious reasons. Never sleep in a spinlock. The seq lock just tells you when you finished your work if a writer changed the data the work was based on. You have to go back and repeat the work in this case. Atomic operations are the fastest synch call, and probably are used in all the above locking mechanisms. You do not want to use atomic operations on all the fields in your shared data. You want to use a lock (mutex, futex, spin, seq, rcu) or a single atomic opertation on a lock flag when you are accessing multiple data fields. My questions go like this: Am I right so far with my assumptions? Does anyone know the cpu cycle cost of the various options? I am adding parallelism to the app so we can get better wall time response at the expense of running fewer app instances per box. Performances is the utmost consideration. I don't want to consume cpu with context switching, spinning, or lots of extra cpu cycles to read and write shared memory. I am absolutely concerned with number of cpu cycles consumed. Which (if any) of the locks prevent interruption of a thread by the scheduler or interrupt...or am I just an idiot and all synchonization mechanisms do this. What kinds of interruption are prevented? Can I block all threads or threads just on the locking thread's CPU? This question stems from my fear of interrupting a thread holding a lock for a very commonly used function. I expect that the scheduler might schedule any number of other workers who will likely run into this function and then block because it was locked. A lot of context switching would be wasted until the thread with the lock gets rescheduled and finishes. I can re-write this function to minimize lock time, but still it is so commonly called I would like to use a lock that prevents interruption...across all processors. I am writing user code...so I get software interrupts, not hardware ones...right? I should stay away from any functions (spin/seq locks) that have the word "irq" in them. Which locks are for writing kernel or driver code and which are meant for user mode? Does anyone think using an atomic operation to have multiple threads move through a linked list is nuts? I am thinking to atomicly change the current item pointer to the next item in the list. If the attempt works, then the thread can safely use the data the current item pointed to before it was moved. Other threads would now be moved along the list. futexes? Any reason to use them instead of mutexes? Is there a better way than using a condition to sleep a thread when there is no work? When using gcc atomic ops, specifically the test_and_set, can I get a performance increase by doing a non atomic test first and then using test_and_set to confirm? *I know this will be case specific, so here is the case. There is a large collection of work items, say thousands. Each work item has a flag that is initialized to 0. When a thread has exclusive access to the work item, the flag will be one. There will be lots of worker threads. Any time a thread is looking for work, they can non atomicly test for 1. If they read a 1, we know for certain that the work is unavailable. If they read a zero, they need to perform the atomic test_and_set to confirm. So if the atomic test_and_set is 500 cpu cycles because it is disabling pipelining, causes cpu's to communicate and L2 caches to flush/fill .... and a simple test is 1 cycle .... then as long as I had a better ratio of 500 to 1 when it came to stumbling upon already completed work items....this would be a win.* I hope to use mutexes or spinlocks to sparilngly protect sections of code that I want only one thread on the SYSTEM (not jsut the CPU) to access at a time. I hope to sparingly use gcc atomic ops to select work and minimize use of mutexes and spinlocks. For instance: a flag in a work item can be checked to see if a thread has worked it (0=no, 1=yes or in progress). A simple test_and_set tells the thread if it has work or needs to move on. I hope to use conditions to wake up threads when there is work. Thanks!

    Read the article

  • Reverse engineering windows mobile live search CellID location awareness protocol (yikes)...

    - by Jean-Charles
    I wasn't sure of how to form the question so I apologize if the title is misleading. Additionally, you may want to get some coffee and take a seat for this one ... It's long. Basically, I'm trying to reverse engineer the protocol used by the Windows Mobile Live Search application to get location based on cellID. Before I go on, I am aware of other open source services (such as OpenCellID) but this is more for the sake of education and a bit for redundancy. According to the packets I captured, a POST request is made to ... mobile.search.live.com/positionlookupservice_1/service.aspx ... with a few specific headers (agent, content-length, etc) and no body. Once this goes through, the server sends back a 100-Continue response. At this point, the application submits this data (I chopped off the packet header): 00 00 00 01 00 00 00 05 55 54 ........UT 46 2d 38 05 65 6e 2d 55 53 05 65 6e 2d 55 53 01 F-8.en-US.en-US. 06 44 65 76 69 63 65 05 64 75 6d 6d 79 01 06 02 .Device.dummy... 50 4c 08 0e 52 65 76 65 72 73 65 47 65 6f 63 6f PL..ReverseGeoco 64 65 01 07 0b 47 50 53 43 68 69 70 49 6e 66 6f de...GPSChipInfo 01 20 06 09 43 65 6c 6c 54 6f 77 65 72 06 03 43 . ..CellTower..C 47 49 08 03 4d 43 43 b6 02 07 03 4d 4e 43 03 34 GI..MCC....MNC.4 31 30 08 03 4c 41 43 cf 36 08 02 43 49 fd 01 00 10..LAC.6..CI... 00 00 00 ... And receives this in response (packet and HTTP response headers chopped): 00 00 00 01 00 00 00 00 01 06 02 50 4c ...........PL 06 08 4c 6f 63 61 6c 69 74 79 06 08 4c 6f 63 61 ..Locality..Loca 74 69 6f 6e 07 03 4c 61 74 09 34 32 2e 33 37 35 tion..Lat.42.375 36 32 31 07 04 4c 6f 6e 67 0a 2d 37 31 2e 31 35 621..Long.-71.15 38 39 33 38 00 07 06 52 61 64 69 75 73 09 32 30 8938...Radius.20 30 30 2e 30 30 30 30 00 42 07 0c 4c 6f 63 61 6c 00.0000.B..Local 69 74 79 4e 61 6d 65 09 57 61 74 65 72 74 6f 77 ityName.Watertow 6e 07 16 41 64 6d 69 6e 69 73 74 72 61 74 69 76 n..Administrativ 65 41 72 65 61 4e 61 6d 65 0d 4d 61 73 73 61 63 eAreaName.Massac 68 75 73 65 74 74 73 07 10 50 6f 73 74 61 6c 43 husetts..PostalC 6f 64 65 4e 75 6d 62 65 72 05 30 32 34 37 32 07 odeNumber.02472. 0b 43 6f 75 6e 74 72 79 4e 61 6d 65 0d 55 6e 69 .CountryName.Uni 74 65 64 20 53 74 61 74 65 73 00 00 00 ted States... Now, here is what I've determined so far: All strings are prepended with one byte that is the decimal equivalent of their length. There seem to be three different casts that are used throughout the request and response. They show up as one byte before the length byte. I've concluded that the three types map out as follows: 0x06 - parent element (subsequent values are children, closed with 0x00) 0x07 - string 0x08 - int? Based on these determinations, here is what the request and response look like in a more readable manner (values surrounded by brackets denote length and values surrounded by parenthesis denote a cast): \0x00\0x00\0x00\0x01\0x00\0x00\0x00 [5]UTF-8 [5]en-US [5]en-US \0x01 [6]Device [5]dummy \0x01 (6)[2]PL (8)[14]ReverseGeocode\0x01 (7)[11]GPSChipInfo[1]\0x20 (6)[9]CellTower (6)[3]CGI (8)[3]MCC\0xB6\0x02 //310 (7)[3]MNC[3]410 //410 (8)[3]LAC\0xCF\0x36 //6991 (8)[2]CI\0xFD\0x01 //259 \0x00 \0x00 \0x00 \0x00 and.. \0x00\0x00\0x00\0x01\0x00\0x00\0x00 \0x00\0x01 (6)[2]PL (6)[8]Locality (6)[8]Location (7)[3]Lat[9]42.375621 (7)[4]Long[10]-71.158938 \0x00 (7)[6]Radius[9]2000.0000 \0x00 \0x42 //"B" ... Has to do with GSM (7)[12]LocalityName[9]Watertown (7)[22]AdministrativeAreaName[13]Massachusetts (7)[16]PostalCodeNumber[5]02472 (7)[11]CountryName[13]United States \0x00 \0x00\0x00 My analysis seems to work out pretty well except for a few things: The 0x01s throughout confuse me ... At first I thought they were some sort of base level element terminators but I'm not certain. I'm not sure the 7-byte header is, in fact, a seven byte header. I wonder if it's maybe 4 bytes and that the three remaining 0x00s are of some other significance. The trailing 0x00s. Why is it that there is only one on the request but two on the response? The type 8 cast mentioned above ... I can't seem to figure out how those values are being encoded. I added comments to those lines with what the values should correspond to. Any advice on these four points will be greatly appreciated. And yes, these packets were captured in Watertown, MA. :)

    Read the article

  • C# Reading and Writing a Char[] to and from a Byte[] - Updated with Solution

    - by Simon G
    Hi, I have a byte array of around 10,000 bytes which is basically a blob from delphi that contains char, string, double and arrays of various types. This need to be read in and updated via C#. I've created a very basic reader that gets the byte array from the db and converts the bytes to the relevant object type when accessing the property which works fine. My problem is when I try to write to a specific char[] item, it doesn't seem to update the byte array. I've created the following extensions for reading and writing: public static class CharExtension { public static byte ToByte( this char c ) { return Convert.ToByte( c ); } public static byte ToByte( this char c, int position, byte[] blob ) { byte b = c.ToByte(); blob[position] = b; return b; } } public static class CharArrayExtension { public static byte[] ToByteArray( this char[] c ) { byte[] b = new byte[c.Length]; for ( int i = 1; i < c.Length; i++ ) { b[i] = c[i].ToByte(); } return b; } public static byte[] ToByteArray( this char[] c, int positon, int length, byte[] blob ) { byte[] b = c.ToByteArray(); Array.Copy( b, 0, blob, positon, length ); return b; } } public static class ByteExtension { public static char ToChar( this byte[] b, int position ) { return Convert.ToChar( b[position] ); } } public static class ByteArrayExtension { public static char[] ToCharArray( this byte[] b, int position, int length ) { char[] c = new char[length]; for ( int i = 0; i < length; i++ ) { c[i] = b.ToChar( position ); position += 1; } return c; } } to read and write chars and char arrays my code looks like: Byte[] _Blob; // set from a db field public char ubin { get { return _tariffBlob.ToChar( 14 ); } set { value.ToByte( 14, _Blob ); } } public char[] usercaplas { get { return _tariffBlob.ToCharArray( 2035, 10 ); } set { value.ToByteArray( 2035, 10, _Blob ); } } So to write to the objects I can do: ubin = 'C'; // this will update the byte[] usercaplas = new char[10] { 'A', 'B', etc. }; // this will update the byte[] usercaplas[3] = 'C'; // this does not update the byte[] I know the reason is that the setter property is not being called but I want to know is there a way around this using code similar to what I already have? I know a possible solution is to use a private variable called _usercaplas that I set and update as needed however as the byte array is nearly 10,000 bytes in length the class is already long and I would like a simpler approach as to reduce the overall code length and complexity. Thank Solution Here's my solution should anyone want it. If you have a better way of doing then let me know please. First I created a new class for the array: public class CharArrayList : ArrayList { char[] arr; private byte[] blob; private int length = 0; private int position = 0; public CharArrayList( byte[] blob, int position, int length ) { this.blob = blob; this.length = length; this.position = position; PopulateInternalArray(); SetArray(); } private void PopulateInternalArray() { arr = blob.ToCharArray( position, length ); } private void SetArray() { foreach ( char c in arr ) { this.Add( c ); } } private void UpdateInternalArray() { this.Clear(); SetArray(); } public char this[int i] { get { return arr[i]; } set { arr[i] = value; UpdateInternalArray(); } } } Then I created a couple of extension methods to help with converting to a byte[] public static byte[] ToByteArray( this CharArrayList c ) { byte[] b = new byte[c.Count]; for ( int i = 0; i < c.Count; i++ ) { b[i] = Convert.ToChar( c[i] ).ToByte(); } return b; } public static byte[] ToByteArray( this CharArrayList c, byte[] blob, int position, int length ) { byte[] b = c.ToByteArray(); Array.Copy( b, 0, blob, position, length ); return b; } So to read and write to the object: private CharArrayList _usercaplass; public CharArrayList usercaplas { get { if ( _usercaplass == null ) _usercaplass = new CharArrayList( _tariffBlob, 2035, 100 ); return _usercaplass; } set { _usercaplass = value; _usercaplass.ToByteArray( _tariffBlob, 2035, 100 ); } } As mentioned before its not an ideal solutions as I have to have private variables and extra code in the setter but I couldnt see a way around it.

    Read the article

  • How to create a SOAP REQUEST using ASP.NET (VB) without using Visual

    - by user311691
    Hi all , I urgently need your help . I am new to consuming a web service using SOAP protocol. I have been given a demo webservice URL which ends in .WSDL and NOT .asml?WSDL. The problem is I cannot add a web reference using Visual studio OR Disco.exe or Wsdl.exe - This webservice has been created on a java platform and for security reasons the only way to make a invoke the webservice is at runtime using SOAP protocol IN asp.net (VB). I I have created some code but cannot seem to send the soap object to the receiving web service. If I could get a solution with step by step instructions on how I can send a SOAP REQUEST. Below is my code and all am trying to do is send a SOAP REQUEST and receive a SOAP RESPONSE which I will display in my browser. <%@ page language="vb" %> <%@ Import Namespace="System.Data"%> <%@ Import Namespace="System.Xml"%> <%@ Import Namespace="System.Net"%> <%@ Import Namespace="System.IO"%> <%@ Import Namespace="System.Text"%> <script runat=server> Private Sub Page_Load() Dim objHTTPReq As HttpWebRequest Dim WebserviceUrl As String = "http://xx.xx.xx:8084/asy/wsdl/asy.wsdl" objHTTPReq = CType(WebRequest.Create(WebserviceUrl), HttpWebRequest) Dim soapXML As String soapXML = "<?xml version='1.0' encoding='utf-8'?>" & _ " <soap:Envelope xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'" & _ " xmlns:xsd='http://www.w3.org/2001/XMLSchema'"& _ " xmlns:soap='http://schemas.xmlsoap.org/soap/envelope/' >"& _ " <soap:Body> "& _ " <validatePaymentData xmlns='http://asybanks.webservices.asycuda.org'> " & _ " <bankCode>"& bankCode &"</bankCode> " & _ " <PaymentDataType>" & _ " <paymentType>"& payment_type &"</paymentType> " & _ " <amount>"& ass_amount &"</amount> " & _ " <ReferenceType>" & _ " <year>"& year &"</year> " & _ " <customsOfficeCode>"& station &"</customsOfficeCode> " & _ " </ReferenceType>" & _ " <accountNumber>"& zra_account &"</accountNumber> " & _ " </PaymentDataType> " & _ " </validatePaymentData> " & _ " </soap:Body> " & _ " </soap:Envelope> " objHTTPReq.Headers.Add("SOAPAction", "http://asybanks.webservices.asycuda.org") objHTTPReq.ContentType = "text/xml; charset=utf-8" objHTTPReq.ContentLength = soapXML.Length objHTTPReq.Accept = "text/xml" objHTTPReq.Method = "POST" Dim objHTTPRes As HttpWebResponse = CType(objHTTPReq.GetResponse(), HttpWebResponse) Dim dataStream As Stream = objHTTPRes.GetResponseStream() Dim reader As StreamReader = new StreamReader(dataStream) Dim responseFromServer As String = reader.ReadToEnd() OurXml.text = responseFromServer End Sub </script> <html xmlns="http://www.w3.org/1999/xhtml"> <head runat="server"> <title> XML TRANSACTION SIMULATION - N@W@ TJ </title> </head> <body> <form id="form1" runat="server"> <div> <p>ZRA test Feedback:</p> <asp:label id="OurXml" runat="server"/> </div> </form> </body> </html> the demo webservice looks like this: <?xml version="1.0" encoding="UTF-8" ?> - <!-- WEB SERVICE JAVA DEMO --> - <definitions targetNamespace="http://asybanks.webservices.asycuda.org" xmlns="http://schemas.xmlsoap.org/wsdl/" xmlns:apachesoap="http://xml.apache.org/xml-soap" xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:y="http://asybanks.webservices.asycuda.org"> - <types> - <xs:schema elementFormDefault="qualified" targetNamespace="http://asybanks.webservices.asycuda.org" xmlns="http://www.w3.org/2001/XMLSchema"> SOME OTHER INFORMATION AT THE BOTTOM <soap:address location="http://xx.xx.xx:8084/asy/services/asy" /> </port> </service> </definitions> From the above excerpt of the wsdl url webservice, I am not sure which namespace to use for soapACTION - please advise.... Please if you could comment every stage of a soap request and provide a working demo - I would be most grateful as I would be learning rather than just assuming stuff :)

    Read the article

  • Pointers to Derived Class Objects Losing vfptr

    - by duckworthd
    To begin, I am trying to write a run-of-the-mill, simple Ray Tracer. In my Ray Tracer, I have multiple types of geometries in the world, all derived from a base class called "SceneObject". I've included the header for it here. /** Interface for all objects that will appear in a scene */ class SceneObject { public: mat4 M, M_inv; Color c; SceneObject(); ~SceneObject(); /** The transformation matrix to be applied to all points of this object. Identity leaves the object in world frame. */ void setMatrix(mat4 M); void setMatrix(MatrixStack mStack); void getMatrix(mat4& M); /** The color of the object */ void setColor(Color c); void getColor(Color& c); /** Alter one portion of the color, leaving the rest as they were. */ void setDiffuse(vec3 rgb); void setSpecular(vec3 rgb); void setEmission(vec3 rgb); void setAmbient(vec3 rgb); void setShininess(double s); /** Fills 'inter' with information regarding an intersection between this object and 'ray'. Ray should be in world frame. */ virtual void intersect(Intersection& inter, Ray ray) = 0; /** Returns a copy of this SceneObject */ virtual SceneObject* clone() = 0; /** Print information regarding this SceneObject for debugging */ virtual void print() = 0; }; As you can see, I've included a couple virtual functions to be implemented elsewhere. In this case, I have only two derived class -- Sphere and Triangle, both of which implement the missing member functions. Finally, I have a Parser class, which is full of static methods that do the actual "Ray Tracing" part. Here's a couple snippets for relevant portions void Parser::trace(Camera cam, Scene scene, string outputFile, int maxDepth) { int width = cam.getNumXPixels(); int height = cam.getNumYPixels(); vector<vector<vec3>> colors; colors.clear(); for (int i = 0; i< width; i++) { vector<vec3> ys; for (int j = 0; j<height; j++) { Intersection intrsct; Ray ray; cam.getRay(ray, i, j); vec3 color; printf("Obtaining color for Ray[%d,%d]\n", i,j); getColor(color, scene, ray, maxDepth); ys.push_back(color); } colors.push_back(ys); } printImage(colors, width, height, outputFile); } void Parser::getColor(vec3& color, Scene scene, Ray ray, int numBounces) { Intersection inter; scene.intersect(inter,ray); if(inter.isIntersecting()){ Color c; inter.getColor(c); c.getAmbient(color); } else { color = vec3(0,0,0); } } Right now, I've forgone the true Ray Tracing part and instead simply return the color of the first object hit, if any. As you have no doubt noticed, the only way the computer knows that a ray has intersected an object is through Scene.intersect(), which I also include. void Scene::intersect(Intersection& i, Ray r) { Intersection result; result.setDistance(numeric_limits<double>::infinity()); result.setIsIntersecting(false); double oldDist; result.getDistance(oldDist); /* Cycle through all objects, making result the closest one */ for(int ind=0; ind<objects.size(); ind++){ SceneObject* thisObj = objects[ind]; Intersection betterIntersect; thisObj->intersect(betterIntersect, r); double newDist; betterIntersect.getDistance(newDist); if (newDist < oldDist){ result = betterIntersect; oldDist = newDist; } } i = result; } Alright, now for the problem. I begin by creating a scene and filling it with objects outside of the Parser::trace() method. Now for some odd reason, I cast Ray for i=j=0 and everything works wonderfully. However, by the time the second ray is cast all of the objects stored in my Scene no longer recognize their vfptr's! I stepped through the code with a debugger and found that the information to all the vfptr's are lost somewhere between the end of getColor() and the continuation of the loop. However, if I change the arguments of getColor() to use a Scene& instead of a Scene, then no loss occurs. What crazy voodoo is this?

    Read the article

  • Help required in adding new methods, properties into existing classes dynamically

    - by Bepenfriends
    Hi All, I am not sure whether it is possible to achieve this kind of implementation in Dot Net. Below are the information Currently we are on an application which is done in COM+, ASP, XSL, XML technologies. It is a multi tier architecture application in which COM+ acts as the BAL. The execution steps for any CRUD operation will be defined using a seperate UI which uses XML to store the information. BAL reads the XML and understands the execution steps which are defined and executes corresponding methods in DLL. Much like EDM we have our custom model (using XML) which determines which property of object is searchable, retrievable etc. Based on this information BAL constructs queries and calls procedures to get the data. In the current application both BAL and DAL are heavily customizable without doing any code change. the results will be transmitted to presentation layer in XML format which constructs the UI based on the data recieved. Now I am creating a migration project which deals with employee information. It is also going to follow the N Tier architecture in which the presentation layer communicates with BAL which connects to DAL to return the Data. Here is the problem, In our existing version we are handling every information as XML in its native form (no converstion of object etc), but in the migration project, Team is really interested in utilizing the OOP model of development where every information which is sent from BAL need to be converted to objects of its respective types (example employeeCollection, Address Collection etc). If we have the static number of data returned from BAL we can have a class which contains those nodes as properties and we can access the same. But in our case the data returned from our BAL need to be customized. How can we handle the customization in presentation layer which is converting the result to an Object. Below is an example of the XML returned <employees> <employee> <firstName>Employee 1 First Name</firstName> <lastName>Employee 1 Last Name</lastName> <addresses> <address> <addressType>1</addressType> <StreetName>Street name1</StreetName> <RegionName>Region name</RegionName> <address> <address> <addressType>2</addressType> <StreetName>Street name2</StreetName> <RegionName>Region name</RegionName> <address> <address> <addressType>3</addressType> <StreetName>Street name3</StreetName> <RegionName>Region name</RegionName> <address> <addresses> </employee> <employee> <firstName>Employee 2 First Name</firstName> <lastName>Employee 2 Last Name</lastName> <addresses> <address> <addressType>1</addressType> <StreetName>Street name1</StreetName> <RegionName>Region name</RegionName> <address> <address> <addressType>2</addressType> <StreetName>Street name2</StreetName> <RegionName>Region name</RegionName> <address> <addresses> </employee> </employees> If these are the only columns then i can write a class which is like public class Address{ public int AddressType {get;set;}; public string StreetName {get;set;}; public string RegionName {get;set;}; } public class Employee{ public string FirstName {get; set;} public string LastName {get; set;} public string AddressCollection {get; set;} } public class EmployeeCollection : List<Employee>{ public bool Add (Employee Data){ .... } } public class AddressCollection : List<Address>{ public bool Add (Address Data){ .... } } This class will be provided to customers and consultants as DLLs. We will not provide the source code for the same. Now when the consultants or customers does customization(example adding country to address and adding passport information object with employee object) they must be able to access those properties in these classes, but without source code they will not be able to do those modifications.which makes the application useless. Is there is any way to acomplish this in DotNet. I thought of using Anonymous classes but, the problem with Anonymous classes are we can not have methods in it. I am not sure how can i fit the collection objects (which will be inturn an anonymous class) Not sure about datagrid / user control binding etc. I also thought of using CODEDom to create classes runtime but not sure about the meory, performance issues. also the classes must be created only once and must use the same till there is another change. Kindly help me out in this problem. Any kind of help meterial/ cryptic code/ links will be helpful.

    Read the article

  • Parse a text file into multiple text file

    - by Vijay Kumar Singh
    I want to get multiple file by parsing a input file Through Java. The Input file contains many fasta format of thousands of protein sequence and I want to generate raw format(i.e., without any comma semicolon and without any extra symbol like "", "[", "]" etc) of each protein sequence. A fasta sequence starts form "" symbol followed by description of protein and then sequence of protein. For example ? lcl|NC_000001.10_cdsid_XP_003403591.1 [gene=LOC100652771] [protein=hypothetical protein LOC100652771] [protein_id=XP_003403591.1] [location=join(12190..12227,12595..12721,13403..13639)] MSESINFSHNLGQLLSPPRCVVMPGMPFPSIRSPELQKTTADLDHTLVSVPSVAESLHHPEITFLTAFCL PSFTRSRPLPDRQLHHCLALCPSFALPAGDGVCHGPGLQGSCYKGETQESVESRVLPGPRHRH Like above formate the input file contains 1000s of protein sequence. I have to generate thousands of raw file containing only individual protein sequence without any special symbol or gaps. I have developed the code for it in Java but out put is : Cannot open a file followed by cannot find file. Please help me to solve my problem. Regards Vijay Kumar Garg Varanasi Bharat (India) The code is /*Java code to convert FASTA format to a raw format*/ import java.io.*; import java.util.*; import java.util.regex.*; import java.io.FileInputStream; // java package for using regular expression public class Arrayren { public static void main(String args[]) throws IOException { String a[]=new String[1000]; String b[][] =new String[1000][1000]; /*open the id file*/ try { File f = new File ("input.txt"); //opening the text document containing genbank ids FileInputStream fis = new FileInputStream("input.txt"); //Reading the file contents through inputstream BufferedInputStream bis = new BufferedInputStream(fis); // Writing the contents to a buffered stream DataInputStream dis = new DataInputStream(bis); //Method for reading Java Standard data types String inputline; String line; String separator = System.getProperty("line.separator"); // reads a line till next line operator is found int i=0; while ((inputline=dis.readLine()) != null) { i++; a[i]=inputline; a[i]=a[i].replaceAll(separator,""); //replaces unwanted patterns like /n with space a[i]=a[i].trim(); // trims out if any space is available a[i]=a[i]+".txt"; //takes the file name into an array try // to handle run time error /*take the sequence in to an array*/ { BufferedReader in = new BufferedReader (new FileReader(a[i])); String inline = null; int j=0; while((inline=in.readLine()) != null) { j++; b[i][j]=inline; Pattern q=Pattern.compile(">"); //Compiling the regular expression Matcher n=q.matcher(inline); //creates the matcher for the above pattern if(n.find()) { /*appending the comment line*/ b[i][j]=b[i][j].replaceAll(">gi",""); //identify the pattern and replace it with a space b[i][j]=b[i][j].replaceAll("[a-zA-Z]",""); b[i][j]=b[i][j].replaceAll("|",""); b[i][j]=b[i][j].replaceAll("\\d{1,15}",""); b[i][j]=b[i][j].replaceAll(".",""); b[i][j]=b[i][j].replaceAll("_",""); b[i][j]=b[i][j].replaceAll("\\(",""); b[i][j]=b[i][j].replaceAll("\\)",""); } /*printing the sequence in to a text file*/ b[i][j]=b[i][j].replaceAll(separator,""); b[i][j]=b[i][j].trim(); // trims out if any space is available File create = new File(inputline+"R.txt"); try { if(!create.exists()) { create.createNewFile(); // creates a new file } else { System.out.println("file already exists"); } } catch(IOException e) // to catch the exception and print the error if cannot open a file { System.err.println("cannot create a file"); } BufferedWriter outt = new BufferedWriter(new FileWriter(inputline+"R.txt", true)); outt.write(b[i][j]); // printing the contents to a text file outt.close(); // closing the text file System.out.println(b[i][j]); } } catch(Exception e) { System.out.println("cannot open a file"); } } } catch(Exception ex) // catch the exception and prints the error if cannot find file { System.out.println("cannot find file "); } } } If you provide me correct it will be much easier to understand.

    Read the article

  • asp .net MVC 2.0 Validation

    - by ANDyW
    Hi I’m trying to do some validation in asp .net MVC 2.0 for my application. I want to have some nice client side validation. Validation should be done most time on model side with DataAnnotations with custom attributes( like CompareTo, StringLenght, MinPasswordLenght (from Membership.MinimumumpassworkdLenght value). For that purpose I tried to use xval with jquery.validation. Some specific thing is that most of forms will be working with ajax and most problems are when I want to validate form with ajax. Here is link for sample project http://www.sendspace.com/file/m9gl54 . I got two forms as controls ValidFormControl1.ascx, ValidFormControl2.ascx <% using (Ajax.BeginForm("CreateValidForm", "Test", new AjaxOptions { HttpMethod = "Post" })) {%> <div id="validationSummary1"> <%= Html.ValidationSummary(true)%> </div> <fieldset> <legend>Fields</legend> <div class="editor-label"> <%= Html.LabelFor(model => model.Name)%> </div> <div class="editor-field"> <%= Html.TextBoxFor(model => model.Name)%> <%= Html.ValidationMessageFor(model => model.Name)%> </div> <div class="editor-label"> <%= Html.LabelFor(model => model.Email)%> </div> <div class="editor-field"> <%= Html.TextBoxFor(model => model.Email)%> <%= Html.ValidationMessageFor(model => model.Email)%> </div> <div class="editor-label"> <%= Html.LabelFor(model => model.Password)%> </div> <div class="editor-field"> <%= Html.TextBoxFor(model => model.Password)%> <%= Html.ValidationMessageFor(model => model.Password)%> </div> <div class="editor-label"> <%= Html.LabelFor(model => model.ConfirmPassword)%> </div> <div class="editor-field"> <%= Html.TextBoxFor(model => model.ConfirmPassword)%> <%= Html.ValidationMessageFor(model => model.ConfirmPassword)%> </div> <p> <input type="submit" value="Create" /> </p> </fieldset> <% } %> <%= Html.ClientSideValidation<ValidModel>() .UseValidationSummary("validationSummary1", "Please fix the following problems:") %> Both look same the difference is only validation summaryID (validationSummary1, validationSummary2). Both controls are rendered on one page : Form2 <%Html.RenderPartial("~/Views/Test/ValidFormControl2.ascx", null); %> Form1 <%Html.RenderPartial("~/Views/Test/ValidFormControl.ascx", null); %> Validation property First problem, when we have two controls with same type to validate it don’t work becosue html elements are rendered by field name ( so we have two element with same name “Password” ). Only first form will be validated by client side. The worst thing is that even if we have different types and their fields name is same validation won’t work too ( this thing is what I need to repair it will be stupid to name some unique properites for validation ). Is there any solution for this ? Custom attributes validation Next thing custom attributes validation ( All those error are when I use Ajax for on normal form validation is working without problem. ): CompareTo - Simple compare to that is done in mvc template for account model ( class attribute saying with two property will be compared ) , and it wasn’t show on page. To do it I created own CachingRulesProvider with compareRule and my Attribute. Maybe there is more easy way to do it? StringLenght with minimum and maximum value, I won’t describe how I done it but is there any easy whey to do it? Validation summary When I have two two control on page all summary validation information goes to first control validation summary element, even xval generated script say that elementID are different for summary. Any one know how to repair it? Validation Information Is there any option to turn on messages on place where is Html.ValidationMessageFor(model = model.ConfirmPassword). Becsoue for me it isn’t show up. I would like to have summary and near field information too not only red border. Any one know how to do it? Ajax submit Anyone know how to do easy without massive code in javascript to do submit via javascript. This will be used to change input submit to href element (a). Both look same the difference is only validation summaryID

    Read the article

  • Condition Variable in Shared Memory - is this code POSIX-conformant?

    - by GrahamS
    We've been trying to use a mutex and condition variable to synchronise access to named shared memory on a LynuxWorks LynxOS-SE system (POSIX-conformant). One shared memory block is called "/sync" and contains the mutex and condition variable, the other is "/data" and contains the actual data we are syncing access to. We're seeing failures from pthread_cond_signal() if both processes don't perform the mmap() calls in exactly the same order, or if one process mmaps in some other piece of shared memory before it mmaps the sync memory. This example code is about as short as I can make it: #include <sys/types.h> #include <sys/stat.h> #include <sys/mman.h> #include <sys/file.h> #include <stdlib.h> #include <pthread.h> #include <errno.h> #include <iostream> #include <string> using namespace std; static const string shm_name_sync("/sync"); static const string shm_name_data("/data"); struct shared_memory_sync { pthread_mutex_t mutex; pthread_cond_t condition; }; struct shared_memory_data { int a; int b; }; //Create 2 shared memory objects // - sync contains 2 shared synchronisation objects (mutex and condition) // - data not important void create() { // Create and map 'sync' shared memory int fd_sync = shm_open(shm_name_sync.c_str(), O_CREAT|O_RDWR, S_IRUSR|S_IWUSR); ftruncate(fd_sync, sizeof(shared_memory_sync)); void* addr_sync = mmap(0, sizeof(shared_memory_sync), PROT_READ|PROT_WRITE, MAP_SHARED, fd_sync, 0); shared_memory_sync* p_sync = static_cast<shared_memory_sync*> (addr_sync); // init the cond and mutex pthread_condattr_t cond_attr; pthread_condattr_init(&cond_attr); pthread_condattr_setpshared(&cond_attr, PTHREAD_PROCESS_SHARED); pthread_cond_init(&(p_sync->condition), &cond_attr); pthread_condattr_destroy(&cond_attr); pthread_mutexattr_t m_attr; pthread_mutexattr_init(&m_attr); pthread_mutexattr_setpshared(&m_attr, PTHREAD_PROCESS_SHARED); pthread_mutex_init(&(p_sync->mutex), &m_attr); pthread_mutexattr_destroy(&m_attr); // Create the 'data' shared memory int fd_data = shm_open(shm_name_data.c_str(), O_CREAT|O_RDWR, S_IRUSR|S_IWUSR); ftruncate(fd_data, sizeof(shared_memory_data)); void* addr_data = mmap(0, sizeof(shared_memory_data), PROT_READ|PROT_WRITE, MAP_SHARED, fd_data, 0); shared_memory_data* p_data = static_cast<shared_memory_data*> (addr_data); // Run the second process while it sleeps here. sleep(10); int res = pthread_cond_signal(&(p_sync->condition)); assert(res==0); // <--- !!!THIS ASSERT WILL FAIL ON LYNXOS!!! munmap(addr_sync, sizeof(shared_memory_sync)); shm_unlink(shm_name_sync.c_str()); munmap(addr_data, sizeof(shared_memory_data)); shm_unlink(shm_name_data.c_str()); } //Open the same 2 shared memory objects but in reverse order // - data // - sync void open() { sleep(2); int fd_data = shm_open(shm_name_data.c_str(), O_RDWR, S_IRUSR|S_IWUSR); void* addr_data = mmap(0, sizeof(shared_memory_data), PROT_READ|PROT_WRITE, MAP_SHARED, fd_data, 0); shared_memory_data* p_data = static_cast<shared_memory_data*> (addr_data); int fd_sync = shm_open(shm_name_sync.c_str(), O_RDWR, S_IRUSR|S_IWUSR); void* addr_sync = mmap(0, sizeof(shared_memory_sync), PROT_READ|PROT_WRITE, MAP_SHARED, fd_sync, 0); shared_memory_sync* p_sync = static_cast<shared_memory_sync*> (addr_sync); // Wait on the condvar pthread_mutex_lock(&(p_sync->mutex)); pthread_cond_wait(&(p_sync->condition), &(p_sync->mutex)); pthread_mutex_unlock(&(p_sync->mutex)); munmap(addr_sync, sizeof(shared_memory_sync)); munmap(addr_data, sizeof(shared_memory_data)); } int main(int argc, char** argv) { if(argc>1) { open(); } else { create(); } return (0); } Run this program with no args, then another copy with args, and the first one will fail at the assert checking the pthread_cond_signal(). But change the open() function to mmap() the "/sync" memory first and it will all work fine. This seems like a major bug in LynxOS but LynuxWorks claim that using mutex and condition variable in this way is not covered by the POSIX standard, so they are not interested. Can anyone determine if this code does violate POSIX? Or does anyone have any convincing documentation that it is POSIX compliant?

    Read the article

  • System.Timers.Timer leaking due to "direct delegate roots"

    - by alimbada
    Apologies for the rather verbose and long-winded post, but this problem's been perplexing me for a few weeks now so I'm posting as much information as I can in order to get this resolved quickly. We have a WPF UserControl which is being loaded by a 3rd party app. The 3rd party app is a presentation application which loads and unloads controls on a schedule defined by an XML file which is downloaded from a server. Our control, when it is loaded into the application makes a web request to a web service and uses the data from the response to display some information. We're using an MVVM architecture for the control. The entry point of the control is a method that is implementing an interface exposed by the main app and this is where the control's configuration is set up. This is also where I set the DataContext of our control to our MainViewModel. The MainViewModel has two other view models as properties and the main UserControl has two child controls. Depending on the data received from the web service, the main UserControl decides which child control to display, e.g. if there is a HTTP error or the data received is not valid, then display child control A, otherwise display child control B. As you'd expect, these two child controls bind two separate view models each of which is a property of MainViewModel. Now child control B (which is displayed when the data is valid) has a RefreshService property/field. RefreshService is an object that is responsible for updating the model in a number of ways and contains 4 System.Timers.Timers; a _modelRefreshTimer a _viewRefreshTimer a _pageSwitchTimer a _retryFeedRetrievalOnErrorTimer (this is only enabled when something goes wrong with retrieving data). I should mention at this point that there are two types of data; the first changes every minute, the second changes every few hours. The controls' configuration decides which type we are using/displaying. If data is of the first type then we update the model quite frequently (every 30 seconds) using the _modelRefreshTimer's events. If the data is of the second type then we update the model after a longer interval. However, the view still needs to be refreshed every 30 seconds as stale data needs to be removed from the view (hence the _viewRefreshTimer). The control also paginates the data so we can see more than we can fit on the display area. This works by breaking the data up into Lists and switching the CurrentPage (which is a List) property of the view model to the right List. This is done by handling the _pageSwitchTimer's Elapsed event. Now the problem My problem is that the control, when removed from the visual tree doesn't dispose of it's timers. This was first noticed when we started getting an unusually high number of requests on the web server end very soon after deploying this control and found that requests were being made at least once a second! We found that the timers were living on and not stopping hours after the control had been removed from view and that the more timers there were the more requests piled up at the web server. My first solution was to implement IDisposable for the RefreshService and do some clean up when the control's UnLoaded event was fired. Within the RefreshServices Dispose method I've set Enabled to false for all the timers, then used the Stop() method on all of them. I've then called Dispose() too and set them to null. None of this worked. After some reading around I found that event handlers may hold references to Timers and prevent them from being disposed and collected. After some more reading and researching I found that the best way around this was to use the Weak Event Pattern. Using this blog and this blog I've managed to work around the shortcomings in the Weak Event pattern. However, none of this solves the problem. Timers are still not being disabled or stopped (let alone disposed) and web requests are continuing to build up. Mem Profiler tells me that "This type has N instances that are directly rooted by a delegate. This can indicate the delegate has not been properly removed" (where N is the number of instances). As far as I can tell though, all listeners of the Elapsed event for the timers are being removed during the cleanup so I can't understand why the timers continue to run. Thanks for reading. Eagerly awaiting your suggestions/comments/solutions (if you got this far :-p)

    Read the article

  • C++0x rvalue references - lvalues-rvalue binding

    - by Doug
    This is a follow-on question to http://stackoverflow.com/questions/2748866/c0x-rvalue-references-and-temporaries In the previous question, I asked how this code should work: void f(const std::string &); //less efficient void f(std::string &&); //more efficient void g(const char * arg) { f(arg); } It seems that the move overload should probably be called because of the implicit temporary, and this happens in GCC but not MSVC (or the EDG front-end used in MSVC's Intellisense). What about this code? void f(std::string &&); //NB: No const string & overload supplied void g1(const char * arg) { f(arg); } void g2(const std::string & arg) { f(arg); } It seems that, based on the answers to my previous question that function g1 is legal (and is accepted by GCC 4.3-4.5, but not by MSVC). However, GCC and MSVC both reject g2 because of clause 13.3.3.1.4/3, which prohibits lvalues from binding to rvalue ref arguments. I understand the rationale behind this - it is explained in N2831 "Fixing a safety problem with rvalue references". I also think that GCC is probably implementing this clause as intended by the authors of that paper, because the original patch to GCC was written by one of the authors (Doug Gregor). However, I don't this is quite intuitive. To me, (a) a const string & is conceptually closer to a string && than a const char *, and (b) the compiler could create a temporary string in g2, as if it were written like this: void g2(const std::string & arg) { f(std::string(arg)); } Indeed, sometimes the copy constructor is considered to be an implicit conversion operator. Syntactically, this is suggested by the form of a copy constructor, and the standard even mentions this specifically in clause 13.3.3.1.2/4, where the copy constructor for derived-base conversions is given a higher conversion rank than other implicit conversions: A conversion of an expression of class type to the same class type is given Exact Match rank, and a conversion of an expression of class type to a base class of that type is given Conversion rank, in spite of the fact that a copy/move constructor (i.e., a user-defined conversion function) is called for those cases. (I assume this is used when passing a derived class to a function like void h(Base), which takes a base class by value.) Motivation My motivation for asking this is something like the question asked in http://stackoverflow.com/questions/2696156/how-to-reduce-redundant-code-when-adding-new-c0x-rvalue-reference-operator-over ("How to reduce redundant code when adding new c++0x rvalue reference operator overloads"). If you have a function that accepts a number of potentially-moveable arguments, and would move them if it can (e.g. a factory function/constructor: Object create_object(string, vector<string>, string) or the like), and want to move or copy each argument as appropriate, you quickly start writing a lot of code. If the argument types are movable, then one could just write one version that accepts the arguments by value, as above. But if the arguments are (legacy) non-movable-but-swappable classes a la C++03, and you can't change them, then writing rvalue reference overloads is more efficient. So if lvalues did bind to rvalues via an implicit copy, then you could write just one overload like create_object(legacy_string &&, legacy_vector<legacy_string> &&, legacy_string &&) and it would more or less work like providing all the combinations of rvalue/lvalue reference overloads - actual arguments that were lvalues would get copied and then bound to the arguments, actual arguments that were rvalues would get directly bound. Questions My questions are then: Is this a valid interpretation of the standard? It seems that it's not the conventional or intended one, at any rate. Does it make intuitive sense? Is there a problem with this idea that I"m not seeing? It seems like you could get copies being quietly created when that's not exactly expected, but that's the status quo in places in C++03 anyway. Also, it would make some overloads viable when they're currently not, but I don't see it being a problem in practice. Is this a significant enough improvement that it would be worth making e.g. an experimental patch for GCC?

    Read the article

  • xs:choice unbounded list

    - by Matt
    I want to define an XSD schema for an XML document, example below: <?xml version="1.0" encoding="utf-8"?> <view xmlns="http://localhost/model_data" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://localhost/model_data XMLSchemaView.xsd" path="wibble" id="wibble"> <text name="PageTitle">Homepage</text> <text name="Keywords">home foo bar</text> <image name="MainImage"> <description>lolem ipsum</description> <title>i haz it</title> <url>/images/main-image.jpg</url> <type>image/jpeg</type> <alt>alt text for image</alt> <width>400</width> <height>300</height> </image> <link name="TermsAndConditionsLink"> <url>/tnc.html</url> <title>Terms and Conditions</title> <target>_blank</target> </link> </view> There's a view root element and then an unknown number of field elements (of various types). I'm using the following XSD schema: <?xml version="1.0" encoding="utf-8"?> <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns="http://localhost/model_data" targetNamespace="http://localhost/model_data" id="XMLSchema1"> <xs:element name="text" type="text_field"/> <xs:element name="view" type="model_data"/> <xs:complexType name="model_data"> <xs:choice maxOccurs="unbounded"> <xs:element name="text" type="text_field"/> <xs:element name="image" type="image_field"/> <xs:element name="link" type="link_field"/> </xs:choice> <xs:attribute name="path" type="xs:string"/> <xs:attribute name="id" type="xs:string"/> </xs:complexType> <xs:complexType name="image_field"> <xs:all> <xs:element name="description" type="xs:string"/> <xs:element name="title" type="xs:string"/> <xs:element name="type" type="xs:string"/> <xs:element name="url" type="xs:string"/> <xs:element name="alt" type="xs:string"/> <xs:element name="height" type="xs:int"/> <xs:element name="width" type="xs:int"/> </xs:all> <xs:attribute name="name" type="xs:string"/> </xs:complexType> <xs:complexType name="text_field"> <xs:simpleContent> <xs:extension base="xs:string"> <xs:attribute name="name" type="xs:string"/> </xs:extension> </xs:simpleContent> </xs:complexType> <xs:complexType name="link_field"> <xs:all> <xs:element name="target" type="xs:string"/> <xs:element name="title" type="xs:string"/> <xs:element name="url" type="xs:string"/> </xs:all> <xs:attribute name="name" type="xs:string"/> </xs:complexType> </xs:schema> This looks like it should work to me, but it doesn't and I always get the following error: Element <text> is not allowed under element <view>. Reason: The following elements are expected at this location (see below) <text> <image> <link> Error location: view / text Details cvc-model-group: Element <text> unexpected by type 'model_data' of element <view>. cvc-elt.5.2.1: The element <view> is not valid with respect to the actual type definition 'model_data'. I've never really used XSD schemas before, so I'd really appreciate it if someone could point out where I'm going wrong.

    Read the article

  • Policy based design and defaults.

    - by Noah Roberts
    Hard to come up with a good title for this question. What I really need is to be able to provide template parameters with different number of arguments in place of a single parameter. Doesn't make a lot of sense so I'll go over the reason: template < typename T, template <typename,typename> class Policy = default_policy > struct policy_based : Policy<T, policy_based<T,Policy> > { // inherits R Policy::fun(arg0, arg1, arg2,...,argn) }; // normal use: policy_base<type_a> instance; // abnormal use: template < typename PolicyBased > // No T since T is always the same when you use this struct custom_policy {}; policy_base<type_b,custom_policy> instance; The deal is that for many abnormal uses the Policy will be based on one single type T, and can't really be parameterized on T so it makes no sense to take T as a parameter. For other uses, including the default, a Policy can make sense with any T. I have a couple ideas but none of them are really favorites. I thought that I had a better answer--using composition instead of policies--but then I realized I have this case where fun() actually needs extra information that the class itself won't have. This is like the third time I've refactored this silly construct and I've got quite a few custom versions of it around that I'm trying to consolidate. I'd like to get something nailed down this time rather than just fish around and hope it works this time. So I'm just fishing for ideas right now hoping that someone has something I'll be so impressed by that I'll switch deities. Anyone have a good idea? Edit: You might be asking yourself why I don't just retrieve T from the definition of policy based in the template for default_policy. The reason is that default_policy is actually specialized for some types T. Since asking the question I have come up with something that may be what I need, which will follow, but I could still use some other ideas. template < typename T > struct default_policy; template < typename T, template < typename > class Policy = default_policy > struct test : Policy<test<T,Policy>> {}; template < typename T > struct default_policy< test<T, default_policy> > { void f() {} }; template < > struct default_policy< test<int, default_policy> > { void f(int) {} }; Edit: Still messing with it. I wasn't too fond of the above since it makes default_policy permanently coupled with "test" and so couldn't be reused in some other method, such as with multiple templates as suggested below. It also doesn't scale at all and requires a list of parameters at least as long as "test" has. Tried a few different approaches that failed until I found another that seems to work so far: template < typename T > struct default_policy; template < typename T, template < typename > class Policy = default_policy > struct test : Policy<test<T,Policy>> {}; template < typename PolicyBased > struct fetch_t; template < typename PolicyBased, typename T > struct default_policy_base; template < typename PolicyBased > struct default_policy : default_policy_base<PolicyBased, typename fetch_t<PolicyBased>::type> {}; template < typename T, template < typename > class Policy > struct fetch_t< test<T,Policy> > { typedef T type; }; template < typename PolicyBased, typename T > struct default_policy_base { void f() {} }; template < typename PolicyBased > struct default_policy_base<PolicyBased,int> { void f(int) {} };

    Read the article

  • Find inner arrays in nested arrays

    - by 50ndr33
    I have a nested array in PHP: array ( '0' => "+5x", '1' => array ( '0' => "+", '1' => "(", '2' => "+3", '3' => array ( '0' => "+", '1' => "(", '2' => array ( // I want to find this one. '0' => "+", '1' => "(", '2' => "+5", '3' => "-3", '4' => ")" ), '3' => "-3", '4' => ")" ), '4' => ")" ) ); I need to process the innermost arrays here, the one with the comment: "I want to find this one." Is there a function for that? I have thought about doing (written as an idea, not as correct PHP): foreach ($array as $id => $value) { if ($value is array) { $name = $id; foreach ($array[$id] as $id_2 => $value_2) { if ($value_2 is array) { $name .= "." . $id_2; foreach ($array[$id][$id_2] as $id_3 => $value_3) { if ($value_3 is array) { $name .= "." . $id_3; foreach ($array[$id][$id_2][$id_3] as $id_4 => $value_4) { if ($value_4 is array) { $name .= "." . $id_4; foreach [and so it goes on]; } else { $listOfInnerArrays[] = $name; break; } } } else { $listOfInnerArrays[] = $name; break; } } } else { $listOfInnerArrays[] = $name; break; } } } } So what it does is it makes $name the current key in the array. If the value is an array, it goes into it with foreach and adds "." and the id of the array. So we would in the example array end up with: array ( '0' => "1.3.2", ) Then I can process those values to access the innner arrays. The problem is that the array that I'm trying to find the inner arrays of is dynamic and made of a user input. (It splits an input string where it finds + or -, and puts it in a separate nested array if it contains brackets. So if the user types a lot of brackets, there will be a lot of nested arrays.) Therefore I need to make this pattern go for 20 times down, and still it will only catch 20 nested arrays no matter what. Is there a function for that, again? Or is there a way to make it do this without my long code? Maybe make a loop make the necessary number of the foreach pattern and run it through eval()? Long answer to J. Bruni: <?php $liste = array ( '0' => "+5x", '1' => array ( '0' => "+", '1' => "(", '2' => "+3", '3' => array ( '0' => "+", '1' => "(", '2' => array ( '0' => "+", '1' => "(", '2' => "+5", '3' => "-3", '4' => ")" ), '3' => "-3", '4' => ")" ), '4' => ")" ) ); function find_deepest( $item, $key ) { echo "0"; if ( !is_array( $item ) ) return false; foreach( $item as $sub_item ) { if ( is_array( $sub_item ) ) return false; } echo "1"; print_r( $item ); return true; } array_walk_recursive( $liste, 'find_deepest' ); echo "<pre>"; print_r($liste); ?> I wrote echo 0 and 1 to see what the script did, and here is the output: 00000000000000 Array ( [0] => +5x [1] => Array ( [0] => + [1] => ( [2] => +3 [3] => Array ( [0] => + [1] => ( [2] => Array ( [0] => + [1] => ( [2] => +5 [3] => -3 [4] => ) ) [3] => -3 [4] => ) ) [4] => ) ) )

    Read the article

  • Linux C: "Interactive session" with separate read and write named pipes?

    - by ~sd-imi
    Hi all, I am trying to work with "Introduction to Interprocess Communication Using Named Pipes - Full-Duplex Communication Using Named Pipes", http://developers.sun.com/solaris/articles/named_pipes.html#5 ; in particular fd_server.c (included below for reference) Here is my info and compile line: :~$ cat /etc/issue Ubuntu 10.04 LTS \n \l :~$ gcc --version gcc (Ubuntu 4.4.3-4ubuntu5) 4.4.3 :~$ gcc fd_server.c -o fd_server fd_server.c creates two named pipes, one for reading and one for writing. What one can do, is: in one terminal, run the server and read (through cat) its write pipe: :~$ ./fd_server & 2/dev/null [1] 11354 :~$ cat /tmp/np2 and in another, write (using echo) to server's read pipe: :~$ echo "heeellloooo" /tmp/np1 going back to first terminal, one can see: :~$ cat /tmp/np2 HEEELLLOOOO 0[1]+ Exit 13 ./fd_server 2 /dev/null What I would like to do, is make sort of a "interactive" (or "shell"-like) session; that is, the server is run as usual, but instead of running "cat" and "echo", I'd like to use something akin to screen. What I mean by that, is that screen can be called like screen /dev/ttyS0 38400, and then it makes a sort of a interactive session, where what is typed in terminal is passed to /dev/ttyS0, and its response is written to terminal. Now, of course, I cannot use screen, because in my case the program has two separate nodes, and as far as I can tell, screen can refer to only one. How would one go about to achieve this sort of "interactive" session in this context (with two separate read/write pipes)? Thanks, Cheers! Code below: #include <stdio.h> #include <errno.h> #include <ctype.h> #include <sys/types.h> #include <sys/stat.h> #include <fcntl.h> //#include <fullduplex.h> /* For name of the named-pipe */ #define NP1 "/tmp/np1" #define NP2 "/tmp/np2" #define MAX_BUF_SIZE 255 #include <stdlib.h> //exit #include <string.h> //strlen int main(int argc, char *argv[]) { int rdfd, wrfd, ret_val, count, numread; char buf[MAX_BUF_SIZE]; /* Create the first named - pipe */ ret_val = mkfifo(NP1, 0666); if ((ret_val == -1) && (errno != EEXIST)) { perror("Error creating the named pipe"); exit (1); } ret_val = mkfifo(NP2, 0666); if ((ret_val == -1) && (errno != EEXIST)) { perror("Error creating the named pipe"); exit (1); } /* Open the first named pipe for reading */ rdfd = open(NP1, O_RDONLY); /* Open the second named pipe for writing */ wrfd = open(NP2, O_WRONLY); /* Read from the first pipe */ numread = read(rdfd, buf, MAX_BUF_SIZE); buf[numread] = '0'; fprintf(stderr, "Full Duplex Server : Read From the pipe : %sn", buf); /* Convert to the string to upper case */ count = 0; while (count < numread) { buf[count] = toupper(buf[count]); count++; } /* * Write the converted string back to the second * pipe */ write(wrfd, buf, strlen(buf)); } Edit: Right, just to clarify - it seems I found a document discussing something very similar, it is http://en.wikibooks.org/wiki/Serial_Programming/Serial_Linux#Configuration_with_stty - a modification of the script there ("For example, the following script configures the device and starts a background process for copying all received data from the serial device to standard output...") for the above program is below: # stty raw # ( ./fd_server 2>/dev/null; )& bgPidS=$! ( cat < /tmp/np2 ; )& bgPid=$! # Read commands from user, send them to device echo $(kill -0 $bgPidS 2>/dev/null ; echo $?) while [ "$(kill -0 $bgPidS 2>/dev/null ; echo $?)" -eq "0" ] && read cmd; do # redirect debug msgs to stderr, as here we're redirected to /tmp/np1 echo "$? - $bgPidS - $bgPid" >&2 echo "$cmd" echo -e "\nproc: $(kill -0 $bgPidS 2>/dev/null ; echo $?)" >&2 done >/tmp/np1 echo OUT # Terminate background read process - if they still exist if [ "$(kill -0 $bgPid 2>/dev/null ; echo $?)" -eq "0" ] ; then kill $bgPid fi if [ "$(kill -0 $bgPidS 2>/dev/null ; echo $?)" -eq "0" ] ; then kill $bgPidS fi # stty cooked So, saving the script as say starter.sh and calling it, results with the following session: $ ./starter.sh 0 i'm typing here and pressing [enter] at end 0 - 13496 - 13497 I'M TYPING HERE AND PRESSING [ENTER] AT END 0~?.N=?(?~? ?????}????@??????~? [garble] proc: 0 OUT which is what I'd call for "interactive session" (ignoring the debug statements) - server waits for me to enter a command; it gives its output after it receives a command (and as in this case it exits after first command, so does the starter script as well). Except that, I'd like to not have buffered input, but sent character by character (meaning the above session should exit after first key press, and print out a single letter only - which is what I expected stty raw would help with, but it doesn't: it just kills reaction to both Enter and Ctrl-C :) ) I was just wandering if there already is an existing command (akin to screen in respect to serial devices, I guess) that would accept two such named pipes as arguments, and establish a "terminal" or "shell" like session through them; or would I have to use scripts as above and/or program own 'client' that will behave as a terminal..

    Read the article

  • Stacked up with web service configuration

    - by Allan Chua
    I'm currently stacked with the web service that im creating right now. when Testing it in local it all works fine but when I try to deploy it to the web server it throws me the following error An error occurred while trying to make a request to URI '...my web service URI here....'. This could be due to attempting to access a service in a cross-domain way without a proper cross-domain policy in place, or a policy that is unsuitable for SOAP services. You may need to contact the owner of the service to publish a cross-domain policy file and to ensure it allows SOAP-related HTTP headers to be sent. This error may also be caused by using internal types in the web service proxy without using the InternalsVisibleToAttribute attribute. Please see the inner exception for more details. here is my web config. <?xml version="1.0"?> <configuration> <configSections> </configSections> <system.webServer> <modules runAllManagedModulesForAllRequests="true"> </modules> <validation validateIntegratedModeConfiguration="false" /> <security> <requestFiltering> <requestLimits maxAllowedContentLength="2000000000" /> </requestFiltering> </security> </system.webServer> <connectionStrings> <add name="........" providerName="System.Data.SqlClient" /> </connectionStrings> <appSettings> <!-- Testing --> <add key="DataConnectionString" value="..........." /> </appSettings> <system.web> <compilation debug="true" targetFramework="4.0"> <buildProviders> <add extension=".rdlc" type="Microsoft.Reporting.RdlBuildProvider, Microsoft.ReportViewer.WebForms, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" /> </buildProviders> </compilation> <httpRuntime executionTimeout="1200" maxRequestLength="2000000" /> </system.web> <system.serviceModel> <serviceHostingEnvironment aspNetCompatibilityEnabled="true" multipleSiteBindingsEnabled="true" /> <behaviors> <serviceBehaviors> <behavior name="Service1"> <serviceMetadata httpGetEnabled="true" /> <serviceDebug includeExceptionDetailInFaults="true" /> <dataContractSerializer maxItemsInObjectGraph="2000000000" /> </behavior> <behavior name=""> <serviceMetadata httpGetEnabled="true" /> <serviceDebug includeExceptionDetailInFaults="false" /> </behavior> <behavior name="nextSPOTServiceBehavior"> <serviceMetadata httpsGetEnabled="true"/> <serviceDebug includeExceptionDetailInFaults="true" /> <dataContractSerializer maxItemsInObjectGraph="2000000000" /> </behavior> </serviceBehaviors> </behaviors> <bindings> <basicHttpBinding> <binding name="SecureBasic" closeTimeout="00:10:00" openTimeout="00:10:00" receiveTimeout="00:10:00" sendTimeout="00:10:00" maxBufferSize="2147483647" maxReceivedMessageSize="2147483647"> <security mode="Transport" /> <readerQuotas maxArrayLength="2000000" maxStringContentLength="2000000"/> </binding> <binding name="BasicHttpBinding_IDownloadManagerService" closeTimeout="00:10:00" openTimeout="00:10:00" receiveTimeout="00:10:00" sendTimeout="00:10:00" maxBufferSize="2147483647" maxReceivedMessageSize="2147483647"> <security mode="Transport" /> </binding> </basicHttpBinding> </bindings> <services> <service behaviorConfiguration="nextSPOTServiceBehavior" name="NextSPOTDownloadManagerWebServiceTester.Web.WebServices.DownloadManagerService"> <endpoint binding="basicHttpBinding" bindingConfiguration="SecureBasic" name="basicHttpSecure" contract="NextSPOTDownloadManagerWebServiceTester.Web.WebServices.IDownloadManagerService" /> <!--<endpoint binding="basicHttpBinding" bindingConfiguration="" name="basicHttp" contract="NextSPOTDownloadManagerWebServiceTester.Web.WebServices.IDownloadManagerService" />--> <!--<endpoint address="" binding="basicHttpBinding" bindingConfiguration="BasicHttpBinding_IDownloadManagerService" contract="NextSPOTDownloadManagerWebServiceTester.Web.WebServices.IDownloadManagerService" /> <endpoint address="mex" binding="mexHttpsBinding" contract="IMetadataExchange" />--> </service> </services > </system.serviceModel> </configuration>

    Read the article

  • OpenSSL in C++ email client - server closes connection with TLSv1 Alert message

    - by mice
    My app connects to a IMAP email server. One client configured his server to reject SSLv2 certificates, and now my app fails to connect to the server. All other email clients connect to this server successfully. My app uses openssl. I debugged by creating minimal openssl client and attempt to connect to the server. Below is the code with connects to the mail server (using Windows sockets, but same problem is with unix sockets). Server sends its initial IMAP greeting message, but after client sends 1st command, server closes connection. In Wireshark, I see that after sending command to server, it returns TLSv1 error message 21 (Encrypted Alert) and connection is gone. I'm looking for proper setup of OpenSSL for this connection to succeed. Thanks #include <stdio.h> #include <memory.h> #include <errno.h> #include <sys/types.h> #include <winsock2.h> #include <openssl/crypto.h> #include <openssl/x509.h> #include <openssl/pem.h> #include <openssl/ssl.h> #include <openssl/err.h> #define CHK_NULL(x) if((x)==NULL) exit(1) #define CHK_ERR(err,s) if((err)==-1) { perror(s); exit(1); } #define CHK_SSL(err) if((err)==-1) { ERR_print_errors_fp(stderr); exit(2); } SSL *ssl; char buf[4096]; void write(const char *s){ int err = SSL_write(ssl, s, strlen(s)); printf("> %s\n", s); CHK_SSL(err); } void read(){ int n = SSL_read(ssl, buf, sizeof(buf) - 1); CHK_SSL(n); if(n==0){ printf("Finished\n"); exit(1); } buf[n] = 0; printf("%s\n", buf); } void main(){ int err=0; SSLeay_add_ssl_algorithms(); SSL_METHOD *meth = SSLv23_client_method(); SSL_load_error_strings(); SSL_CTX *ctx = SSL_CTX_new(meth); CHK_NULL(ctx); WSADATA data; WSAStartup(0x202, &data); int sd = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP); CHK_ERR(sd, "socket"); struct sockaddr_in sa; memset(&sa, 0, sizeof(sa)); sa.sin_family = AF_INET; sa.sin_addr.s_addr = inet_addr("195.137.27.14"); sa.sin_port = htons(993); err = connect(sd,(struct sockaddr*) &sa, sizeof(sa)); CHK_ERR(err, "connect"); /* ----------------------------------------------- */ /* Now we have TCP connection. Start SSL negotiation. */ ssl = SSL_new(ctx); CHK_NULL(ssl); SSL_set_fd(ssl, sd); err = SSL_connect(ssl); CHK_SSL(err); // Following two steps are optional and not required for data exchange to be successful. /* printf("SSL connection using %s\n", SSL_get_cipher(ssl)); X509 *server_cert = SSL_get_peer_certificate(ssl); CHK_NULL(server_cert); printf("Server certificate:\n"); char *str = X509_NAME_oneline(X509_get_subject_name(server_cert),0,0); CHK_NULL(str); printf(" subject: %s\n", str); OPENSSL_free(str); str = X509_NAME_oneline(X509_get_issuer_name (server_cert),0,0); CHK_NULL(str); printf(" issuer: %s\n", str); OPENSSL_free(str); // We could do all sorts of certificate verification stuff here before deallocating the certificate. X509_free(server_cert); */ printf("\n\n"); read(); // get initial IMAP greeting write("1 CAPABILITY\r\n"); // send 1st command read(); // get reply to cmd; server closes connection here write("2 LOGIN a b\r\n"); read(); SSL_shutdown(ssl); closesocket(sd); SSL_free(ssl); SSL_CTX_free(ctx); }

    Read the article

  • Is multithreading the right way to go for my case?

    - by Julien Lebosquain
    Hello, I'm currently designing a multi-client / server application. I'm using plain good old sockets because WCF or similar technology is not what I need. Let me explain: it isn't the classical case of a client simply calling a service; all clients can 'interact' with each other by sending a packet to the server, which will then do some action, and possible re-dispatch an answer message to one or more clients. Although doable with WCF, the application will get pretty complex with hundreds of different messages. For each connected client, I'm of course using asynchronous methods to send and receive bytes. I've got the messages fully working, everything's fine. Except that for each line of code I'm writing, my head just burns because of multithreading issues. Since there could be around 200 clients connected at the same time, I chose to go the fully multithreaded way: each received message on a socket is immediately processed on the thread pool thread it was received, not on a single consumer thread. Since each client can interact with other clients, and indirectly with shared objects on the server, I must protect almost every object that is mutable. I first went with a ReaderWriterLockSlim for each resource that must be protected, but quickly noticed that there are more writes overall than reads in the server application, and switched to the well-known Monitor to simplify the code. So far, so good. Each resource is protected, I have helper classes that I must use to get a lock and its protected resource, so I can't use an object without getting a lock. Moreover, each client has its own lock that is entered as soon as a packet is received from its socket. It's done to prevent other clients from making changes to the state of this client while it has some messages being processed, which is something that will happen frequently. Now, I don't just need to protect resources from concurrent accesses. I must keep every client in sync with the server for some collections I have. One tricky part that I'm currently struggling with is the following: I have a collection of clients. Each client has its own unique ID. When a client connects, it must receive the IDs of every connected client, and each one of them must be notified of the newcomer's ID. When a client disconnects, every other client must know it so that its ID is no longer valid for them. Every client must always have, at a given time, the same clients collection as the server so that I can assume that everybody knows everybody. This way if I'm sending a message to client #1 telling "Client #2 has done something", I know that it will always be correctly interpreted: Client 1 will never wonder "but who is Client 2 anyway?". My first attempt for handling the connection of a new client (let's call it X) was this pseudo-code (remember that newClient is already locked here): lock (clients) { foreach (var client in clients) { lock (client) { client.Send("newClient with id X has connected"); } } clients.Add(newClient); newClient.Send("the list of other clients"); } Now imagine that in the same time, another client has sent a packet that translates into a message that must be broadcasted to every connected client, the pseudo-code will be something like this (remember that the current client - let's call it Y - is already locked here): lock (clients) { foreach (var client in clients) { lock (client) { client.Send("something"); } } } An obvious deadlock occurs here: on one thread X is locked, the clients lock has been entered, started looping through the clients, and at one moment must get Y's lock... which is already acquired on the second thread, itself waiting for the clients collection lock to be released! This is not the only case like this in the server application. There are other collections which must be kept in sync with the clients, some properties on a client can be changed by another one, etc. I tried other types of locks, lock-free mechanisms and a bunch of other things. Either there were obvious deadlocks when I'm using too much locks for safety, or obvious race conditions otherwise. When I finally find a good middle point between the two, it usually comes with very subtle race conditions / dead locks and other multi-threading issues... my head hurts very quickly since for any single line of code I'm writing I have to review almost the whole application to ensure everything will behave correctly with any number of threads. So here's my final question: how would you resolve this specific case, the general case, and more importantly: aren't I going the wrong way here? I have little problems with the .NET framework, C#, simple concurrency or algorithms in general. Still, I'm lost here. I know I could use only one thread processing the incoming requests and everything will be fine. However, that won't scale well at all with more clients... But I'm thinking more and more to go this simple way. What do you think? Thanks in advance to you, StackOverflow people which have taken the time to read this huge question. I really had to explain the whole context if I want to get some help.

    Read the article

  • With XSLT, how can I use this if-test with an array, when search element is returned by a template call inside the for loop?

    - by codesforcoffee
    I think this simple example might ask the question a lot more clearly. I have an input file with multiple products. There are 10 types of product (2 product IDs is fine enough for this example), but the input will have 200 products, and I only want to output the info for the first product of each type. (Output info for the lowest priced one, so the first one will be the lowest price because I sort by Price first.) So I want to read in each product, but only output the product's info if I haven't already output a product with that same ID. I couldn't figure out how to get the processID template to return a value that I need to do my if-check on, that uses parameters from inside the for-each Product loop -then properly close the if tag in the right place so it won't output the open Product tag unless it passes the if test. I know the following code does not work, but it illustrates the idea and gives me a place to start: <?xml version="1.0" encoding="utf-8"?> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> <xsl:output method="xml" encoding="UTF-8" indent="yes" cdata-section-elements="prod_name adv_notes"/> <xsl:template match="/"> <List> <xsl:for-each select="ProductGroup"> <xsl:sort select="ActiveProducts/Product/Rate"/> <xsl:variable name="IDarray"> <xsl:for-each select="ActiveProducts/Product"> <xsl:variable name="CurrentID"> <xsl:call-template name="processID"> <xsl:with-param name="ProductCode" select="ProductCode" /> </xsl:call-template> </xsl:variable> <xsl:if test="not(contains($IDarray, $CurrentID))"> <child elem="{@elem}"> <xsl:select value-of="$CurrentID" /> </child> <Product> <xsl:attribute name="ID"> <xsl:select value-of="$CurrentID" /> </xsl:attribute> <prod_name> <xsl:value-of select="../ProductName"/> </prod_name> <rate> <xsl:value-of select="../Rate"/> </rate> </Product> </xsl:if> </xsl:for-each> </xsl:variable> </xsl:for-each> </List> </xsl:template> <xsl:template name="processID"> <xsl:param name="ProductCode"/> <xsl:choose> <xsl:when test="starts-with($ProductCode, '515')">5</xsl:when> <xsl:when test="starts-with($ProductCode, '205')">2</xsl:when> </xsl:choose> </xsl:template> Thanks so much in advance, I know some of the awesome programmers here can help! :) -Holly An input would look like this: <ProductGroup> <ActiveProducts> <Product> <ProductCode> 5155 </ProductCode> <ProductName> House </ProductName> <Rate> 3.99 </Rate> </Product> <Product> <ProductCode> 5158 </ProductCode> <ProductName> House </ProductName> <Rate> 4.99 </Rate> </Product> </ActiveProducts> </ProductGroup> <ProductGroup> <ActiveProducts> <Product> <ProductCode> 2058 </ProductCode> <ProductName> House </ProductName> <Rate> 2.99 </Rate> </Product> <Product> <ProductCode> 2055 </ProductCode> <ProductName> House </ProductName> <Rate> 7.99 </Rate> </Product> </ActiveProducts> </ProductGroup> 200 of those with different attributes. I have the translation working, just needed to add that array and if statement somehow. Output would be this for only that simple input file:

    Read the article

  • Adapting non-iterable containers to be iterated via custom templatized iterator

    - by DAldridge
    I have some classes, which for various reasons out of scope of this discussion, I cannot modify (irrelevant implementation details omitted): class Foo { /* ... irrelevant public interface ... */ }; class Bar { public: Foo& get_foo(size_t index) { /* whatever */ } size_t size_foo() { /* whatever */ } }; (There are many similar 'Foo' and 'Bar' classes I'm dealing with, and it's all generated code from elsewhere and stuff I don't want to subclass, etc.) [Edit: clarification - although there are many similar 'Foo' and 'Bar' classes, it is guaranteed that each "outer" class will have the getter and size methods. Only the getter method name and return type will differ for each "outer", based on whatever it's "inner" contained type is. So, if I have Baz which contains Quux instances, there will be Quux& Baz::get_quux(size_t index), and size_t Baz::size_quux().] Given the design of the Bar class, you cannot easily use it in STL algorithms (e.g. for_each, find_if, etc.), and must do imperative loops rather than taking a functional approach (reasons why I prefer the latter is also out of scope for this discussion): Bar b; size_t numFoo = b.size_foo(); for (int fooIdx = 0; fooIdx < numFoo; ++fooIdx) { Foo& f = b.get_foo(fooIdx); /* ... do stuff with 'f' ... */ } So... I've never created a custom iterator, and after reading various questions/answers on S.O. about iterator_traits and the like, I came up with this (currently half-baked) "solution": First, the custom iterator mechanism (NOTE: all uses of 'function' and 'bind' are from std::tr1 in MSVC9): // Iterator mechanism... template <typename TOuter, typename TInner> class ContainerIterator : public std::iterator<std::input_iterator_tag, TInner> { public: typedef function<TInner& (size_t)> func_type; ContainerIterator(const ContainerIterator& other) : mFunc(other.mFunc), mIndex(other.mIndex) {} ContainerIterator& operator++() { ++mIndex; return *this; } bool operator==(const ContainerIterator& other) { return ((mFunc.target<TOuter>() == other.mFunc.target<TOuter>()) && (mIndex == other.mIndex)); } bool operator!=(const ContainerIterator& other) { return !(*this == other); } TInner& operator*() { return mFunc(mIndex); } private: template<typename TOuter, typename TInner> friend class ContainerProxy; ContainerIterator(func_type func, size_t index = 0) : mFunc(func), mIndex(index) {} function<TInner& (size_t)> mFunc; size_t mIndex; }; Next, the mechanism by which I get valid iterators representing begin and end of the inner container: // Proxy(?) to the outer class instance, providing a way to get begin() and end() // iterators to the inner contained instances... template <typename TOuter, typename TInner> class ContainerProxy { public: typedef function<TInner& (size_t)> access_func_type; typedef function<size_t ()> size_func_type; typedef ContainerIterator<TOuter, TInner> iter_type; ContainerProxy(access_func_type accessFunc, size_func_type sizeFunc) : mAccessFunc(accessFunc), mSizeFunc(sizeFunc) {} iter_type begin() const { size_t numItems = mSizeFunc(); if (0 == numItems) return end(); else return ContainerIterator<TOuter, TInner>(mAccessFunc, 0); } iter_type end() const { size_t numItems = mSizeFunc(); return ContainerIterator<TOuter, TInner>(mAccessFunc, numItems); } private: access_func_type mAccessFunc; size_func_type mSizeFunc; }; I can use these classes in the following manner: // Sample function object for taking action on an LMX inner class instance yielded // by iteration... template <typename TInner> class SomeTInnerFunctor { public: void operator()(const TInner& inner) { /* ... whatever ... */ } }; // Example of iterating over an outer class instance's inner container... Bar b; /* assume populated which contained items ... */ ContainerProxy<Bar, Foo> bProxy( bind(&Bar::get_foo, b, _1), bind(&Bar::size_foo, b)); for_each(bProxy.begin(), bProxy.end(), SomeTInnerFunctor<Foo>()); Empirically, this solution functions correctly (minus any copy/paste or typos I may have introduced when editing the above for brevity). So, finally, the actual question: I don't like requiring the use of bind() and _1 placeholders, etcetera by the caller. All they really care about is: outer type, inner type, outer type's method to fetch inner instances, outer type's method to fetch count inner instances. Is there any way to "hide" the bind in the body of the template classes somehow? I've been unable to find a way to separately supply template parameters for the types and inner methods separately... Thanks! David

    Read the article

  • JAVA-SQL- Data Migration - ResultSets comparing Failing JUnit test

    - by user1865053
    I CANNOT get this JUnit Test to pass for the life of me. Can somebody point out where this has gone wrong. I am doing a data migration(MSSQL SERVER 2005), but I have the sourceDBUrl and the targetDCUrl the same URL so to narrow it down to syntax errors. So that is what I have, a syntax error. I am comparing the results of a table for the query SELECT programmeapproval, resourceapproval FROM tr_timesheet WHERE timesheetid = ? and the test always fails, but passes for other junit tests I have developed. I created 3 diffemt resultSetsEqual methods and none work. Yet, some other JUnit tests I have developed have PASSED. THE QUERY: SELECT timesheetid, programmeapproval, resourceapproval FROM tr_timesheet Returns three columns timesheetid (PK,int, not null) (populated with a range of numbers 2240 - 2282) programmeapproval (smallint,not null) (populated with the number 1 in every field) resourceapproval (smallint, not null) (populated with a number 1 in every field) When I run the query that is embedded in the code it only returns one row with the programmeapproval and resourceapproval columns and both field populated with the number 1. I have all jdbc drivers correctly installed and tested for connectivity. The JUnit Test is failing at this point according to the IDE. assertTrue(helper.resultSetsEqual2(sourceVal,targetVal)); This is the code: /*THIS IS A JUNIT CLASS****? package a7.unittests.dao; import static org.junit.Assert.assertTrue; import java.sql.Connection; import java.sql.PreparedStatement; import java.sql.ResultSet; import java.sql.Types; import org.junit.Test; import artemispm.tritonalerts.TimesheetAlert; public class UnitTestTimesheetAlert { @Test public void testQUERY_CHECKALERT() throws Exception{ UnitTestHelper helper = new UnitTestHelper(); Connection con = helper.getConnection(helper.sourceDBUrl); Connection conTarget = helper.getConnection(helper.targetDBUrl); PreparedStatement stmt = con.prepareStatement("select programmeapproval, resourceapproval from tr_timesheet where timesheetid = ?"); stmt.setInt(1, 2240); ResultSet sourceVal = stmt.executeQuery(); stmt = conTarget.prepareStatement("select programmeapproval, resourceapproval from tr_timesheet where timesheetid = ?"); stmt.setInt(1,2240); ResultSet targetVal = stmt.executeQuery(); assertTrue(helper.resultSetsEqual2(sourceVal,targetVal)); }} /*END**/ /*THIS IS A REGULAR CLASS**/ package a7.unittests.dao; import java.sql.Connection; import java.sql.DriverManager; import java.sql.ResultSet; import java.sql.ResultSetMetaData; import java.sql.SQLException; public class UnitTestHelper { static String sourceDBUrl = "jdbc:sqlserver://127.0.0.1:1433;databaseName=a7itm;user=a7user;password=a7user"; static String targetDBUrl = "jdbc:sqlserver://127.0.0.1:1433;databaseName=a7itm;user=a7user;password=a7user"; public Connection getConnection(String url)throws Exception{ return DriverManager.getConnection(url); } public boolean resultSetsEqual3 (ResultSet rs1, ResultSet rs2) throws SQLException { int col = 1; //ResultSetMetaData metadata = rs1.getMetaData(); //int count = metadata.getColumnCount(); while (rs1.next() && rs2.next()) { final Object res1 = rs1.getObject(col); final Object res2 = rs2.getObject(col); // Check values if (!res1.equals(res2)) { throw new RuntimeException(String.format("%s and %s aren't equal at common position %d", res1, res2, col)); } // rs1 and rs2 must reach last row in the same iteration if ((rs1.isLast() != rs2.isLast())) { throw new RuntimeException("The two ResultSets contains different number of columns!"); } } return true; } public boolean resultSetsEqual (ResultSet source, ResultSet target) throws SQLException{ while(source.next()) { target.next(); ResultSetMetaData metadata = source.getMetaData(); int count = metadata.getColumnCount(); for (int i =1; i<=count; i++) { if(source.getObject(i) != target.getObject(i)) { return false; } } } return true; } public boolean resultSetsEqual2 (ResultSet source, ResultSet target) throws SQLException{ while(source.next()) { target.next(); ResultSetMetaData metadata = source.getMetaData(); int count = metadata.getColumnCount(); for (int i =1; i<=count; i++) { if(source.getObject(i).equals(target.getObject(i))) { return false; } } } return true; } } /END***/ /*PASTED NEW CLASS - THIS IS A JUNIT TEST CLASS*/ package a7.unittests.dao; import static org.junit.Assert.*; import java.sql.Connection; import java.sql.DriverManager; import org.junit.Test; public class TestDatabaseConnection { @Test public void testConnection() throws Exception{ UnitTestHelper helper = new UnitTestHelper(); Connection con = helper.getConnection(helper.sourceDBUrl); Connection conTarget = helper.getConnection(helper.targetDBUrl); assertTrue(con != null && conTarget != null); } } /**END***/

    Read the article

  • How to set size for divs with different parents

    - by user340524
    I want to create a div layout which is similiar to the following table result: <html> <head> <title>Basic</title> <style> table { border: 1px solid;} </style> </head> <body> <table style="border: 1px solid;"> <tr> <td> Asia</td> <td> <table> <tr> <td>South Asia</td> </td> <td><table> <tr> <td>Republic</td> <td><table> <tr><td>Singapore</td></tr> <tr><td>India</td></tr> </table></td> </tr> <tr> <td>Monarchy</td> <td><table> <tr><td>Bhutan</td></tr> <tr><td>Nepal</td></tr> </table></td> </tr> </table></td> </tr> <tr> <td>East Asia</td> <td><table> <tr> <td>Republic</td> <td><table> <tr><td>China</td></tr> <tr><td>South Corea</td></tr> </table></td> </tr> <tr> <td>Constitutional Monarchy</td> <td><table> <tr><td>something</td></tr> <tr><td>Japan</td></tr> </table></td> </tr> </table></td> </tr> </table></td> </tr> </table> </body> </html> I managed to replicate this with some effort. The problem is that I want the names of the countries to be in a column or if you will - the containers for the government types to be the same width so other containers will align. If I don't do it in nested containers (in the example - nested tables) the rows will get displaced. Currently rows are shown exactly how I want them - the text is in the vertical middle of the what they refer to. Only thing that comes up to my mind is to set the text in the same columns as class=column1, class=column2, etc. and then somehow define the width for the column classes. Problem is the data is defined dynamically and I can't say how much pixels or % of the page I can give to a column, I just need it to stretch with the text. This is my first time I ask about help here so if I am doing it wrong, tell me how do improve my inquiry.

    Read the article

  • Microsoft and jQuery

    - by Rick Strahl
    The jQuery JavaScript library has been steadily getting more popular and with recent developments from Microsoft, jQuery is also getting ever more exposure on the ASP.NET platform including now directly from Microsoft. jQuery is a light weight, open source DOM manipulation library for JavaScript that has changed how many developers think about JavaScript. You can download it and find more information on jQuery on www.jquery.com. For me jQuery has had a huge impact on how I develop Web applications and was probably the main reason I went from dreading to do JavaScript development to actually looking forward to implementing client side JavaScript functionality. It has also had a profound impact on my JavaScript skill level for me by seeing how the library accomplishes things (and often reviewing the terse but excellent source code). jQuery made an uncomfortable development platform (JavaScript + DOM) a joy to work on. Although jQuery is by no means the only JavaScript library out there, its ease of use, small size, huge community of plug-ins and pure usefulness has made it easily the most popular JavaScript library available today. As a long time jQuery user, I’ve been excited to see the developments from Microsoft that are bringing jQuery to more ASP.NET developers and providing more integration with jQuery for ASP.NET’s core features rather than relying on the ASP.NET AJAX library. Microsoft and jQuery – making Friends jQuery is an open source project but in the last couple of years Microsoft has really thrown its weight behind supporting this open source library as a supported component on the Microsoft platform. When I say supported I literally mean supported: Microsoft now offers actual tech support for jQuery as part of their Product Support Services (PSS) as jQuery integration has become part of several of the ASP.NET toolkits and ships in several of the default Web project templates in Visual Studio 2010. The ASP.NET MVC 3 framework (still in Beta) also uses jQuery for a variety of client side support features including client side validation and we can look forward toward more integration of client side functionality via jQuery in both MVC and WebForms in the future. In other words jQuery is becoming an optional but included component of the ASP.NET platform. PSS support means that support staff will answer jQuery related support questions as part of any support incidents related to ASP.NET which provides some piece of mind to some corporate development shops that require end to end support from Microsoft. In addition to including jQuery and supporting it, Microsoft has also been getting involved in providing development resources for extending jQuery’s functionality via plug-ins. Microsoft’s last version of the Microsoft Ajax Library – which is the successor to the native ASP.NET AJAX Library – included some really cool functionality for client templates, databinding and localization. As it turns out Microsoft has rebuilt most of that functionality using jQuery as the base API and provided jQuery plug-ins of these components. Very recently these three plug-ins were submitted and have been approved for inclusion in the official jQuery plug-in repository and been taken over by the jQuery team for further improvements and maintenance. Even more surprising: The jQuery-templates component has actually been approved for inclusion in the next major update of the jQuery core in jQuery V1.5, which means it will become a native feature that doesn’t require additional script files to be loaded. Imagine this – an open source contribution from Microsoft that has been accepted into a major open source project for a core feature improvement. Microsoft has come a long way indeed! What the Microsoft Involvement with jQuery means to you For Microsoft jQuery support is a strategic decision that affects their direction in client side development, but nothing stopped you from using jQuery in your applications prior to Microsoft’s official backing and in fact a large chunk of developers did so readily prior to Microsoft’s announcement. Official support from Microsoft brings a few benefits to developers however. jQuery support in Visual Studio 2010 means built-in support for jQuery IntelliSense, automatically added jQuery scripts in many projects types and a common base for client side functionality that actually uses what most developers are already using. If you have already been using jQuery and were worried about straying from the Microsoft line and their internal Microsoft Ajax Library – worry no more. With official support and the change in direction towards jQuery Microsoft is now following along what most in the ASP.NET community had already been doing by using jQuery, which is likely the reason for Microsoft’s shift in direction in the first place. ASP.NET AJAX and the Microsoft AJAX Library weren’t bad technology – there was tons of useful functionality buried in these libraries. However, these libraries never got off the ground, mainly because early incarnations were squarely aimed at control/component developers rather than application developers. For all the functionality that these controls provided for control developers they lacked in useful and easily usable application developer functionality that was easily accessible in day to day client side development. The result was that even though Microsoft shipped support for these tools in the box (in .NET 3.5 and 4.0), other than for the internal support in ASP.NET for things like the UpdatePanel and the ASP.NET AJAX Control Toolkit as well as some third party vendors, the Microsoft client libraries were largely ignored by the developer community opening the door for other client side solutions. Microsoft seems to be acknowledging developer choice in this case: Many more developers were going down the jQuery path rather than using the Microsoft built libraries and there seems to be little sense in continuing development of a technology that largely goes unused by the majority of developers. Kudos for Microsoft for recognizing this and gracefully changing directions. Note that even though there will be no further development in the Microsoft client libraries they will continue to be supported so if you’re using them in your applications there’s no reason to start running for the exit in a panic and start re-writing everything with jQuery. Although that might be a reasonable choice in some cases, jQuery and the Microsoft libraries work well side by side so that you can leave existing solutions untouched even as you enhance them with jQuery. The Microsoft jQuery Plug-ins – Solid Core Features One of the most interesting developments in Microsoft’s embracing of jQuery is that Microsoft has started contributing to jQuery via standard mechanism set for jQuery developers: By submitting plug-ins. Microsoft took some of the nicest new features of the unpublished Microsoft Ajax Client Library and re-wrote these components for jQuery and then submitted them as plug-ins to the jQuery plug-in repository. Accepted plug-ins get taken over by the jQuery team and that’s exactly what happened with the three plug-ins submitted by Microsoft with the templating plug-in even getting slated to be published as part of the jQuery core in the next major release (1.5). The following plug-ins are provided by Microsoft: jQuery Templates – a client side template rendering engine jQuery Data Link – a client side databinder that can synchronize changes without code jQuery Globalization – provides formatting and conversion features for dates and numbers The first two are ports of functionality that was slated for the Microsoft Ajax Library while functionality for the globalization library provides functionality that was already found in the original ASP.NET AJAX library. To me all three plug-ins address a pressing need in client side applications and provide functionality I’ve previously used in other incarnations, but with more complete implementations. Let’s take a close look at these plug-ins. jQuery Templates http://api.jquery.com/category/plugins/templates/ Client side templating is a key component for building rich JavaScript applications in the browser. Templating on the client lets you avoid from manually creating markup by creating DOM nodes and injecting them individually into the document via code. Rather you can create markup templates – similar to the way you create classic ASP server markup – and merge data into these templates to render HTML which you can then inject into the document or replace existing content with. Output from templates are rendered as a jQuery matched set and can then be easily inserted into the document as needed. Templating is key to minimize client side code and reduce repeated code for rendering logic. Instead a single template can be used in many places for updating and adding content to existing pages. Further if you build pure AJAX interfaces that rely entirely on client rendering of the initial page content, templates allow you to a use a single markup template to handle all rendering of each specific HTML section/element. I’ve used a number of different client rendering template engines with jQuery in the past including jTemplates (a PHP style templating engine) and a modified version of John Resig’s MicroTemplating engine which I built into my own set of libraries because it’s such a commonly used feature in my client side applications. jQuery templates adds a much richer templating model that allows for sub-templates and access to the data items. Like John Resig’s original Micro Template engine, the core basics of the templating engine create JavaScript code which means that templates can include JavaScript code. To give you a basic idea of how templates work imagine I have an application that downloads a set of stock quotes based on a symbol list then displays them in the document. To do this you can create an ‘item’ template that describes how each of the quotes is renderd as a template inside of the document: <script id="stockTemplate" type="text/x-jquery-tmpl"> <div id="divStockQuote" class="errordisplay" style="width: 500px;"> <div class="label">Company:</div><div><b>${Company}(${Symbol})</b></div> <div class="label">Last Price:</div><div>${LastPrice}</div> <div class="label">Net Change:</div><div> {{if NetChange > 0}} <b style="color:green" >${NetChange}</b> {{else}} <b style="color:red" >${NetChange}</b> {{/if}} </div> <div class="label">Last Update:</div><div>${LastQuoteTimeString}</div> </div> </script> The ‘template’ is little more than HTML with some markup expressions inside of it that define the template language. Notice the embedded ${} expressions which reference data from the quote objects returned from an AJAX call on the server. You can embed any JavaScript or value expression in these template expressions. There are also a number of structural commands like {{if}} and {{each}} that provide for rudimentary logic inside of your templates as well as commands ({{tmpl}} and {{wrap}}) for nesting templates. You can find more about the full set of markup expressions available in the documentation. To load up this data you can use code like the following: <script type="text/javascript"> //var Proxy = new ServiceProxy("../PageMethods/PageMethodsService.asmx/"); $(document).ready(function () { $("#btnGetQuotes").click(GetQuotes); }); function GetQuotes() { var symbols = $("#txtSymbols").val().split(","); $.ajax({ url: "../PageMethods/PageMethodsService.asmx/GetStockQuotes", data: JSON.stringify({ symbols: symbols }), // parameter map type: "POST", // data has to be POSTed contentType: "application/json", timeout: 10000, dataType: "json", success: function (result) { var quotes = result.d; var jEl = $("#stockTemplate").tmpl(quotes); $("#quoteDisplay").empty().append(jEl); }, error: function (xhr, status) { alert(status + "\r\n" + xhr.responseText); } }); }; </script> In this case an ASMX AJAX service is called to retrieve the stock quotes. The service returns an array of quote objects. The result is returned as an object with the .d property (in Microsoft service style) that returns the actual array of quotes. The template is applied with: var jEl = $("#stockTemplate").tmpl(quotes); which selects the template script tag and uses the .tmpl() function to apply the data to it. The result is a jQuery matched set of elements that can then be appended to the quote display element in the page. The template is merged against an array in this example. When the result is an array the template is automatically applied to each each array item. If you pass a single data item – like say a stock quote – the template works exactly the same way but is applied only once. Templates also have access to a $data item which provides the current data item and information about the tempalte that is currently executing. This makes it possible to keep context within the context of the template itself and also to pass context from a parent template to a child template which is very powerful. Templates can be evaluated by using the template selector and calling the .tmpl() function on the jQuery matched set as shown above or you can use the static $.tmpl() function to provide a template as a string. This allows you to dynamically create templates in code or – more likely – to load templates from the server via AJAX calls. In short there are options The above shows off some of the basics, but there’s much for functionality available in the template engine. Check the documentation link for more information and links to additional examples. The plug-in download also comes with a number of examples that demonstrate functionality. jQuery templates will become a native component in jQuery Core 1.5, so it’s definitely worthwhile checking out the engine today and get familiar with this interface. As much as I’m stoked about templating becoming part of the jQuery core because it’s such an integral part of many applications, there are also a couple shortcomings in the current incarnation: Lack of Error Handling Currently if you embed an expression that is invalid it’s simply not rendered. There’s no error rendered into the template nor do the various  template functions throw errors which leaves finding of bugs as a runtime exercise. I would like some mechanism – optional if possible – to be able to get error info of what is failing in a template when it’s rendered. No String Output Templates are always rendered into a jQuery matched set and there’s no way that I can see to directly render to a string. String output can be useful for debugging as well as opening up templating for creating non-HTML string output. Limited JavaScript Access Unlike John Resig’s original MicroTemplating Engine which was entirely based on JavaScript code generation these templates are limited to a few structured commands that can ‘execute’. There’s no code execution inside of script code which means you’re limited to calling expressions available in global objects or the data item passed in. This may or may not be a big deal depending on the complexity of your template logic. Error handling has been discussed quite a bit and it’s likely there will be some solution to that particualar issue by the time jQuery templates ship. The others are relatively minor issues but something to think about anyway. jQuery Data Link http://api.jquery.com/category/plugins/data-link/ jQuery Data Link provides the ability to do two-way data binding between input controls and an underlying object’s properties. The typical scenario is linking a textbox to a property of an object and have the object updated when the text in the textbox is changed and have the textbox change when the value in the object or the entire object changes. The plug-in also supports converter functions that can be applied to provide the conversion logic from string to some other value typically necessary for mapping things like textbox string input to say a number property and potentially applying additional formatting and calculations. In theory this sounds great, however in reality this plug-in has some serious usability issues. Using the plug-in you can do things like the following to bind data: person = { firstName: "rick", lastName: "strahl"}; $(document).ready( function() { // provide for two-way linking of inputs $("form").link(person); // bind to non-input elements explicitly $("#objFirst").link(person, { firstName: { name: "objFirst", convertBack: function (value, source, target) { $(target).text(value); } } }); $("#objLast").link(person, { lastName: { name: "objLast", convertBack: function (value, source, target) { $(target).text(value); } } }); }); This code hooks up two-way linking between a couple of textboxes on the page and the person object. The first line in the .ready() handler provides mapping of object to form field with the same field names as properties on the object. Note that .link() does NOT bind items into the textboxes when you call .link() – changes are mapped only when values change and you move out of the field. Strike one. The two following commands allow manual binding of values to specific DOM elements which is effectively a one-way bind. You specify the object and a then an explicit mapping where name is an ID in the document. The converter is required to explicitly assign the value to the element. Strike two. You can also detect changes to the underlying object and cause updates to the input elements bound. Unfortunately the syntax to do this is not very natural as you have to rely on the jQuery data object. To update an object’s properties and get change notification looks like this: function updateFirstName() { $(person).data("firstName", person.firstName + " (code updated)"); } This works fine in causing any linked fields to be updated. In the bindings above both the firstName input field and objFirst DOM element gets updated. But the syntax requires you to use a jQuery .data() call for each property change to ensure that the changes are tracked properly. Really? Sure you’re binding through multiple layers of abstraction now but how is that better than just manually assigning values? The code savings (if any) are going to be minimal. As much as I would like to have a WPF/Silverlight/Observable-like binding mechanism in client script, this plug-in doesn’t help much towards that goal in its current incarnation. While you can bind values, the ‘binder’ is too limited to be really useful. If initial values can’t be assigned from the mappings you’re going to end up duplicating work loading the data using some other mechanism. There’s no easy way to re-bind data with a different object altogether since updates trigger only through the .data members. Finally, any non-input elements have to be bound via code that’s fairly verbose and frankly may be more voluminous than what you might write by hand for manual binding and unbinding. Two way binding can be very useful but it has to be easy and most importantly natural. If it’s more work to hook up a binding than writing a couple of lines to do binding/unbinding this sort of thing helps very little in most scenarios. In talking to some of the developers the feature set for Data Link is not complete and they are still soliciting input for features and functionality. If you have ideas on how you want this feature to be more useful get involved and post your recommendations. As it stands, it looks to me like this component needs a lot of love to become useful. For this component to really provide value, bindings need to be able to be refreshed easily and work at the object level, not just the property level. It seems to me we would be much better served by a model binder object that can perform these binding/unbinding tasks in bulk rather than a tool where each link has to be mapped first. I also find the choice of creating a jQuery plug-in questionable – it seems a standalone object – albeit one that relies on the jQuery library – would provide a more intuitive interface than the current forcing of options onto a plug-in style interface. Out of the three Microsoft created components this is by far the least useful and least polished implementation at this point. jQuery Globalization http://github.com/jquery/jquery-global Globalization in JavaScript applications often gets short shrift and part of the reason for this is that natively in JavaScript there’s little support for formatting and parsing of numbers and dates. There are a number of JavaScript libraries out there that provide some support for globalization, but most are limited to a particular portion of globalization. As .NET developers we’re fairly spoiled by the richness of APIs provided in the framework and when dealing with client development one really notices the lack of these features. While you may not necessarily need to localize your application the globalization plug-in also helps with some basic tasks for non-localized applications: Dealing with formatting and parsing of dates and time values. Dates in particular are problematic in JavaScript as there are no formatters whatsoever except the .toString() method which outputs a verbose and next to useless long string. With the globalization plug-in you get a good chunk of the formatting and parsing functionality that the .NET framework provides on the server. You can write code like the following for example to format numbers and dates: var date = new Date(); var output = $.format(date, "MMM. dd, yy") + "\r\n" + $.format(date, "d") + "\r\n" + // 10/25/2010 $.format(1222.32213, "N2") + "\r\n" + $.format(1222.33, "c") + "\r\n"; alert(output); This becomes even more useful if you combine it with templates which can also include any JavaScript expressions. Assuming the globalization plug-in is loaded you can create template expressions that use the $.format function. Here’s the template I used earlier for the stock quote again with a couple of formats applied: <script id="stockTemplate" type="text/x-jquery-tmpl"> <div id="divStockQuote" class="errordisplay" style="width: 500px;"> <div class="label">Company:</div><div><b>${Company}(${Symbol})</b></div> <div class="label">Last Price:</div> <div>${$.format(LastPrice,"N2")}</div> <div class="label">Net Change:</div><div> {{if NetChange > 0}} <b style="color:green" >${NetChange}</b> {{else}} <b style="color:red" >${NetChange}</b> {{/if}} </div> <div class="label">Last Update:</div> <div>${$.format(LastQuoteTime,"MMM dd, yyyy")}</div> </div> </script> There are also parsing methods that can parse dates and numbers from strings into numbers easily: alert($.parseDate("25.10.2010")); alert($.parseInt("12.222")); // de-DE uses . for thousands separators As you can see culture specific options are taken into account when parsing. The globalization plugin provides rich support for a variety of locales: Get a list of all available cultures Query cultures for culture items (like currency symbol, separators etc.) Localized string names for all calendar related items (days of week, months) Generated off of .NET’s supported locales In short you get much of the same functionality that you already might be using in .NET on the server side. The plugin includes a huge number of locales and an Globalization.all.min.js file that contains the text defaults for each of these locales as well as small locale specific script files that define each of the locale specific settings. It’s highly recommended that you NOT use the huge globalization file that includes all locales, but rather add script references to only those languages you explicitly care about. Overall this plug-in is a welcome helper. Even if you use it with a single locale (like en-US) and do no other localization, you’ll gain solid support for number and date formatting which is a vital feature of many applications. Changes for Microsoft It’s good to see Microsoft coming out of its shell and away from the ‘not-built-here’ mentality that has been so pervasive in the past. It’s especially good to see it applied to jQuery – a technology that has stood in drastic contrast to Microsoft’s own internal efforts in terms of design, usage model and… popularity. It’s great to see that Microsoft is paying attention to what customers prefer to use and supporting the customer sentiment – even if it meant drastically changing course of policy and moving into a more open and sharing environment in the process. The additional jQuery support that has been introduced in the last two years certainly has made lives easier for many developers on the ASP.NET platform. It’s also nice to see Microsoft submitting proposals through the standard jQuery process of plug-ins and getting accepted for various very useful projects. Certainly the jQuery Templates plug-in is going to be very useful to many especially since it will be baked into the jQuery core in jQuery 1.5. I hope we see more of this type of involvement from Microsoft in the future. Kudos!© Rick Strahl, West Wind Technologies, 2005-2010Posted in jQuery  ASP.NET  

    Read the article

  • Using FiddlerCore to capture HTTP Requests with .NET

    - by Rick Strahl
    Over the last few weeks I’ve been working on my Web load testing utility West Wind WebSurge. One of the key components of a load testing tool is the ability to capture URLs effectively so that you can play them back later under load. One of the options in WebSurge for capturing URLs is to use its built-in capture tool which acts as an HTTP proxy to capture any HTTP and HTTPS traffic from most Windows HTTP clients, including Web Browsers as well as standalone Windows applications and services. To make this happen, I used Eric Lawrence’s awesome FiddlerCore library, which provides most of the functionality of his desktop Fiddler application, all rolled into an easy to use library that you can plug into your own applications. FiddlerCore makes it almost too easy to capture HTTP content! For WebSurge I needed to capture all HTTP traffic in order to capture the full HTTP request – URL, headers and any content posted by the client. The result of what I ended up creating is this semi-generic capture form: In this post I’m going to demonstrate how easy it is to use FiddlerCore to build this HTTP Capture Form.  If you want to jump right in here are the links to get Telerik’s Fiddler Core and the code for the demo provided here. FiddlerCore Download FiddlerCore on NuGet Show me the Code (WebSurge Integration code from GitHub) Download the WinForms Sample Form West Wind Web Surge (example implementation in live app) Note that FiddlerCore is bound by a license for commercial usage – see license.txt in the FiddlerCore distribution for details. Integrating FiddlerCore FiddlerCore is a library that simply plugs into your application. You can download it from the Telerik site and manually add the assemblies to your project, or you can simply install the NuGet package via:       PM> Install-Package FiddlerCore The library consists of the FiddlerCore.dll as well as a couple of support libraries (CertMaker.dll and BCMakeCert.dll) that are used for installing SSL certificates. I’ll have more on SSL captures and certificate installation later in this post. But first let’s see how easy it is to use FiddlerCore to capture HTTP content by looking at how to build the above capture form. Capturing HTTP Content Once the library is installed it’s super easy to hook up Fiddler functionality. Fiddler includes a number of static class methods on the FiddlerApplication object that can be called to hook up callback events as well as actual start monitoring HTTP URLs. In the following code directly lifted from WebSurge, I configure a few filter options on Form level object, from the user inputs shown on the form by assigning it to a capture options object. In the live application these settings are persisted configuration values, but in the demo they are one time values initialized and set on the form. Once these options are set, I hook up the AfterSessionComplete event to capture every URL that passes through the proxy after the request is completed and start up the Proxy service:void Start() { if (tbIgnoreResources.Checked) CaptureConfiguration.IgnoreResources = true; else CaptureConfiguration.IgnoreResources = false; string strProcId = txtProcessId.Text; if (strProcId.Contains('-')) strProcId = strProcId.Substring(strProcId.IndexOf('-') + 1).Trim(); strProcId = strProcId.Trim(); int procId = 0; if (!string.IsNullOrEmpty(strProcId)) { if (!int.TryParse(strProcId, out procId)) procId = 0; } CaptureConfiguration.ProcessId = procId; CaptureConfiguration.CaptureDomain = txtCaptureDomain.Text; FiddlerApplication.AfterSessionComplete += FiddlerApplication_AfterSessionComplete; FiddlerApplication.Startup(8888, true, true, true); } The key lines for FiddlerCore are just the last two lines of code that include the event hookup code as well as the Startup() method call. Here I only hook up to the AfterSessionComplete event but there are a number of other events that hook various stages of the HTTP request cycle you can also hook into. Other events include BeforeRequest, BeforeResponse, RequestHeadersAvailable, ResponseHeadersAvailable and so on. In my case I want to capture the request data and I actually have several options to capture this data. AfterSessionComplete is the last event that fires in the request sequence and it’s the most common choice to capture all request and response data. I could have used several other events, but AfterSessionComplete is one place where you can look both at the request and response data, so this will be the most common place to hook into if you’re capturing content. The implementation of AfterSessionComplete is responsible for capturing all HTTP request headers and it looks something like this:private void FiddlerApplication_AfterSessionComplete(Session sess) { // Ignore HTTPS connect requests if (sess.RequestMethod == "CONNECT") return; if (CaptureConfiguration.ProcessId > 0) { if (sess.LocalProcessID != 0 && sess.LocalProcessID != CaptureConfiguration.ProcessId) return; } if (!string.IsNullOrEmpty(CaptureConfiguration.CaptureDomain)) { if (sess.hostname.ToLower() != CaptureConfiguration.CaptureDomain.Trim().ToLower()) return; } if (CaptureConfiguration.IgnoreResources) { string url = sess.fullUrl.ToLower(); var extensions = CaptureConfiguration.ExtensionFilterExclusions; foreach (var ext in extensions) { if (url.Contains(ext)) return; } var filters = CaptureConfiguration.UrlFilterExclusions; foreach (var urlFilter in filters) { if (url.Contains(urlFilter)) return; } } if (sess == null || sess.oRequest == null || sess.oRequest.headers == null) return; string headers = sess.oRequest.headers.ToString(); var reqBody = sess.GetRequestBodyAsString(); // if you wanted to capture the response //string respHeaders = session.oResponse.headers.ToString(); //var respBody = session.GetResponseBodyAsString(); // replace the HTTP line to inject full URL string firstLine = sess.RequestMethod + " " + sess.fullUrl + " " + sess.oRequest.headers.HTTPVersion; int at = headers.IndexOf("\r\n"); if (at < 0) return; headers = firstLine + "\r\n" + headers.Substring(at + 1); string output = headers + "\r\n" + (!string.IsNullOrEmpty(reqBody) ? reqBody + "\r\n" : string.Empty) + Separator + "\r\n\r\n"; BeginInvoke(new Action<string>((text) => { txtCapture.AppendText(text); UpdateButtonStatus(); }), output); } The code starts by filtering out some requests based on the CaptureOptions I set before the capture is started. These options/filters are applied when requests actually come in. This is very useful to help narrow down the requests that are captured for playback based on options the user picked. I find it useful to limit requests to a certain domain for captures, as well as filtering out some request types like static resources – images, css, scripts etc. This is of course optional, but I think it’s a common scenario and WebSurge makes good use of this feature. AfterSessionComplete like other FiddlerCore events, provides a Session object parameter which contains all the request and response details. There are oRequest and oResponse objects to hold their respective data. In my case I’m interested in the raw request headers and body only, as you can see in the commented code you can also retrieve the response headers and body. Here the code captures the request headers and body and simply appends the output to the textbox on the screen. Note that the Fiddler events are asynchronous, so in order to display the content in the UI they have to be marshaled back the UI thread with BeginInvoke, which here simply takes the generated headers and appends it to the existing textbox test on the form. As each request is processed, the headers are captured and appended to the bottom of the textbox resulting in a Session HTTP capture in the format that Web Surge internally supports, which is basically raw request headers with a customized 1st HTTP Header line that includes the full URL rather than a server relative URL. When the capture is done the user can either copy the raw HTTP session to the clipboard, or directly save it to file. This raw capture format is the same format WebSurge and also Fiddler use to import/export request data. While this code is application specific, it demonstrates the kind of logic that you can easily apply to the request capture process, which is one of the reasonsof why FiddlerCore is so powerful. You get to choose what content you want to look up as part of your own application logic and you can then decide how to capture or use that data as part of your application. The actual captured data in this case is only a string. The user can edit the data by hand or in the the case of WebSurge, save it to disk and automatically open the captured session as a new load test. Stopping the FiddlerCore Proxy Finally to stop capturing requests you simply disconnect the event handler and call the FiddlerApplication.ShutDown() method:void Stop() { FiddlerApplication.AfterSessionComplete -= FiddlerApplication_AfterSessionComplete; if (FiddlerApplication.IsStarted()) FiddlerApplication.Shutdown(); } As you can see, adding HTTP capture functionality to an application is very straight forward. FiddlerCore offers tons of features I’m not even touching on here – I suspect basic captures are the most common scenario, but a lot of different things can be done with FiddlerCore’s simple API interface. Sky’s the limit! The source code for this sample capture form (WinForms) is provided as part of this article. Adding Fiddler Certificates with FiddlerCore One of the sticking points in West Wind WebSurge has been that if you wanted to capture HTTPS/SSL traffic, you needed to have the full version of Fiddler and have HTTPS decryption enabled. Essentially you had to use Fiddler to configure HTTPS decryption and the associated installation of the Fiddler local client certificate that is used for local decryption of incoming SSL traffic. While this works just fine, requiring to have Fiddler installed and then using a separate application to configure the SSL functionality isn’t ideal. Fortunately FiddlerCore actually includes the tools to register the Fiddler Certificate directly using FiddlerCore. Why does Fiddler need a Certificate in the first Place? Fiddler and FiddlerCore are essentially HTTP proxies which means they inject themselves into the HTTP conversation by re-routing HTTP traffic to a special HTTP port (8888 by default for Fiddler) and then forward the HTTP data to the original client. Fiddler injects itself as the system proxy in using the WinInet Windows settings  which are the same settings that Internet Explorer uses and that are configured in the Windows and Internet Explorer Internet Settings dialog. Most HTTP clients running on Windows pick up and apply these system level Proxy settings before establishing new HTTP connections and that’s why most clients automatically work once Fiddler – or FiddlerCore/WebSurge are running. For plain HTTP requests this just works – Fiddler intercepts the HTTP requests on the proxy port and then forwards them to the original port (80 for HTTP and 443 for SSL typically but it could be any port). For SSL however, this is not quite as simple – Fiddler can easily act as an HTTPS/SSL client to capture inbound requests from the server, but when it forwards the request to the client it has to also act as an SSL server and provide a certificate that the client trusts. This won’t be the original certificate from the remote site, but rather a custom local certificate that effectively simulates an SSL connection between the proxy and the client. If there is no custom certificate configured for Fiddler the SSL request fails with a certificate validation error. The key for this to work is that a custom certificate has to be installed that the HTTPS client trusts on the local machine. For a much more detailed description of the process you can check out Eric Lawrence’s blog post on Certificates. If you’re using the desktop version of Fiddler you can install a local certificate into the Windows certificate store. Fiddler proper does this from the Options menu: This operation does several things: It installs the Fiddler Root Certificate It sets trust to this Root Certificate A new client certificate is generated for each HTTPS site monitored Certificate Installation with FiddlerCore You can also provide this same functionality using FiddlerCore which includes a CertMaker class. Using CertMaker is straight forward to use and it provides an easy way to create some simple helpers that can install and uninstall a Fiddler Root certificate:public static bool InstallCertificate() { if (!CertMaker.rootCertExists()) { if (!CertMaker.createRootCert()) return false; if (!CertMaker.trustRootCert()) return false; } return true; } public static bool UninstallCertificate() { if (CertMaker.rootCertExists()) { if (!CertMaker.removeFiddlerGeneratedCerts(true)) return false; } return true; } InstallCertificate() works by first checking whether the root certificate is already installed and if it isn’t goes ahead and creates a new one. The process of creating the certificate is a two step process – first the actual certificate is created and then it’s moved into the certificate store to become trusted. I’m not sure why you’d ever split these operations up since a cert created without trust isn’t going to be of much value, but there are two distinct steps. When you trigger the trustRootCert() method, a message box will pop up on the desktop that lets you know that you’re about to trust a local private certificate. This is a security feature to ensure that you really want to trust the Fiddler root since you are essentially installing a man in the middle certificate. It’s quite safe to use this generated root certificate, because it’s been specifically generated for your machine and thus is not usable from external sources, the only way to use this certificate in a trusted way is from the local machine. IOW, unless somebody has physical access to your machine, there’s no useful way to hijack this certificate and use it for nefarious purposes (see Eric’s post for more details). Once the Root certificate has been installed, FiddlerCore/Fiddler create new certificates for each site that is connected to with HTTPS. You can end up with quite a few temporary certificates in your certificate store. To uninstall you can either use Fiddler and simply uncheck the Decrypt HTTPS traffic option followed by the remove Fiddler certificates button, or you can use FiddlerCore’s CertMaker.removeFiddlerGeneratedCerts() which removes the root cert and any of the intermediary certificates Fiddler created. Keep in mind that when you uninstall you uninstall the certificate for both FiddlerCore and Fiddler, so use UninstallCertificate() with care and realize that you might affect the Fiddler application’s operation by doing so as well. When to check for an installed Certificate Note that the check to see if the root certificate exists is pretty fast, while the actual process of installing the certificate is a relatively slow operation that even on a fast machine takes a few seconds. Further the trust operation pops up a message box so you probably don’t want to install the certificate repeatedly. Since the check for the root certificate is fast, you can easily put a call to InstallCertificate() in any capture startup code – in which case the certificate installation only triggers when a certificate is in fact not installed. Personally I like to make certificate installation explicit – just like Fiddler does, so in WebSurge I use a small drop down option on the menu to install or uninstall the SSL certificate:   This code calls the InstallCertificate and UnInstallCertificate functions respectively – the experience with this is similar to what you get in Fiddler with the extra dialog box popping up to prompt confirmation for installation of the root certificate. Once the cert is installed you can then capture SSL requests. There’s a gotcha however… Gotcha: FiddlerCore Certificates don’t stick by Default When I originally tried to use the Fiddler certificate installation I ran into an odd problem. I was able to install the certificate and immediately after installation was able to capture HTTPS requests. Then I would exit the application and come back in and try the same HTTPS capture again and it would fail due to a missing certificate. CertMaker.rootCertExists() would return false after every restart and if re-installed the certificate a new certificate would get added to the certificate store resulting in a bunch of duplicated root certificates with different keys. What the heck? CertMaker and BcMakeCert create non-sticky CertificatesI turns out that FiddlerCore by default uses different components from what the full version of Fiddler uses. Fiddler uses a Windows utility called MakeCert.exe to create the Fiddler Root certificate. FiddlerCore however installs the CertMaker.dll and BCMakeCert.dll assemblies, which use a different crypto library (Bouncy Castle) for certificate creation than MakeCert.exe which uses the Windows Crypto API. The assemblies provide support for non-windows operation for Fiddler under Mono, as well as support for some non-Windows certificate platforms like iOS and Android for decryption. The bottom line is that the FiddlerCore provided bouncy castle assemblies are not sticky by default as the certificates created with them are not cached as they are in Fiddler proper. To get certificates to ‘stick’ you have to explicitly cache the certificates in Fiddler’s internal preferences. A cache aware version of InstallCertificate looks something like this:public static bool InstallCertificate() { if (!CertMaker.rootCertExists()) { if (!CertMaker.createRootCert()) return false; if (!CertMaker.trustRootCert()) return false; App.Configuration.UrlCapture.Cert = FiddlerApplication.Prefs.GetStringPref("fiddler.certmaker.bc.cert", null); App.Configuration.UrlCapture.Key = FiddlerApplication.Prefs.GetStringPref("fiddler.certmaker.bc.key", null); } return true; } public static bool UninstallCertificate() { if (CertMaker.rootCertExists()) { if (!CertMaker.removeFiddlerGeneratedCerts(true)) return false; } App.Configuration.UrlCapture.Cert = null; App.Configuration.UrlCapture.Key = null; return true; } In this code I store the Fiddler cert and private key in an application configuration settings that’s stored with the application settings (App.Configuration.UrlCapture object). These settings automatically persist when WebSurge is shut down. The values are read out of Fiddler’s internal preferences store which is set after a new certificate has been created. Likewise I clear out the configuration settings when the certificate is uninstalled. In order for these setting to be used you have to also load the configuration settings into the Fiddler preferences *before* a call to rootCertExists() is made. I do this in the capture form’s constructor:public FiddlerCapture(StressTestForm form) { InitializeComponent(); CaptureConfiguration = App.Configuration.UrlCapture; MainForm = form; if (!string.IsNullOrEmpty(App.Configuration.UrlCapture.Cert)) { FiddlerApplication.Prefs.SetStringPref("fiddler.certmaker.bc.key", App.Configuration.UrlCapture.Key); FiddlerApplication.Prefs.SetStringPref("fiddler.certmaker.bc.cert", App.Configuration.UrlCapture.Cert); }} This is kind of a drag to do and not documented anywhere that I could find, so hopefully this will save you some grief if you want to work with the stock certificate logic that installs with FiddlerCore. MakeCert provides sticky Certificates and the same functionality as Fiddler But there’s actually an easier way. If you want to skip the above Fiddler preference configuration code in your application you can choose to distribute MakeCert.exe instead of certmaker.dll and bcmakecert.dll. When you use MakeCert.exe, the certificates settings are stored in Windows so they are available without any custom configuration inside of your application. It’s easier to integrate and as long as you run on Windows and you don’t need to support iOS or Android devices is simply easier to deal with. To integrate into your project, you can remove the reference to CertMaker.dll (and the BcMakeCert.dll assembly) from your project. Instead copy MakeCert.exe into your output folder. To make sure MakeCert.exe gets pushed out, include MakeCert.exe in your project and set the Build Action to None, and Copy to Output Directory to Copy if newer. Note that the CertMaker.dll reference in the project has been removed and on disk the files for Certmaker.dll, as well as the BCMakeCert.dll files on disk. Keep in mind that these DLLs are resources of the FiddlerCore NuGet package, so updating the package may end up pushing those files back into your project. Once MakeCert.exe is distributed FiddlerCore checks for it first before using the assemblies so as long as MakeCert.exe exists it’ll be used for certificate creation (at least on Windows). Summary FiddlerCore is a pretty sweet tool, and it’s absolutely awesome that we get to plug in most of the functionality of Fiddler right into our own applications. A few years back I tried to build this sort of functionality myself for an app and ended up giving up because it’s a big job to get HTTP right – especially if you need to support SSL. FiddlerCore now provides that functionality as a turnkey solution that can be plugged into your own apps easily. The only downside is FiddlerCore’s documentation for more advanced features like certificate installation which is pretty sketchy. While for the most part FiddlerCore’s feature set is easy to work with without any documentation, advanced features are often not intuitive to gleam by just using Intellisense or the FiddlerCore help file reference (which is not terribly useful). While Eric Lawrence is very responsive on his forum and on Twitter, there simply isn’t much useful documentation on Fiddler/FiddlerCore available online. If you run into trouble the forum is probably the first place to look and then ask a question if you can’t find the answer. The best documentation you can find is Eric’s Fiddler Book which covers a ton of functionality of Fiddler and FiddlerCore. The book is a great reference to Fiddler’s feature set as well as providing great insights into the HTTP protocol. The second half of the book that gets into the innards of HTTP is an excellent read for anybody who wants to know more about some of the more arcane aspects and special behaviors of HTTP – it’s well worth the read. While the book has tons of information in a very readable format, it’s unfortunately not a great reference as it’s hard to find things in the book and because it’s not available online you can’t electronically search for the great content in it. But it’s hard to complain about any of this given the obvious effort and love that’s gone into this awesome product for all of these years. A mighty big thanks to Eric Lawrence  for having created this useful tool that so many of us use all the time, and also to Telerik for picking up Fiddler/FiddlerCore and providing Eric the resources to support and improve this wonderful tool full time and keeping it free for all. Kudos! Resources FiddlerCore Download FiddlerCore NuGet Fiddler Capture Sample Form Fiddler Capture Form in West Wind WebSurge (GitHub) Eric Lawrence’s Fiddler Book© Rick Strahl, West Wind Technologies, 2005-2014Posted in .NET  HTTP   Tweet !function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0];if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src="//platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs"); (function() { var po = document.createElement('script'); po.type = 'text/javascript'; po.async = true; po.src = 'https://apis.google.com/js/plusone.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s); })();

    Read the article

  • .NET HTML Sanitation for rich HTML Input

    - by Rick Strahl
    Recently I was working on updating a legacy application to MVC 4 that included free form text input. When I set up the new site my initial approach was to not allow any rich HTML input, only simple text formatting that would respect a few simple HTML commands for bold, lists etc. and automatically handles line break processing for new lines and paragraphs. This is typical for what I do with most multi-line text input in my apps and it works very well with very little development effort involved. Then the client sprung another note: Oh by the way we have a bunch of customers (real estate agents) who need to post complete HTML documents. Oh uh! There goes the simple theory. After some discussion and pleading on my part (<snicker>) to try and avoid this type of raw HTML input because of potential XSS issues, the client decided to go ahead and allow raw HTML input anyway. There has been lots of discussions on this subject on StackOverFlow (and here and here) but to after reading through some of the solutions I didn't really find anything that would work even closely for what I needed. Specifically we need to be able to allow just about any HTML markup, with the exception of script code. Remote CSS and Images need to be loaded, links need to work and so. While the 'legit' HTML posted by these agents is basic in nature it does span most of the full gamut of HTML (4). Most of the solutions XSS prevention/sanitizer solutions I found were way to aggressive and rendered the posted output unusable mostly because they tend to strip any externally loaded content. In short I needed a custom solution. I thought the best solution to this would be to use an HTML parser - in this case the Html Agility Pack - and then to run through all the HTML markup provided and remove any of the blacklisted tags and a number of attributes that are prone to JavaScript injection. There's much discussion on whether to use blacklists vs. whitelists in the discussions mentioned above, but I found that whitelists can make sense in simple scenarios where you might allow manual HTML input, but when you need to allow a larger array of HTML functionality a blacklist is probably easier to manage as the vast majority of elements and attributes could be allowed. Also white listing gets a bit more complex with HTML5 and the new proliferation of new HTML tags and most new tags generally don't affect XSS issues directly. Pure whitelisting based on elements and attributes also doesn't capture many edge cases (see some of the XSS cheat sheets listed below) so even with a white list, custom logic is still required to handle many of those edge cases. The Microsoft Web Protection Library (AntiXSS) My first thought was to check out the Microsoft AntiXSS library. Microsoft has an HTML Encoding and Sanitation library in the Microsoft Web Protection Library (formerly AntiXSS Library) on CodePlex, which provides stricter functions for whitelist encoding and sanitation. Initially I thought the Sanitation class and its static members would do the trick for me,but I found that this library is way too restrictive for my needs. Specifically the Sanitation class strips out images and links which rendered the full HTML from our real estate clients completely useless. I didn't spend much time with it, but apparently I'm not alone if feeling this library is not really useful without some way to configure operation. To give you an example of what didn't work for me with the library here's a small and simple HTML fragment that includes script, img and anchor tags. I would expect the script to be stripped and everything else to be left intact. Here's the original HTML:var value = "<b>Here</b> <script>alert('hello')</script> we go. Visit the " + "<a href='http://west-wind.com'>West Wind</a> site. " + "<img src='http://west-wind.com/images/new.gif' /> " ; and the code to sanitize it with the AntiXSS Sanitize class:@Html.Raw(Microsoft.Security.Application.Sanitizer.GetSafeHtmlFragment(value)) This produced a not so useful sanitized string: Here we go. Visit the <a>West Wind</a> site. While it removed the <script> tag (good) it also removed the href from the link and the image tag altogether (bad). In some situations this might be useful, but for most tasks I doubt this is the desired behavior. While links can contain javascript: references and images can 'broadcast' information to a server, without configuration to tell the library what to restrict this becomes useless to me. I couldn't find any way to customize the white list, nor is there code available in this 'open source' library on CodePlex. Using Html Agility Pack for HTML Parsing The WPL library wasn't going to cut it. After doing a bit of research I decided the best approach for a custom solution would be to use an HTML parser and inspect the HTML fragment/document I'm trying to import. I've used the HTML Agility Pack before for a number of apps where I needed an HTML parser without requiring an instance of a full browser like the Internet Explorer Application object which is inadequate in Web apps. In case you haven't checked out the Html Agility Pack before, it's a powerful HTML parser library that you can use from your .NET code. It provides a simple, parsable HTML DOM model to full HTML documents or HTML fragments that let you walk through each of the elements in your document. If you've used the HTML or XML DOM in a browser before you'll feel right at home with the Agility Pack. Blacklist based HTML Parsing to strip XSS Code For my purposes of HTML sanitation, the process involved is to walk the HTML document one element at a time and then check each element and attribute against a blacklist. There's quite a bit of argument of what's better: A whitelist of allowed items or a blacklist of denied items. While whitelists tend to be more secure, they also require a lot more configuration. In the case of HTML5 a whitelist could be very extensive. For what I need, I only want to ensure that no JavaScript is executed, so a blacklist includes the obvious <script> tag plus any tag that allows loading of external content including <iframe>, <object>, <embed> and <link> etc. <form>  is also excluded to avoid posting content to a different location. I also disallow <head> and <meta> tags in particular for my case, since I'm only allowing posting of HTML fragments. There is also some internal logic to exclude some attributes or attributes that include references to JavaScript or CSS expressions. The default tag blacklist reflects my use case, but is customizable and can be added to. Here's my HtmlSanitizer implementation:using System.Collections.Generic; using System.IO; using System.Xml; using HtmlAgilityPack; namespace Westwind.Web.Utilities { public class HtmlSanitizer { public HashSet<string> BlackList = new HashSet<string>() { { "script" }, { "iframe" }, { "form" }, { "object" }, { "embed" }, { "link" }, { "head" }, { "meta" } }; /// <summary> /// Cleans up an HTML string and removes HTML tags in blacklist /// </summary> /// <param name="html"></param> /// <returns></returns> public static string SanitizeHtml(string html, params string[] blackList) { var sanitizer = new HtmlSanitizer(); if (blackList != null && blackList.Length > 0) { sanitizer.BlackList.Clear(); foreach (string item in blackList) sanitizer.BlackList.Add(item); } return sanitizer.Sanitize(html); } /// <summary> /// Cleans up an HTML string by removing elements /// on the blacklist and all elements that start /// with onXXX . /// </summary> /// <param name="html"></param> /// <returns></returns> public string Sanitize(string html) { var doc = new HtmlDocument(); doc.LoadHtml(html); SanitizeHtmlNode(doc.DocumentNode); //return doc.DocumentNode.WriteTo(); string output = null; // Use an XmlTextWriter to create self-closing tags using (StringWriter sw = new StringWriter()) { XmlWriter writer = new XmlTextWriter(sw); doc.DocumentNode.WriteTo(writer); output = sw.ToString(); // strip off XML doc header if (!string.IsNullOrEmpty(output)) { int at = output.IndexOf("?>"); output = output.Substring(at + 2); } writer.Close(); } doc = null; return output; } private void SanitizeHtmlNode(HtmlNode node) { if (node.NodeType == HtmlNodeType.Element) { // check for blacklist items and remove if (BlackList.Contains(node.Name)) { node.Remove(); return; } // remove CSS Expressions and embedded script links if (node.Name == "style") { if (string.IsNullOrEmpty(node.InnerText)) { if (node.InnerHtml.Contains("expression") || node.InnerHtml.Contains("javascript:")) node.ParentNode.RemoveChild(node); } } // remove script attributes if (node.HasAttributes) { for (int i = node.Attributes.Count - 1; i >= 0; i--) { HtmlAttribute currentAttribute = node.Attributes[i]; var attr = currentAttribute.Name.ToLower(); var val = currentAttribute.Value.ToLower(); span style="background: white; color: green">// remove event handlers if (attr.StartsWith("on")) node.Attributes.Remove(currentAttribute); // remove script links else if ( //(attr == "href" || attr== "src" || attr == "dynsrc" || attr == "lowsrc") && val != null && val.Contains("javascript:")) node.Attributes.Remove(currentAttribute); // Remove CSS Expressions else if (attr == "style" && val != null && val.Contains("expression") || val.Contains("javascript:") || val.Contains("vbscript:")) node.Attributes.Remove(currentAttribute); } } } // Look through child nodes recursively if (node.HasChildNodes) { for (int i = node.ChildNodes.Count - 1; i >= 0; i--) { SanitizeHtmlNode(node.ChildNodes[i]); } } } } } Please note: Use this as a starting point only for your own parsing and review the code for your specific use case! If your needs are less lenient than mine were you can you can make this much stricter by not allowing src and href attributes or CSS links if your HTML doesn't allow it. You can also check links for external URLs and disallow those - lots of options.  The code is simple enough to make it easy to extend to fit your use cases more specifically. It's also quite easy to make this code work using a WhiteList approach if you want to go that route. The code above is semi-generic for allowing full featured HTML fragments that only disallow script related content. The Sanitize method walks through each node of the document and then recursively drills into all of its children until the entire document has been traversed. Note that the code here uses an XmlTextWriter to write output - this is done to preserve XHTML style self-closing tags which are otherwise left as non-self-closing tags. The sanitizer code scans for blacklist elements and removes those elements not allowed. Note that the blacklist is configurable either in the instance class as a property or in the static method via the string parameter list. Additionally the code goes through each element's attributes and looks for a host of rules gleaned from some of the XSS cheat sheets listed at the end of the post. Clearly there are a lot more XSS vulnerabilities, but a lot of them apply to ancient browsers (IE6 and versions of Netscape) - many of these glaring holes (like CSS expressions - WTF IE?) have been removed in modern browsers. What a Pain To be honest this is NOT a piece of code that I wanted to write. I think building anything related to XSS is better left to people who have far more knowledge of the topic than I do. Unfortunately, I was unable to find a tool that worked even closely for me, or even provided a working base. For the project I was working on I had no choice and I'm sharing the code here merely as a base line to start with and potentially expand on for specific needs. It's sad that Microsoft Web Protection Library is currently such a train wreck - this is really something that should come from Microsoft as the systems vendor or possibly a third party that provides security tools. Luckily for my application we are dealing with a authenticated and validated users so the user base is fairly well known, and relatively small - this is not a wide open Internet application that's directly public facing. As I mentioned earlier in the post, if I had my way I would simply not allow this type of raw HTML input in the first place, and instead rely on a more controlled HTML input mechanism like MarkDown or even a good HTML Edit control that can provide some limits on what types of input are allowed. Alas in this case I was overridden and we had to go forward and allow *any* raw HTML posted. Sometimes I really feel sad that it's come this far - how many good applications and tools have been thwarted by fear of XSS (or worse) attacks? So many things that could be done *if* we had a more secure browser experience and didn't have to deal with every little script twerp trying to hack into Web pages and obscure browser bugs. So much time wasted building secure apps, so much time wasted by others trying to hack apps… We're a funny species - no other species manages to waste as much time, effort and resources as we humans do :-) Resources Code on GitHub Html Agility Pack XSS Cheat Sheet XSS Prevention Cheat Sheet Microsoft Web Protection Library (AntiXss) StackOverflow Links: http://stackoverflow.com/questions/341872/html-sanitizer-for-net http://blog.stackoverflow.com/2008/06/safe-html-and-xss/ http://code.google.com/p/subsonicforums/source/browse/trunk/SubSonic.Forums.Data/HtmlScrubber.cs?r=61© Rick Strahl, West Wind Technologies, 2005-2012Posted in Security  HTML  ASP.NET  JavaScript   Tweet !function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0];if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src="//platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs"); (function() { var po = document.createElement('script'); po.type = 'text/javascript'; po.async = true; po.src = 'https://apis.google.com/js/plusone.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s); })();

    Read the article

< Previous Page | 343 344 345 346 347 348 349 350 351 352 353 354  | Next Page >