leftovers - Page 2 - Developer IT

How to select the first property with unknown name and first item from array in JSON

- by Oscar Godson

I actually have two questions, both are probably simple, but for some odd reason I cant figure it out... I've worked with JSON 100s of times before too! but here is the JSON in question: {"69256":{ "streaminfo":{ "stream_ID":"1025", "sourceowner_ID":"2", "sourceowner_avatar":"http:\/\/content.nozzlmedia.com\/images\/sourceowner_avatar2.jpg", "sourceownertype_ID":"1", "stream_name":"Twitter", "streamtype":"Social media" "appsarray":[] }, "item":{ "headline":"Charboy", "main_image":"http:\/\/content.nozzlmedia.com\/images\/author_avatar173212.jpg", "summary":"ate a tomato and avocado for dinner...", "nozzl_captured":"2010-05-12 23:02:12", "geoarray":[{ "state":"OR", "county":"Multnomah", "city":"Portland", "neighborhood":"Downtown", "zip":"97205", "street":"462 SW 11th Ave", "latitude":"45.5219", "longitude":"-122.682" }], "full_content":"ate a tomato and avocado for dinner tonight. such tasty foods. just enjoyable.", "body_text":"ate a tomato and avocado for dinner tonight. such tasty foods. just enjoyable.", "author_name":"Charboy", "author_avatar":"http:\/\/content.nozzlmedia.com\/images\/author_avatar173212.jpg", "fulltext_url":"http:\/\/twitter.com\/charboy\/statuses\/13889868936", "leftovers":{ "twitter_id":"tag:search.twitter.com,2005:13889868936", "date":"2010-05-13T02:59:59Z", "location":"iPhone: 45.521866,-122.682262" }, "wordarray":{ "0":"ate", "1":"tomato", "2":"avocado", "3":"dinner", "4":"tonight", "5":"tasty", "6":"foods", "7":"just", "8":"enjoyable", "9":"Charboy", "11":"Twitter", "13":"state:OR", "14":"county:Multnomah, OR", "15":"city:Portland, OR", "16":"neighborhood:Downtown", "17":"zip:97205" } } } } Question 1: How do I loop through each item (69256) when the number is random? e.g. item 1 is 123, item2 is 646? Like, for example, a normal JSON feed would have something like: {'item':{'blah':'lorem'},'item':{'blah':'ipsum'}} the JS would be like console.log(item.blah) to return lorem then ipsum in a loop How do I do it when i dont know the first item of the object? Question 2: How do I select items from the geoarray object? I tried: json.test.item.geoarray.latitude and json.test.item.geoarray['latitude']

Read the article

Silverlight Socket Constantly Returns With Empty Buffer

- by Benny

I am using Silverlight to interact with a proxy application that I have developed but, without the proxy sending a message to the Silverlight application, it executes the receive completed handler with an empty buffer ('\0's). Is there something I'm doing wrong? It is causing a major memory leak. this._rawBuffer = new Byte[this.BUFFER_SIZE]; SocketAsyncEventArgs receiveArgs = new SocketAsyncEventArgs(); receiveArgs.SetBuffer(_rawBuffer, 0, _rawBuffer.Length); receiveArgs.Completed += new EventHandler<SocketAsyncEventArgs>(ReceiveComplete); this._client.ReceiveAsync(receiveArgs); if (args.SocketError == SocketError.Success && args.LastOperation == SocketAsyncOperation.Receive) { // Read the current bytes from the stream buffer int bytesRecieved = this._client.ReceiveBufferSize; // If there are bytes to process else the connection is lost if (bytesRecieved > 0) { try { //Find out what we just received string messagePart = UTF8Encoding.UTF8.GetString(_rawBuffer, 0, _rawBuffer.GetLength(0)); //Take out any trailing empty characters from the message messagePart = messagePart.Replace('\0'.ToString(), ""); //Concatenate our current message with any leftovers from previous receipts string fullMessage = _theRest + messagePart; int seperator; //While the index of the seperator (LINE_END defined & initiated as private member) while ((seperator = fullMessage.IndexOf((char)Messages.MessageSeperator.Terminator)) > 0) { //Pull out the first message available (up to the seperator index string message = fullMessage.Substring(0, seperator); //Queue up our new message _messageQueue.Enqueue(message); //Take out our line end character fullMessage = fullMessage.Remove(0, seperator + 1); } //Save whatever was NOT a full message to the private variable used to store the rest _theRest = fullMessage; //Empty the queue of messages if there are any while (this._messageQueue.Count > 0) { ... } } catch (Exception e) { throw e; } // Wait for a new message if (this._isClosing != true) Receive(); } } Thanks in advance.

Read the article

How to select this with JSON...

- by Oscar Godson

I actually have two questions, both are probably simple, but for some odd reason I cant figure it out... I've worked with JSON 100s of times before too! but here is the JSON in question: {"69256":{"streaminfo":{"stream_ID":"1025","sourceowner_ID":"2","sourceowner_avatar":"http:\/\/content.nozzlmedia.com\/images\/sourceowner_avatar2.jpg","sourceownertype_ID":"1","stream_name":"Twitter","streamtype":"Social media","appsarray":[]},"item":{"headline":"Charboy","main_image":"http:\/\/content.nozzlmedia.com\/images\/author_avatar173212.jpg","summary":"ate a tomato and avocado for dinner...","nozzl_captured":"2010-05-12 23:02:12","geoarray":[{"state":"OR","county":"Multnomah","city":"Portland","neighborhood":"Downtown","zip":"97205","street":"462 SW 11th Ave","latitude":"45.5219","longitude":"-122.682"}],"full_content":"ate a tomato and avocado for dinner tonight. such tasty foods. just enjoyable.","body_text":"ate a tomato and avocado for dinner tonight. such tasty foods. just enjoyable.","author_name":"Charboy","author_avatar":"http:\/\/content.nozzlmedia.com\/images\/author_avatar173212.jpg","fulltext_url":"http:\/\/twitter.com\/charboy\/statuses\/13889868936","leftovers":{"twitter_id":"tag:search.twitter.com,2005:13889868936","date":"2010-05-13T02:59:59Z","location":"iPhone: 45.521866,-122.682262"},"wordarray":{"0":"ate","1":"tomato","2":"avocado","3":"dinner","4":"tonight","5":"tasty","6":"foods","7":"just","8":"enjoyable","9":"Charboy","11":"Twitter","13":"state:OR","14":"county:Multnomah, OR","15":"city:Portland, OR","16":"neighborhood:Downtown","17":"zip:97205"}}}} Question 1: How do I loop through each item (69256) when the number is random? e.g. item 1 is 123, item2 is 646? Like, for example, a normal JSON feed would have something like: {'item':{'blah':'lorem'},'item':{'blah':'ipsum'}} the JS would be like console.log(item.blah) to return lorem then ipsum in a loop How do I do it when i dont know the first item of the object? Question 2: How do I select items from the geoarray object? I tried: json.test.item.geoarray.latitude and json.test.item.geoarray['latitude']

Read the article

How to select the first property with unknown name from JSON and how to select first item from array

- by Oscar Godson

I actually have two questions, both are probably simple, but for some odd reason I cant figure it out... I've worked with JSON 100s of times before too! but here is the JSON in question: {"69256":{"streaminfo":{"stream_ID":"1025","sourceowner_ID":"2","sourceowner_avatar":"http:\/\/content.nozzlmedia.com\/images\/sourceowner_avatar2.jpg","sourceownertype_ID":"1","stream_name":"Twitter","streamtype":"Social media","appsarray":[]},"item":{"headline":"Charboy","main_image":"http:\/\/content.nozzlmedia.com\/images\/author_avatar173212.jpg","summary":"ate a tomato and avocado for dinner...","nozzl_captured":"2010-05-12 23:02:12","geoarray":[{"state":"OR","county":"Multnomah","city":"Portland","neighborhood":"Downtown","zip":"97205","street":"462 SW 11th Ave","latitude":"45.5219","longitude":"-122.682"}],"full_content":"ate a tomato and avocado for dinner tonight. such tasty foods. just enjoyable.","body_text":"ate a tomato and avocado for dinner tonight. such tasty foods. just enjoyable.","author_name":"Charboy","author_avatar":"http:\/\/content.nozzlmedia.com\/images\/author_avatar173212.jpg","fulltext_url":"http:\/\/twitter.com\/charboy\/statuses\/13889868936","leftovers":{"twitter_id":"tag:search.twitter.com,2005:13889868936","date":"2010-05-13T02:59:59Z","location":"iPhone: 45.521866,-122.682262"},"wordarray":{"0":"ate","1":"tomato","2":"avocado","3":"dinner","4":"tonight","5":"tasty","6":"foods","7":"just","8":"enjoyable","9":"Charboy","11":"Twitter","13":"state:OR","14":"county:Multnomah, OR","15":"city:Portland, OR","16":"neighborhood:Downtown","17":"zip:97205"}}}} Question 1: How do I loop through each item (69256) when the number is random? e.g. item 1 is 123, item2 is 646? Like, for example, a normal JSON feed would have something like: {'item':{'blah':'lorem'},'item':{'blah':'ipsum'}} the JS would be like console.log(item.blah) to return lorem then ipsum in a loop How do I do it when i dont know the first item of the object? Question 2: How do I select items from the geoarray object? I tried: json.test.item.geoarray.latitude and json.test.item.geoarray['latitude']

Read the article

Partition Wise Joins

- by jean-pierre.dijcks

Some say they are the holy grail of parallel computing and PWJ is the basis for a shared nothing system and the only join method that is available on a shared nothing system (yes this is oversimplified!). The magic in Oracle is of course that is one of many ways to join data. And yes, this is the old flexibility vs. simplicity discussion all over, so I won't go there... the point is that what you must do in a shared nothing system, you can do in Oracle with the same speed and methods. The Theory A partition wise join is a join between (for simplicity) two tables that are partitioned on the same column with the same partitioning scheme. In shared nothing this is effectively hard partitioning locating data on a specific node / storage combo. In Oracle is is logical partitioning. If you now join the two tables on that partitioned column you can break up the join in smaller joins exactly along the partitions in the data. Since they are partitioned (grouped) into the same buckets, all values required to do the join live in the equivalent bucket on either sides. No need to talk to anyone else, no need to redistribute data to anyone else... in short, the optimal join method for parallel processing of two large data sets. PWJ's in Oracle Since we do not hard partition the data across nodes in Oracle we use the Partitioning option to the database to create the buckets, then set the Degree of Parallelism (or run Auto DOP - see here) and get our PWJs. The main questions always asked are: How many partitions should I create? What should my DOP be? In a shared nothing system the answer is of course, as many partitions as there are nodes which will be your DOP. In Oracle we do want you to look at the workload and concurrency, and once you know that to understand the following rules of thumb. Within Oracle we have more ways of joining of data, so it is important to understand some of the PWJ ideas and what it means if you have an uneven distribution across processes. Assume we have a simple scenario where we partition the data on a hash key resulting in 4 hash partitions (H1 -H4). We have 2 parallel processes that have been tasked with reading these partitions (P1 - P2). The work is evenly divided assuming the partitions are the same size and we can scan this in time t1 as shown below. Now assume that we have changed the system and have a 5th partition but still have our 2 workers P1 and P2. The time it takes is actually 50% more assuming the 5th partition has the same size as the original H1 - H4 partitions. In other words to scan these 5 partitions, the time t2 it takes is not 1/5th more expensive, it is a lot more expensive and some other join plans may now start to look exciting to the optimizer. Just to post the disclaimer, it is not as simple as I state it here, but you get the idea on how much more expensive this plan may now look... Based on this little example there are a few rules of thumb to follow to get the partition wise joins. First, choose a DOP that is a factor of two (2). So always choose something like 2, 4, 8, 16, 32 and so on... Second, choose a number of partitions that is larger or equal to 2* DOP. Third, make sure the number of partitions is divisible through 2 without orphans. This is also known as an even number... Fourth, choose a stable partition count strategy, which is typically hash, which can be a sub partitioning strategy rather than the main strategy (range - hash is a popular one). Fifth, make sure you do this on the join key between the two large tables you want to join (and this should be the obvious one...). Translating this into an example: DOP = 8 (determined based on concurrency or by using Auto DOP with a cap due to concurrency) says that the number of partitions >= 16. Number of hash (sub) partitions = 32, which gives each process four partitions to work on. This number is somewhat arbitrary and depends on your data and system. In this case my main reasoning is that if you get more room on the box you can easily move the DOP for the query to 16 without repartitioning... and of course it makes for no leftovers on the table... And yes, we recommend up-to-date statistics. And before you start complaining, do read this post on a cool way to do stats in 11.

Read the article

C#: Why Decorate When You Can Intercept

- by James Michael Hare

We've all heard of the old Decorator Design Pattern (here) or used it at one time or another either directly or indirectly. A decorator is a class that wraps a given abstract class or interface and presents the same (or a superset) public interface but "decorated" with additional functionality. As a really simplistic example, consider the System.IO.BufferedStream, it itself is a descendent of System.IO.Stream and wraps the given stream with buffering logic while still presenting System.IO.Stream's public interface: 1: Stream buffStream = new BufferedStream(rawStream); Now, let's take a look at a custom-code example. Let's say that we have a class in our data access layer that retrieves a list of products from a database: 1: // a class that handles our CRUD operations for products 2: public class ProductDao 3: { 4: ... 5: 6: // a method that would retrieve all available products 7: public IEnumerable<Product> GetAvailableProducts() 8: { 9: var results = new List<Product>(); 10: 11: // must create the connection 12: using (var con = _factory.CreateConnection()) 13: { 14: con.ConnectionString = _productsConnectionString; 15: con.Open(); 16: 17: // create the command 18: using (var cmd = _factory.CreateCommand()) 19: { 20: cmd.Connection = con; 21: cmd.CommandText = _getAllProductsStoredProc; 22: cmd.CommandType = CommandType.StoredProcedure; 23: 24: // get a reader and pass back all results 25: using (var reader = cmd.ExecuteReader()) 26: { 27: while(reader.Read()) 28: { 29: results.Add(new Product 30: { 31: Name = reader["product_name"].ToString(), 32: ... 33: }); 34: } 35: } 36: } 37: } 38: 39: return results; 40: } 41: } Yes, you could use EF or any myriad other choices for this sort of thing, but the germaine point is that you have some operation that takes a non-trivial amount of time. What if, during the production day I notice that my application is performing slowly and I want to see how much of that slowness is in the query versus my code. Well, I could easily wrap the logic block in a System.Diagnostics.Stopwatch and log the results to log4net or other logging flavor of choice: 1: // a class that handles our CRUD operations for products 2: public class ProductDao 3: { 4: private static readonly ILog _log = LogManager.GetLogger(typeof(ProductDao)); 5: ... 6: 7: // a method that would retrieve all available products 8: public IEnumerable<Product> GetAvailableProducts() 9: { 10: var results = new List<Product>(); 11: var timer = Stopwatch.StartNew(); 12: 13: // must create the connection 14: using (var con = _factory.CreateConnection()) 15: { 16: con.ConnectionString = _productsConnectionString; 17: 18: // and all that other DB code... 19: ... 20: } 21: 22: timer.Stop(); 23: 24: if (timer.ElapsedMilliseconds > 5000) 25: { 26: _log.WarnFormat("Long query in GetAvailableProducts() took {0} ms", 27: timer.ElapsedMillseconds); 28: } 29: 30: return results; 31: } 32: } In my eye, this is very ugly. It violates Single Responsibility Principle (SRP), which says that a class should only ever have one responsibility, where responsibility is often defined as a reason to change. This class (and in particular this method) has two reasons to change: If the method of retrieving products changes. If the method of logging changes. Well, we could “simplify” this using the Decorator Design Pattern (here). If we followed the pattern to the letter, we'd need to create a base decorator that implements the DAOs public interface and forwards to the wrapped instance. So let's assume we break out the ProductDAO interface into IProductDAO using your refactoring tool of choice (Resharper is great for this). Now, ProductDao will implement IProductDao and get rid of all logging logic: 1: public class ProductDao : IProductDao 2: { 3: // this reverts back to original version except for the interface added 4: } 5: And we create the base Decorator that also implements the interface and forwards all calls: 1: public class ProductDaoDecorator : IProductDao 2: { 3: private readonly IProductDao _wrappedDao; 4: 5: // constructor takes the dao to wrap 6: public ProductDaoDecorator(IProductDao wrappedDao) 7: { 8: _wrappedDao = wrappedDao; 9: } 10: 11: ... 12: 13: // and then all methods just forward their calls 14: public IEnumerable<Product> GetAvailableProducts() 15: { 16: return _wrappedDao.GetAvailableProducts(); 17: } 18: } This defines our base decorator, then we can create decorators that add items of interest, and for any methods we don't decorate, we'll get the default behavior which just forwards the call to the wrapper in the base decorator: 1: public class TimedThresholdProductDaoDecorator : ProductDaoDecorator 2: { 3: private static readonly ILog _log = LogManager.GetLogger(typeof(TimedThresholdProductDaoDecorator)); 4: 5: public TimedThresholdProductDaoDecorator(IProductDao wrappedDao) : 6: base(wrappedDao) 7: { 8: } 9: 10: ... 11: 12: public IEnumerable<Product> GetAvailableProducts() 13: { 14: var timer = Stopwatch.StartNew(); 15: 16: var results = _wrapped.GetAvailableProducts(); 17: 18: timer.Stop(); 19: 20: if (timer.ElapsedMilliseconds > 5000) 21: { 22: _log.WarnFormat("Long query in GetAvailableProducts() took {0} ms", 23: timer.ElapsedMillseconds); 24: } 25: 26: return results; 27: } 28: } Well, it's a bit better. Now the logging is in its own class, and the database logic is in its own class. But we've essentially multiplied the number of classes. We now have 3 classes and one interface! Now if you want to do that same logging decorating on all your DAOs, imagine the code bloat! Sure, you can simplify and avoid creating the base decorator, or chuck it all and just inherit directly. But regardless all of these have the problem of tying the logging logic into the code itself. Enter the Interceptors. Things like this to me are a perfect example of when it's good to write an Interceptor using your class library of choice. Sure, you could design your own perfectly generic decorator with delegates and all that, but personally I'm a big fan of Castle's Dynamic Proxy (here) which is actually used by many projects including Moq. What DynamicProxy allows you to do is intercept calls into any object by wrapping it with a proxy on the fly that intercepts the method and allows you to add functionality. Essentially, the code would now look like this using DynamicProxy: 1: // Note: I like hiding DynamicProxy behind the scenes so users 2: // don't have to explicitly add reference to Castle's libraries. 3: public static class TimeThresholdInterceptor 4: { 5: // Our logging handle 6: private static readonly ILog _log = LogManager.GetLogger(typeof(TimeThresholdInterceptor)); 7: 8: // Handle to Castle's proxy generator 9: private static readonly ProxyGenerator _generator = new ProxyGenerator(); 10: 11: // generic form for those who prefer it 12: public static object Create<TInterface>(object target, TimeSpan threshold) 13: { 14: return Create(typeof(TInterface), target, threshold); 15: } 16: 17: // Form that uses type instead 18: public static object Create(Type interfaceType, object target, TimeSpan threshold) 19: { 20: return _generator.CreateInterfaceProxyWithTarget(interfaceType, target, 21: new TimedThreshold(threshold, level)); 22: } 23: 24: // The interceptor that is created to intercept the interface calls. 25: // Hidden as a private inner class so not exposing Castle libraries. 26: private class TimedThreshold : IInterceptor 27: { 28: // The threshold as a positive timespan that triggers a log message. 29: private readonly TimeSpan _threshold; 30: 31: // interceptor constructor 32: public TimedThreshold(TimeSpan threshold) 33: { 34: _threshold = threshold; 35: } 36: 37: // Intercept functor for each method invokation 38: public void Intercept(IInvocation invocation) 39: { 40: // time the method invocation 41: var timer = Stopwatch.StartNew(); 42: 43: // the Castle magic that tells the method to go ahead 44: invocation.Proceed(); 45: 46: timer.Stop(); 47: 48: // check if threshold is exceeded 49: if (timer.Elapsed > _threshold) 50: { 51: _log.WarnFormat("Long execution in {0} took {1} ms", 52: invocation.Method.Name, 53: timer.ElapsedMillseconds); 54: } 55: } 56: } 57: } Yes, it's a bit longer, but notice that: This class ONLY deals with logging long method calls, no DAO interface leftovers. This class can be used to time ANY class that has an interface or virtual methods. Personally, I like to wrap and hide the usage of DynamicProxy and IInterceptor so that anyone who uses this class doesn't need to know to add a Castle library reference. As far as they are concerned, they're using my interceptor. If I change to a new library if a better one comes along, they're insulated. Now, all we have to do to use this is to tell it to wrap our ProductDao and it does the rest: 1: // wraps a new ProductDao with a timing interceptor with a threshold of 5 seconds 2: IProductDao dao = TimeThresholdInterceptor.Create<IProductDao>(new ProductDao(), 5000); Automatic decoration of all methods! You can even refine the proxy so that it only intercepts certain methods. This is ideal for so many things. These are just some of the interceptors we've dreamed up and use: Log parameters and returns of methods to XML for auditing. Block invocations to methods and return default value (stubbing). Throw exception if certain methods are called (good for blocking access to deprecated methods). Log entrance and exit of a method and the duration. Log a message if a method takes more than a given time threshold to execute. Whether you use DynamicProxy or some other technology, I hope you see the benefits this adds. Does it completely eliminate all need for the Decorator pattern? No, there may still be cases where you want to decorate a particular class with functionality that doesn't apply to the world at large. But for all those cases where you are using Decorator to add functionality that's truly generic. I strongly suggest you give this a try!

Developer IT