Search Results

Search found 2136 results on 86 pages for 'dominik str'.

Page 85/86 | < Previous Page | 81 82 83 84 85 86  | Next Page >

• Parallelism in .NET – Part 9, Configuration in PLINQ and TPL

  - by Reed

  Parallel LINQ and the Task Parallel Library contain many options for configuration.  Although the default configuration options are often ideal, there are times when customizing the behavior is desirable.  Both frameworks provide full configuration support. When working with Data Parallelism, there is one primary configuration option we often need to control – the number of threads we want the system to use when parallelizing our routine.  By default, PLINQ and the TPL both use the ThreadPool to schedule tasks.  Given the major improvements in the ThreadPool in CLR 4, this default behavior is often ideal.  However, there are times that the default behavior is not appropriate.  For example, if you are working on multiple threads simultaneously, and want to schedule parallel operations from within both threads, you might want to consider restricting each parallel operation to using a subset of the processing cores of the system.  Not doing this might over-parallelize your routine, which leads to inefficiencies from having too many context switches. In the Task Parallel Library, configuration is handled via the ParallelOptions class.  All of the methods of the Parallel class have an overload which accepts a ParallelOptions argument. We configure the Parallel class by setting the ParallelOptions.MaxDegreeOfParallelism property.  For example, let’s revisit one of the simple data parallel examples from Part 2: Parallel.For(0, pixelData.GetUpperBound(0), row => { for (int col=0; col < pixelData.GetUpperBound(1); ++col) { pixelData[row, col] = AdjustContrast(pixelData[row, col], minPixel, maxPixel); } }); .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } Here, we’re looping through an image, and calling a method on each pixel in the image.  If this was being done on a separate thread, and we knew another thread within our system was going to be doing a similar operation, we likely would want to restrict this to using half of the cores on the system.  This could be accomplished easily by doing: var options = new ParallelOptions(); options.MaxDegreeOfParallelism = Math.Max(Environment.ProcessorCount / 2, 1); Parallel.For(0, pixelData.GetUpperBound(0), options, row => { for (int col=0; col < pixelData.GetUpperBound(1); ++col) { pixelData[row, col] = AdjustContrast(pixelData[row, col], minPixel, maxPixel); } }); Now, we’re restricting this routine to using no more than half the cores in our system.  Note that I included a check to prevent a single core system from supplying zero; without this check, we’d potentially cause an exception.  I also did not hard code a specific value for the MaxDegreeOfParallelism property.  One of our goals when parallelizing a routine is allowing it to scale on better hardware.  Specifying a hard-coded value would contradict that goal. Parallel LINQ also supports configuration, and in fact, has quite a few more options for configuring the system.  The main configuration option we most often need is the same as our TPL option: we need to supply the maximum number of processing threads.  In PLINQ, this is done via a new extension method on ParallelQuery<T>: ParallelEnumerable.WithDegreeOfParallelism. Let’s revisit our declarative data parallelism sample from Part 6: double min = collection.AsParallel().Min(item => item.PerformComputation()); Here, we’re performing a computation on each element in the collection, and saving the minimum value of this operation.  If we wanted to restrict this to a limited number of threads, we would add our new extension method: int maxThreads = Math.Max(Environment.ProcessorCount / 2, 1); double min = collection .AsParallel() .WithDegreeOfParallelism(maxThreads) .Min(item => item.PerformComputation()); This automatically restricts the PLINQ query to half of the threads on the system. PLINQ provides some additional configuration options.  By default, PLINQ will occasionally revert to processing a query in parallel.  This occurs because many queries, if parallelized, typically actually cause an overall slowdown compared to a serial processing equivalent.  By analyzing the “shape” of the query, PLINQ often decides to run a query serially instead of in parallel.  This can occur for (taken from MSDN): Queries that contain a Select, indexed Where, indexed SelectMany, or ElementAt clause after an ordering or filtering operator that has removed or rearranged original indices. Queries that contain a Take, TakeWhile, Skip, SkipWhile operator and where indices in the source sequence are not in the original order. Queries that contain Zip or SequenceEquals, unless one of the data sources has an originally ordered index and the other data source is indexable (i.e. an array or IList(T)). Queries that contain Concat, unless it is applied to indexable data sources. Queries that contain Reverse, unless applied to an indexable data source. If the specific query follows these rules, PLINQ will run the query on a single thread.  However, none of these rules look at the specific work being done in the delegates, only at the “shape” of the query.  There are cases where running in parallel may still be beneficial, even if the shape is one where it typically parallelizes poorly.  In these cases, you can override the default behavior by using the WithExecutionMode extension method.  This would be done like so: var reversed = collection .AsParallel() .WithExecutionMode(ParallelExecutionMode.ForceParallelism) .Select(i => i.PerformComputation()) .Reverse(); Here, the default behavior would be to not parallelize the query unless collection implemented IList<T>.  We can force this to run in parallel by adding the WithExecutionMode extension method in the method chain. Finally, PLINQ has the ability to configure how results are returned.  When a query is filtering or selecting an input collection, the results will need to be streamed back into a single IEnumerable<T> result.  For example, the method above returns a new, reversed collection.  In this case, the processing of the collection will be done in parallel, but the results need to be streamed back to the caller serially, so they can be enumerated on a single thread. This streaming introduces overhead.  IEnumerable<T> isn’t designed with thread safety in mind, so the system needs to handle merging the parallel processes back into a single stream, which introduces synchronization issues.  There are two extremes of how this could be accomplished, but both extremes have disadvantages. The system could watch each thread, and whenever a thread produces a result, take that result and send it back to the caller.  This would mean that the calling thread would have access to the data as soon as data is available, which is the benefit of this approach.  However, it also means that every item is introducing synchronization overhead, since each item needs to be merged individually. On the other extreme, the system could wait until all of the results from all of the threads were ready, then push all of the results back to the calling thread in one shot.  The advantage here is that the least amount of synchronization is added to the system, which means the query will, on a whole, run the fastest.  However, the calling thread will have to wait for all elements to be processed, so this could introduce a long delay between when a parallel query begins and when results are returned. The default behavior in PLINQ is actually between these two extremes.  By default, PLINQ maintains an internal buffer, and chooses an optimal buffer size to maintain.  Query results are accumulated into the buffer, then returned in the IEnumerable<T> result in chunks.  This provides reasonably fast access to the results, as well as good overall throughput, in most scenarios. However, if we know the nature of our algorithm, we may decide we would prefer one of the other extremes.  This can be done by using the WithMergeOptions extension method.  For example, if we know that our PerformComputation() routine is very slow, but also variable in runtime, we may want to retrieve results as they are available, with no bufferring.  This can be done by changing our above routine to: var reversed = collection .AsParallel() .WithExecutionMode(ParallelExecutionMode.ForceParallelism) .WithMergeOptions(ParallelMergeOptions.NotBuffered) .Select(i => i.PerformComputation()) .Reverse(); On the other hand, if are already on a background thread, and we want to allow the system to maximize its speed, we might want to allow the system to fully buffer the results: var reversed = collection .AsParallel() .WithExecutionMode(ParallelExecutionMode.ForceParallelism) .WithMergeOptions(ParallelMergeOptions.FullyBuffered) .Select(i => i.PerformComputation()) .Reverse(); Notice, also, that you can specify multiple configuration options in a parallel query.  By chaining these extension methods together, we generate a query that will always run in parallel, and will always complete before making the results available in our IEnumerable<T>.

  Read the article

• Parallelism in .NET – Part 2, Simple Imperative Data Parallelism

  - by Reed

  In my discussion of Decomposition of the problem space, I mentioned that Data Decomposition is often the simplest abstraction to use when trying to parallelize a routine.  If a problem can be decomposed based off the data, we will often want to use what MSDN refers to as Data Parallelism as our strategy for implementing our routine.  The Task Parallel Library in .NET 4 makes implementing Data Parallelism, for most cases, very simple. Data Parallelism is the main technique we use to parallelize a routine which can be decomposed based off data.  Data Parallelism refers to taking a single collection of data, and having a single operation be performed concurrently on elements in the collection.  One side note here: Data Parallelism is also sometimes referred to as the Loop Parallelism Pattern or Loop-level Parallelism.  In general, for this series, I will try to use the terminology used in the MSDN Documentation for the Task Parallel Library.  This should make it easier to investigate these topics in more detail. Once we’ve determined we have a problem that, potentially, can be decomposed based on data, implementation using Data Parallelism in the TPL is quite simple.  Let’s take our example from the Data Decomposition discussion – a simple contrast stretching filter.  Here, we have a collection of data (pixels), and we need to run a simple operation on each element of the pixel.  Once we know the minimum and maximum values, we most likely would have some simple code like the following: for (int row=0; row < pixelData.GetUpperBound(0); ++row) { for (int col=0; col < pixelData.GetUpperBound(1); ++col) { pixelData[row, col] = AdjustContrast(pixelData[row, col], minPixel, maxPixel); } } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } This simple routine loops through a two dimensional array of pixelData, and calls the AdjustContrast routine on each pixel. As I mentioned, when you’re decomposing a problem space, most iteration statements are potentially candidates for data decomposition.  Here, we’re using two for loops – one looping through rows in the image, and a second nested loop iterating through the columns.  We then perform one, independent operation on each element based on those loop positions. This is a prime candidate – we have no shared data, no dependencies on anything but the pixel which we want to change.  Since we’re using a for loop, we can easily parallelize this using the Parallel.For method in the TPL: Parallel.For(0, pixelData.GetUpperBound(0), row => { for (int col=0; col < pixelData.GetUpperBound(1); ++col) { pixelData[row, col] = AdjustContrast(pixelData[row, col], minPixel, maxPixel); } }); Here, by simply changing our first for loop to a call to Parallel.For, we can parallelize this portion of our routine.  Parallel.For works, as do many methods in the TPL, by creating a delegate and using it as an argument to a method.  In this case, our for loop iteration block becomes a delegate creating via a lambda expression.  This lets you write code that, superficially, looks similar to the familiar for loop, but functions quite differently at runtime. We could easily do this to our second for loop as well, but that may not be a good idea.  There is a balance to be struck when writing parallel code.  We want to have enough work items to keep all of our processors busy, but the more we partition our data, the more overhead we introduce.  In this case, we have an image of data – most likely hundreds of pixels in both dimensions.  By just parallelizing our first loop, each row of pixels can be run as a single task.  With hundreds of rows of data, we are providing fine enough granularity to keep all of our processors busy. If we parallelize both loops, we’re potentially creating millions of independent tasks.  This introduces extra overhead with no extra gain, and will actually reduce our overall performance.  This leads to my first guideline when writing parallel code: Partition your problem into enough tasks to keep each processor busy throughout the operation, but not more than necessary to keep each processor busy. Also note that I parallelized the outer loop.  I could have just as easily partitioned the inner loop.  However, partitioning the inner loop would have led to many more discrete work items, each with a smaller amount of work (operate on one pixel instead of one row of pixels).  My second guideline when writing parallel code reflects this: Partition your problem in a way to place the most work possible into each task. This typically means, in practice, that you will want to parallelize the routine at the “highest” point possible in the routine, typically the outermost loop.  If you’re looking at parallelizing methods which call other methods, you’ll want to try to partition your work high up in the stack – as you get into lower level methods, the performance impact of parallelizing your routines may not overcome the overhead introduced. Parallel.For works great for situations where we know the number of elements we’re going to process in advance.  If we’re iterating through an IList<T> or an array, this is a typical approach.  However, there are other iteration statements common in C#.  In many situations, we’ll use foreach instead of a for loop.  This can be more understandable and easier to read, but also has the advantage of working with collections which only implement IEnumerable<T>, where we do not know the number of elements involved in advance. As an example, lets take the following situation.  Say we have a collection of Customers, and we want to iterate through each customer, check some information about the customer, and if a certain case is met, send an email to the customer and update our instance to reflect this change.  Normally, this might look something like: foreach(var customer in customers) { // Run some process that takes some time... DateTime lastContact = theStore.GetLastContact(customer); TimeSpan timeSinceContact = DateTime.Now - lastContact; // If it's been more than two weeks, send an email, and update... if (timeSinceContact.Days > 14) { theStore.EmailCustomer(customer); customer.LastEmailContact = DateTime.Now; } } Here, we’re doing a fair amount of work for each customer in our collection, but we don’t know how many customers exist.  If we assume that theStore.GetLastContact(customer) and theStore.EmailCustomer(customer) are both side-effect free, thread safe operations, we could parallelize this using Parallel.ForEach: Parallel.ForEach(customers, customer => { // Run some process that takes some time... DateTime lastContact = theStore.GetLastContact(customer); TimeSpan timeSinceContact = DateTime.Now - lastContact; // If it's been more than two weeks, send an email, and update... if (timeSinceContact.Days > 14) { theStore.EmailCustomer(customer); customer.LastEmailContact = DateTime.Now; } }); Just like Parallel.For, we rework our loop into a method call accepting a delegate created via a lambda expression.  This keeps our new code very similar to our original iteration statement, however, this will now execute in parallel.  The same guidelines apply with Parallel.ForEach as with Parallel.For. The other iteration statements, do and while, do not have direct equivalents in the Task Parallel Library.  These, however, are very easy to implement using Parallel.ForEach and the yield keyword. Most applications can benefit from implementing some form of Data Parallelism.  Iterating through collections and performing “work” is a very common pattern in nearly every application.  When the problem can be decomposed by data, we often can parallelize the workload by merely changing foreach statements to Parallel.ForEach method calls, and for loops to Parallel.For method calls.  Any time your program operates on a collection, and does a set of work on each item in the collection where that work is not dependent on other information, you very likely have an opportunity to parallelize your routine.

  Read the article

• Parallelism in .NET – Part 4, Imperative Data Parallelism: Aggregation

  - by Reed

  In the article on simple data parallelism, I described how to perform an operation on an entire collection of elements in parallel.  Often, this is not adequate, as the parallel operation is going to be performing some form of aggregation. Simple examples of this might include taking the sum of the results of processing a function on each element in the collection, or finding the minimum of the collection given some criteria.  This can be done using the techniques described in simple data parallelism, however, special care needs to be taken into account to synchronize the shared data appropriately.  The Task Parallel Library has tools to assist in this synchronization. The main issue with aggregation when parallelizing a routine is that you need to handle synchronization of data.  Since multiple threads will need to write to a shared portion of data.  Suppose, for example, that we wanted to parallelize a simple loop that looked for the minimum value within a dataset: double min = double.MaxValue; foreach(var item in collection) { double value = item.PerformComputation(); min = System.Math.Min(min, value); } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } This seems like a good candidate for parallelization, but there is a problem here.  If we just wrap this into a call to Parallel.ForEach, we’ll introduce a critical race condition, and get the wrong answer.  Let’s look at what happens here: // Buggy code! Do not use! double min = double.MaxValue; Parallel.ForEach(collection, item => { double value = item.PerformComputation(); min = System.Math.Min(min, value); }); This code has a fatal flaw: min will be checked, then set, by multiple threads simultaneously.  Two threads may perform the check at the same time, and set the wrong value for min.  Say we get a value of 1 in thread 1, and a value of 2 in thread 2, and these two elements are the first two to run.  If both hit the min check line at the same time, both will determine that min should change, to 1 and 2 respectively.  If element 1 happens to set the variable first, then element 2 sets the min variable, we’ll detect a min value of 2 instead of 1.  This can lead to wrong answers. Unfortunately, fixing this, with the Parallel.ForEach call we’re using, would require adding locking.  We would need to rewrite this like: // Safe, but slow double min = double.MaxValue; // Make a "lock" object object syncObject = new object(); Parallel.ForEach(collection, item => { double value = item.PerformComputation(); lock(syncObject) min = System.Math.Min(min, value); }); This will potentially add a huge amount of overhead to our calculation.  Since we can potentially block while waiting on the lock for every single iteration, we will most likely slow this down to where it is actually quite a bit slower than our serial implementation.  The problem is the lock statement – any time you use lock(object), you’re almost assuring reduced performance in a parallel situation.  This leads to two observations I’ll make: When parallelizing a routine, try to avoid locks. That being said: Always add any and all required synchronization to avoid race conditions. These two observations tend to be opposing forces – we often need to synchronize our algorithms, but we also want to avoid the synchronization when possible.  Looking at our routine, there is no way to directly avoid this lock, since each element is potentially being run on a separate thread, and this lock is necessary in order for our routine to function correctly every time. However, this isn’t the only way to design this routine to implement this algorithm.  Realize that, although our collection may have thousands or even millions of elements, we have a limited number of Processing Elements (PE).  Processing Element is the standard term for a hardware element which can process and execute instructions.  This typically is a core in your processor, but many modern systems have multiple hardware execution threads per core.  The Task Parallel Library will not execute the work for each item in the collection as a separate work item. Instead, when Parallel.ForEach executes, it will partition the collection into larger “chunks” which get processed on different threads via the ThreadPool.  This helps reduce the threading overhead, and help the overall speed.  In general, the Parallel class will only use one thread per PE in the system. Given the fact that there are typically fewer threads than work items, we can rethink our algorithm design.  We can parallelize our algorithm more effectively by approaching it differently.  Because the basic aggregation we are doing here (Min) is communitive, we do not need to perform this in a given order.  We knew this to be true already – otherwise, we wouldn’t have been able to parallelize this routine in the first place.  With this in mind, we can treat each thread’s work independently, allowing each thread to serially process many elements with no locking, then, after all the threads are complete, “merge” together the results. This can be accomplished via a different set of overloads in the Parallel class: Parallel.ForEach<TSource,TLocal>.  The idea behind these overloads is to allow each thread to begin by initializing some local state (TLocal).  The thread will then process an entire set of items in the source collection, providing that state to the delegate which processes an individual item.  Finally, at the end, a separate delegate is run which allows you to handle merging that local state into your final results. To rewriting our routine using Parallel.ForEach<TSource,TLocal>, we need to provide three delegates instead of one.  The most basic version of this function is declared as: public static ParallelLoopResult ForEach<TSource, TLocal>( IEnumerable<TSource> source, Func<TLocal> localInit, Func<TSource, ParallelLoopState, TLocal, TLocal> body, Action<TLocal> localFinally ) The first delegate (the localInit argument) is defined as Func<TLocal>.  This delegate initializes our local state.  It should return some object we can use to track the results of a single thread’s operations. The second delegate (the body argument) is where our main processing occurs, although now, instead of being an Action<T>, we actually provide a Func<TSource, ParallelLoopState, TLocal, TLocal> delegate.  This delegate will receive three arguments: our original element from the collection (TSource), a ParallelLoopState which we can use for early termination, and the instance of our local state we created (TLocal).  It should do whatever processing you wish to occur per element, then return the value of the local state after processing is completed. The third delegate (the localFinally argument) is defined as Action<TLocal>.  This delegate is passed our local state after it’s been processed by all of the elements this thread will handle.  This is where you can merge your final results together.  This may require synchronization, but now, instead of synchronizing once per element (potentially millions of times), you’ll only have to synchronize once per thread, which is an ideal situation. Now that I’ve explained how this works, lets look at the code: // Safe, and fast! double min = double.MaxValue; // Make a "lock" object object syncObject = new object(); Parallel.ForEach( collection, // First, we provide a local state initialization delegate. () => double.MaxValue, // Next, we supply the body, which takes the original item, loop state, // and local state, and returns a new local state (item, loopState, localState) => { double value = item.PerformComputation(); return System.Math.Min(localState, value); }, // Finally, we provide an Action<TLocal>, to "merge" results together localState => { // This requires locking, but it's only once per used thread lock(syncObj) min = System.Math.Min(min, localState); } ); Although this is a bit more complicated than the previous version, it is now both thread-safe, and has minimal locking.  This same approach can be used by Parallel.For, although now, it’s Parallel.For<TLocal>.  When working with Parallel.For<TLocal>, you use the same triplet of delegates, with the same purpose and results. Also, many times, you can completely avoid locking by using a method of the Interlocked class to perform the final aggregation in an atomic operation.  The MSDN example demonstrating this same technique using Parallel.For uses the Interlocked class instead of a lock, since they are doing a sum operation on a long variable, which is possible via Interlocked.Add. By taking advantage of local state, we can use the Parallel class methods to parallelize algorithms such as aggregation, which, at first, may seem like poor candidates for parallelization.  Doing so requires careful consideration, and often requires a slight redesign of the algorithm, but the performance gains can be significant if handled in a way to avoid excessive synchronization.

  Read the article

• Parallelism in .NET – Part 11, Divide and Conquer via Parallel.Invoke

  - by Reed

  Many algorithms are easily written to work via recursion.  For example, most data-oriented tasks where a tree of data must be processed are much more easily handled by starting at the root, and recursively “walking” the tree.  Some algorithms work this way on flat data structures, such as arrays, as well.  This is a form of divide and conquer: an algorithm design which is based around breaking up a set of work recursively, “dividing” the total work in each recursive step, and “conquering” the work when the remaining work is small enough to be solved easily. Recursive algorithms, especially ones based on a form of divide and conquer, are often a very good candidate for parallelization. This is apparent from a common sense standpoint.  Since we’re dividing up the total work in the algorithm, we have an obvious, built-in partitioning scheme.  Once partitioned, the data can be worked upon independently, so there is good, clean isolation of data. Implementing this type of algorithm is fairly simple.  The Parallel class in .NET 4 includes a method suited for this type of operation: Parallel.Invoke.  This method works by taking any number of delegates defined as an Action, and operating them all in parallel.  The method returns when every delegate has completed: Parallel.Invoke( () => { Console.WriteLine("Action 1 executing in thread {0}", Thread.CurrentThread.ManagedThreadId); }, () => { Console.WriteLine("Action 2 executing in thread {0}", Thread.CurrentThread.ManagedThreadId); }, () => { Console.WriteLine("Action 3 executing in thread {0}", Thread.CurrentThread.ManagedThreadId); } ); .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } Running this simple example demonstrates the ease of using this method.  For example, on my system, I get three separate thread IDs when running the above code.  By allowing any number of delegates to be executed directly, concurrently, the Parallel.Invoke method provides us an easy way to parallelize any algorithm based on divide and conquer.  We can divide our work in each step, and execute each task in parallel, recursively. For example, suppose we wanted to implement our own quicksort routine.  The quicksort algorithm can be designed based on divide and conquer.  In each iteration, we pick a pivot point, and use that to partition the total array.  We swap the elements around the pivot, then recursively sort the lists on each side of the pivot.  For example, let’s look at this simple, sequential implementation of quicksort: public static void QuickSort<T>(T[] array) where T : IComparable<T> { QuickSortInternal(array, 0, array.Length - 1); } private static void QuickSortInternal<T>(T[] array, int left, int right) where T : IComparable<T> { if (left >= right) { return; } SwapElements(array, left, (left + right) / 2); int last = left; for (int current = left + 1; current <= right; ++current) { if (array[current].CompareTo(array[left]) < 0) { ++last; SwapElements(array, last, current); } } SwapElements(array, left, last); QuickSortInternal(array, left, last - 1); QuickSortInternal(array, last + 1, right); } static void SwapElements<T>(T[] array, int i, int j) { T temp = array[i]; array[i] = array[j]; array[j] = temp; } Here, we implement the quicksort algorithm in a very common, divide and conquer approach.  Running this against the built-in Array.Sort routine shows that we get the exact same answers (although the framework’s sort routine is slightly faster).  On my system, for example, I can use framework’s sort to sort ten million random doubles in about 7.3s, and this implementation takes about 9.3s on average. Looking at this routine, though, there is a clear opportunity to parallelize.  At the end of QuickSortInternal, we recursively call into QuickSortInternal with each partition of the array after the pivot is chosen.  This can be rewritten to use Parallel.Invoke by simply changing it to: // Code above is unchanged... SwapElements(array, left, last); Parallel.Invoke( () => QuickSortInternal(array, left, last - 1), () => QuickSortInternal(array, last + 1, right) ); } This routine will now run in parallel.  When executing, we now see the CPU usage across all cores spike while it executes.  However, there is a significant problem here – by parallelizing this routine, we took it from an execution time of 9.3s to an execution time of approximately 14 seconds!  We’re using more resources as seen in the CPU usage, but the overall result is a dramatic slowdown in overall processing time. This occurs because parallelization adds overhead.  Each time we split this array, we spawn two new tasks to parallelize this algorithm!  This is far, far too many tasks for our cores to operate upon at a single time.  In effect, we’re “over-parallelizing” this routine.  This is a common problem when working with divide and conquer algorithms, and leads to an important observation: When parallelizing a recursive routine, take special care not to add more tasks than necessary to fully utilize your system. This can be done with a few different approaches, in this case.  Typically, the way to handle this is to stop parallelizing the routine at a certain point, and revert back to the serial approach.  Since the first few recursions will all still be parallelized, our “deeper” recursive tasks will be running in parallel, and can take full advantage of the machine.  This also dramatically reduces the overhead added by parallelizing, since we’re only adding overhead for the first few recursive calls.  There are two basic approaches we can take here.  The first approach would be to look at the total work size, and if it’s smaller than a specific threshold, revert to our serial implementation.  In this case, we could just check right-left, and if it’s under a threshold, call the methods directly instead of using Parallel.Invoke. The second approach is to track how “deep” in the “tree” we are currently at, and if we are below some number of levels, stop parallelizing.  This approach is a more general-purpose approach, since it works on routines which parse trees as well as routines working off of a single array, but may not work as well if a poor partitioning strategy is chosen or the tree is not balanced evenly. This can be written very easily.  If we pass a maxDepth parameter into our internal routine, we can restrict the amount of times we parallelize by changing the recursive call to: // Code above is unchanged... SwapElements(array, left, last); if (maxDepth < 1) { QuickSortInternal(array, left, last - 1, maxDepth); QuickSortInternal(array, last + 1, right, maxDepth); } else { --maxDepth; Parallel.Invoke( () => QuickSortInternal(array, left, last - 1, maxDepth), () => QuickSortInternal(array, last + 1, right, maxDepth)); } We no longer allow this to parallelize indefinitely – only to a specific depth, at which time we revert to a serial implementation.  By starting the routine with a maxDepth equal to Environment.ProcessorCount, we can restrict the total amount of parallel operations significantly, but still provide adequate work for each processing core. With this final change, my timings are much better.  On average, I get the following timings: Framework via Array.Sort: 7.3 seconds Serial Quicksort Implementation: 9.3 seconds Naive Parallel Implementation: 14 seconds Parallel Implementation Restricting Depth: 4.7 seconds Finally, we are now faster than the framework’s Array.Sort implementation.

  Read the article

• Building and Deploying Windows Azure Web Sites using Git and GitHub for Windows

  - by shiju

  Microsoft Windows Azure team has released a new version of Windows Azure which is providing many excellent features. The new Windows Azure provides Web Sites which allows you to deploy up to 10 web sites  for free in a multitenant shared environment and you can easily upgrade this web site to a private, dedicated virtual server when the traffic is grows. The Meet Windows Azure Fact Sheet provides the following information about a Windows Azure Web Site: Windows Azure Web Sites enable developers to easily build and deploy websites with support for multiple frameworks and popular open source applications, including ASP.NET, PHP and Node.js. With just a few clicks, developers can take advantage of Windows Azure’s global scale without having to worry about operations, servers or infrastructure. It is easy to deploy existing sites, if they run on Internet Information Services (IIS) 7, or to build new sites, with a free offer of 10 websites upon signup, with the ability to scale up as needed with reserved instances. Windows Azure Web Sites includes support for the following: Multiple frameworks including ASP.NET, PHP and Node.js Popular open source software apps including WordPress, Joomla!, Drupal, Umbraco and DotNetNuke Windows Azure SQL Database and MySQL databases Multiple types of developer tools and protocols including Visual Studio, Git, FTP, Visual Studio Team Foundation Services and Microsoft WebMatrix Signup to Windows and Enable Azure Web Sites You can signup for a 90 days free trial account in Windows Azure from here. After creating an account in Windows Azure, go to https://account.windowsazure.com/ , and select to preview features to view the available previews. In the Web Sites section of the preview features, click “try it now” which will enables the web sites feature Create Web Site in Windows Azure To create a web sites, login to the Windows Azure portal, and select Web Sites from and click New icon from the left corner  Click WEB SITE, QUICK CREATE and put values for URL and REGION dropdown. You can see the all web sites from the dashboard of the Windows Azure portal Set up Git Publishing Select your web site from the dashboard, and select Set up Git publishing To enable Git publishing , you must give user name and password which will initialize a Git repository Clone Git Repository We can use GitHub for Windows to publish apps to non-GitHub repositories which is well explained by Phil Haack on his blog post. Here we are going to deploy the web site using GitHub for Windows. Let’s clone a Git repository using the Git Url which will be getting from the Windows Azure portal. Let’s copy the Git url and execute the “git clone” with the git url. You can use the Git Shell provided by GitHub for Windows. To get it, right on the GitHub for Windows, and select open shell here as shown in the below picture. When executing the Git Clone command, it will ask for a password where you have to give password which specified in the Windows Azure portal. After cloning the GIT repository, you can drag and drop the local Git repository folder to GitHub for Windows GUI. This will automatically add the Windows Azure Web Site repository onto GitHub for Windows where you can commit your changes and publish your web sites to Windows Azure. Publish the Web Site using GitHub for Windows We can add multiple framework level files including ASP.NET, PHP and Node.js, to the local repository folder can easily publish to Windows Azure from GitHub for Windows GUI. For this demo, let me just add a simple Node.js file named Server.js which handles few request handlers. 1: var http = require('http'); 2: var port=process.env.PORT; 3: var querystring = require('querystring'); 4: var utils = require('util'); 5: var url = require("url"); 6:   7: var server = http.createServer(function(req, res) { 8: switch (req.url) { //checking the request url 9: case '/': 10: homePageHandler (req, res); //handler for home page 11: break; 12: case '/register': 13: registerFormHandler (req, res);//hamdler for register 14: break; 15: default: 16: nofoundHandler (req, res);// handler for 404 not found 17: break; 18: } 19: }); 20: server.listen(port); 21: //function to display the html form 22: function homePageHandler (req, res) { 23: console.log('Request handler home was called.'); 24: res.writeHead(200, {'Content-Type': 'text/html'}); 25: var body = '<html>'+ 26: '<head>'+ 27: '<meta http-equiv="Content-Type" content="text/html; '+ 28: 'charset=UTF-8" />'+ 29: '</head>'+ 30: '<body>'+ 31: '<form action="/register" method="post">'+ 32: 'Name:<input type=text value="" name="name" size=15></br>'+ 33: 'Email:<input type=text value="" name="email" size=15></br>'+ 34: '<input type="submit" value="Submit" />'+ 35: '</form>'+ 36: '</body>'+ 37: '</html>'; 38: //response content 39: res.end(body); 40: } 41: //handler for Post request 42: function registerFormHandler (req, res) { 43: console.log('Request handler register was called.'); 44: var pathname = url.parse(req.url).pathname; 45: console.log("Request for " + pathname + " received."); 46: var postData = ""; 47: req.on('data', function(chunk) { 48: // append the current chunk of data to the postData variable 49: postData += chunk.toString(); 50: }); 51: req.on('end', function() { 52: // doing something with the posted data 53: res.writeHead(200, "OK", {'Content-Type': 'text/html'}); 54: // parse the posted data 55: var decodedBody = querystring.parse(postData); 56: // output the decoded data to the HTTP response 57: res.write('<html><head><title>Post data</title></head><body><pre>'); 58: res.write(utils.inspect(decodedBody)); 59: res.write('</pre></body></html>'); 60: res.end(); 61: }); 62: } 63: //Error handler for 404 no found 64: function nofoundHandler(req, res) { 65: console.log('Request handler nofound was called.'); 66: res.writeHead(404, {'Content-Type': 'text/plain'}); 67: res.end('404 Error - Request handler not found'); 68: } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } If there is any change in the local repository folder, GitHub for Windows will automatically detect the changes. In the above step, we have just added a Server.js file so that GitHub for Windows will detect the changes. Let’s commit the changes to the local repository before publishing the web site to Windows Azure. After committed the all changes, you can click publish button which will publish the all changes to Windows Azure repository. The following screen shot shows deployment history from the Windows Azure portal.   GitHub for Windows is providing a sync button which can use for synchronizing between local repository and Windows Azure repository after making any commit on the local repository after any changes. Our web site is running after the deployment using Git Summary Windows Azure Web Sites lets the developers to easily build and deploy websites with support for multiple framework including ASP.NET, PHP and Node.js and can easily deploy the Web Sites using Visual Studio, Git, FTP, Visual Studio Team Foundation Services and Microsoft WebMatrix. In this demo, we have deployed a Node.js Web Site to Windows Azure using Git. We can use GitHub for Windows to publish apps to non-GitHub repositories and can use to publish Web SItes to Windows Azure.

  Read the article

• Can static methods be called using object/instance in .NET

  Ans is Yes and No   Yes in C++, Java and VB.NET No in C#   This is only compiler restriction in c#. You might see in some websites that we can break this restriction using reflection and delegates, but we can’t, according to my little research J I shall try to explain you…   Following is code sample to break this rule using reflection, it seems that it is possible to call a static method using an object, p1 using System; namespace T {     class Program     {         static void Main()         {             var p1 = new Person() { Name = "Smith" };             typeof(Person).GetMethod("TestStatMethod").Invoke(p1, new object[] { });                     }         class Person         {             public string Name { get; set; }             public static void TestStatMethod()             {                 Console.WriteLine("Hello");             }         }     } } but I do not think so this method is being called using p1 rather Type Name “Person”. I shall try to prove this… look at another example…  Test2 has been inherited from Test1. Let’s see various scenarios… Scenario1 using System; namespace T {     class Program     {         static void Main()         {             Test1 t = new Test1();            typeof(Test2).GetMethod("Method1").Invoke(t,                                  new object[] { });         }     }     class Test1     {         public static void Method1()         {             Console.WriteLine("At test1::Method1");         }     }       class Test2 : Test1     {         public static void Method1()         {             Console.WriteLine("At test1::Method2");         }     } } Output:   At test1::Method2 Scenario2         static void Main()         {             Test2 t = new Test2();            typeof(Test2).GetMethod("Method1").Invoke(t,                                          new object[] { });         }   Output:   At test1::Method2   Scenario3         static void Main()         {             Test1 t = new Test2();            typeof(Test2).GetMethod("Method1").Invoke(t,                             new object[] { });         }   Output: At test1::Method2 In all above scenarios output is same, that means, Reflection also not considering the object what you pass to Invoke method in case of static methods. It is always considering the type which you specify in typeof(). So, what is the use passing instance to “Invoke”. Let see below sample using System; namespace T {     class Program     {         static void Main()         {            typeof(Test2).GetMethod("Method1").                Invoke(null, new object[] { });         }     }       class Test1     {         public static void Method1()         {             Console.WriteLine("At test1::Method1");         }     }     class Test2 : Test1     {         public static void Method1()         {             Console.WriteLine("At test1::Method2");         }     } }   Output is   At test1::Method2   I was able to call Invoke “Method1” of Test2 without any object.  Yes, there no wonder here as Method1 is static. So we may conclude that static methods cannot be called using instances (only in c#) Why Microsoft has restricted it in C#? Ans: Really there Is no use calling static methods using objects because static methods are stateless. but still Java and C++ latest compilers allow calling static methods using instances. Java sample class Test {      public static void main(String str[])      {            Person p = new Person();            System.out.println(p.GetCount());      } }   class Person {   public static int GetCount()   {      return 100;   } }   Output          100 span.fullpost {display:none;}

  Read the article

• CodePlex Daily Summary for Sunday, May 25, 2014

  CodePlex Daily Summary for Sunday, May 25, 2014Popular ReleasesClosedXML - The easy way to OpenXML: ClosedXML 0.71.0: Major improvement when saving large files.SimCityPak: SimCityPak 0.3.1.0: Main New Features: Fixed Importing of Instance Names (get rid of the Dutch translations) Added advanced editor for Decal Dictionaries Added possibility to import .PNG to generate new decals Added advanced editor for Path display entriesTiny Deduplicator: Tiny Deduplicator 1.0.1.0: Increased version number to 1.0.1.0 Moved all options to a separate 'Options' dialog window. Allows the user to specify a selection strategy which will help when dealing with large numbers of duplicate files. Available options are "None," "Keep First," and "Keep Last"SEToolbox: SEToolbox 01.031.009 Release 1: Added mirroring of ConveyorTubeCurved. Updated Ship cube rotation to rotate ship back to original location (cubes are reoriented but ship appears no different to outsider), and to rotate Grouped items. Repair now fixes the loss of Grouped controls due to changes in Space Engineers 01.030. Added export asteroids. Rejoin ships will merge grouping and conveyor systems (even though broken ships currently only maintain the Grouping on one part of the ship). Installation of this version wi...Player Framework by Microsoft: Player Framework for Windows and WP v2.0: Support for new Universal and Windows Phone 8.1 projects for both Xaml and JavaScript projects. See a detailed list of improvements, breaking changes and a general overview of version 2 ADDITIONAL DOWNLOADSSmooth Streaming Client SDK for Windows 8 Applications Smooth Streaming Client SDK for Windows 8.1 Applications Smooth Streaming Client SDK for Windows Phone 8.1 Applications Microsoft PlayReady Client SDK for Windows 8 Applications Microsoft PlayReady Client SDK for Windows 8.1 Applicat...TerraMap (Terraria World Map Viewer): TerraMap 1.0.6: Added support for the new Terraria v1.2.4 update. New items, walls, and tiles Added the ability to select multiple highlighted block types. Added a dynamic, interactive highlight opacity slider, making it easier to find highlighted tiles with dark colors (and fixed blurriness from 1.0.5 alpha). Added ability to find Enchanted Swords (in the stone) and Water Bolt books Fixed Issue 35206: Hightlight/Find doesn't work for Demon Altars Fixed finding Demon Hearts/Shadow Orbs Fixed inst...DotNet.Highcharts: DotNet.Highcharts 4.0 with Examples: DotNet.Highcharts 4.0 Tested and adapted to the latest version of Highcharts 4.0.1 Added new chart type: Heatmap Added new type PointPlacement which represents enumeration or number for the padding of the X axis. Changed target framework from .NET Framework 4 to .NET Framework 4.5. Closed issues: 974: Add 'overflow' property to PlotOptionsColumnDataLabels class 997: Split container from JS 1006: Series/Categories with numeric names don't render DotNet.Highcharts.Samples Updated s...51Degrees - Device Detection and Redirection: 3.1.1.12: Version 3.1 HighlightsDevice detection algorithm is over 100 times faster. Regular expressions and levenshtein distance calculations are no longer used. The device detection algorithm performance is no longer limited by the number of device combinations contained in the dataset. Two modes of operation are available: Memory – the detection data set is loaded into memory and there is no continuous connection to the source data file. Slower initialisation time but faster detection performanc...VisioAutomation: Visio PowerShell Module (VisioPS) 1.2.0: DocumentationDocumentation is here http://sdrv.ms/11AWkp7 Screencasthttp://vimeo.com/61329170 FilesFor easy installation, download and run the MSI file. If you want to manually install, a ZIP file is provided. ChangeLogInvoke-* cmdlets replaced with more specific PowerShell Verbs Enhanced handling of values in User-Defined Cells Get-VisioShape works more unituitivelyNino Seisei Code Generator: Nino Seisei v2.1.1: Mejoras en la interfaz, posibilidad de ejecutar instrucciones Berta cíclicas una dentro de otra con la nueva instrucción @BSETRECURSIVITY_ON. GUI Improvements, posibility of running ciclic Berta instructions one inside another with the new @BSETRECURSIVITY_ON instruction.PowerShell App Deployment Toolkit: PowerShell App Deployment Toolkit v3.1.3: Added CompressLogs option to the config file. Each Install / Uninstall creates a timestamped zip file with all MSI and PSAppDeployToolkit logs contained within Added variable expansion to all paths in the configuration file Added documentation for each of the Toolkit internal variables that can be used Changed Install-MSUpdates to continue if any errors are encountered when installing updates Implement /Force parameter on Update-GroupPolicy (ensure that any logoff message is ignored) ...ULS Log Viewer: Alpha 0.2: Changeset CI&T ULS Log 22/05/2014 - Inclusão do botão de limpar filtro - Inclusão da possibilidade de filtrar as entradas pelo texto da mensagem; - Inclusão da opção de abrir mais de um arquivo de log no mesmo grid para análise; - Inclusão da tela de Sobre. - Campo de Filtro Rápido Level carrega somente os Levels encontrados no arquivo de log carregado; - Campo de exibição rápida de mensagem setado para somente leitura; - Inclusão da Barra de Status com informações do nome do arquivo ...Application Parameters for Microsoft Dynamics CRM: Application Parameters (1.2.0.1): Fix plugin when updating parameters without changing parameter typeWordMat: WordMat v. 1.07: A quick fix because scientific notation was broken in v. 1.06 read more at http://wordmat.blogspot.com????: 《????》: 《????》（c???）??“????”???????，???????????????C?????????。???????,???????????????????????. ??????????????????????????????????；????????????????????????????。Mini SQL Query: Mini SQL Query (1.0.72.457): Apologies for the previous update! FK issue fixed and also a template data cache issue.Wsus Package Publisher: Release v1.3.1405.17: Add Russian translation (thanks to VSharmanov) Fix a bug that make WPP to crash if the user click on "Connect/Reload" while the Report Tab is loading. Enhance the way WPP store the password for remote computers command.MoreTerra (Terraria World Viewer): More Terra 1.12.9: =========== = Compatibility = =========== Updated to account for new format 1.2.4.1 =========== = Issues = =========== all items have not been added. Some colors for new tiles may be off. I wanted to get this out so people have a usable program.LINQ to Twitter: LINQ to Twitter v3.0.3: Supports .NET 4.5x, Windows Phone 8.x, Windows 8.x, Windows Azure, Xamarin.Android, and Xamarin.iOS. New features include Status/Lookup, Mute APIs, and bug fixes. 100% Twitter API v1.1 coverage, Async, Portable Class Library (PCL).CS-Script for Notepad++ (C# intellisense and code execution): Release v1.0.26.0: Added access to the Release Notes during 'Check for Updates...'' Debug panels Added support for generic types members Members are grouped into 'Raw View' and 'Non-Public members' categories Implemented dedicated (array-like) view for Lists and Dictionaries http://download-codeplex.sec.s-msft.com/Download?ProjectName=csscriptnpp&DownloadId=846498New Projects2111110107: Thanh Loi2111110152: Nguy?n Doãn Tu?nASP.NET MVC4 Warehouse management system: WMSNet is an easy to use warehouse management solution for manually operated warehouses and can smoothly be customized according to your requirements. Code Snippets: Code snippets to empower the developers to write quality code faster while adhering to the industry standards.CRM 2011 / CRM 2013 Form Helper: Library of CRM 2011 / CRM 2013 Web Resources that can be used on Forms for making input simpler; ex Automatic title case Kinect Translation Tool: From Sign Language to spoken text and vice versa: Software System Component 1. Kinect SDKver.1.7 for the Kinect sensor. 2. Windows 7 standard APIs- The audio, speech, and media APIs in Windows 7MDriven Getting started - MVC: This is the suggested getting started template for doing MVC with MDriven. Fork it and begin to fill up with your model. Rules Engine Validator: A simple rules validator. It's based on a rule manager component (an implementation of the Command GoF pattern), with a main method called Validate().Simple Connect To Db: ????? ???? ?? ??????? ?? ????? ??? ? ??????? ????? ??z3-str-purdue: test geekchina.com

  Read the article

• ??11.2 RAC??OCR?Votedisk??ASM Diskgroup?????

  - by Liu Maclean(???)

  ????????Oracle Allstarts??????????ocr?votedisk?ASM diskgroup??11gR2 RAC cluster?????????,????«?11gR2 RAC???ASM DISK Path????»??????,??????CRS??????11.2??ASM???????, ????????????”crsctl start crs -excl -nocrs “; ?????????,??ASM????ocr?????votedisk?????,??11.2????ocr?votedisk???ASM?,?ASM???????ocr?votedisk,?????ocr?votedisk????????cluter??????;???????????CRS????,?????diskgroup??????????,?????????????????? ??:?????????????????ASM LUN DISK,???OCR?????,????????4??????????,???????$GI_HOME,?????????;????votedisk?? ????: ??dd????ocr?votedisk??diskgroup header,??diskgroup corruption: 1. ??votedisk? ocr?? [root@vrh1 ~]# crsctl query css votedisk ## STATE File Universal Id File Name Disk group -- ----- ----------------- --------- --------- 1. ONLINE a853d6204bbc4feabfd8c73d4c3b3001 (/dev/asm-diskh) [SYSTEMDG] 2. ONLINE a5b37704c3574f0fbf21d1d9f58c4a6b (/dev/asm-diskg) [SYSTEMDG] 3. ONLINE 36e5c51ff0294fc3bf2a042266650331 (/dev/asm-diski) [SYSTEMDG] 4. ONLINE af337d1512824fe4bf6ad45283517aaa (/dev/asm-diskj) [SYSTEMDG] 5. ONLINE 3c4a349e2e304ff6bf64b2b1c9d9cf5d (/dev/asm-diskk) [SYSTEMDG] Located 5 voting disk(s). su - grid [grid@vrh1 ~]$ ocrconfig -showbackup PROT-26: Oracle Cluster Registry backup locations were retrieved from a local copy vrh1 2012/08/09 01:59:56 /g01/11.2.0/maclean/grid/cdata/vrh-cluster/backup00.ocr vrh1 2012/08/08 21:59:56 /g01/11.2.0/maclean/grid/cdata/vrh-cluster/backup01.ocr vrh1 2012/08/08 17:59:55 /g01/11.2.0/maclean/grid/cdata/vrh-cluster/backup02.ocr vrh1 2012/08/08 05:59:54 /g01/11.2.0/grid/cdata/vrh-cluster/day.ocr vrh1 2012/08/08 05:59:54 /g01/11.2.0/grid/cdata/vrh-cluster/week.ocr PROT-25: Manual backups for the Oracle Cluster Registry are not available 2. ??????????clusterware ,OHASD crsctl stop has -f 3. GetAsmDH.sh ==> GetAsmDH.sh?ASM disk header????? ????????,????????asm header [grid@vrh1 ~]$ ./GetAsmDH.sh ############################################ 1) Collecting Information About the Disks: ############################################ SQL*Plus: Release 11.2.0.3.0 Production on Thu Aug 9 03:28:13 2012 Copyright (c) 1982, 2011, Oracle. All rights reserved. SQL> Connected. SQL> SQL> SQL> SQL> SQL> SQL> SQL> 1 0 /dev/asm-diske 1 1 /dev/asm-diskd 2 0 /dev/asm-diskb 2 1 /dev/asm-diskc 2 2 /dev/asm-diskf 3 0 /dev/asm-diskh 3 1 /dev/asm-diskg 3 2 /dev/asm-diski 3 3 /dev/asm-diskj 3 4 /dev/asm-diskk SQL> SQL> Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production With the Real Application Clusters and Automatic Storage Management options -rw-r--r-- 1 grid oinstall 1048 Aug 9 03:28 /tmp/HC/asmdisks.lst ############################################ 2) Generating asm_diskh.sh script. ############################################ -rwx------ 1 grid oinstall 666 Aug 9 03:28 /tmp/HC/asm_diskh.sh ############################################ 3) Executing asm_diskh.sh script to generate dd dumps. ############################################ -rw-r--r-- 1 grid oinstall 1048576 Aug 9 03:28 /tmp/HC/dsk_1_0.dd -rw-r--r-- 1 grid oinstall 1048576 Aug 9 03:28 /tmp/HC/dsk_1_1.dd -rw-r--r-- 1 grid oinstall 1048576 Aug 9 03:28 /tmp/HC/dsk_2_0.dd -rw-r--r-- 1 grid oinstall 1048576 Aug 9 03:28 /tmp/HC/dsk_2_1.dd -rw-r--r-- 1 grid oinstall 1048576 Aug 9 03:28 /tmp/HC/dsk_2_2.dd -rw-r--r-- 1 grid oinstall 1048576 Aug 9 03:28 /tmp/HC/dsk_3_0.dd -rw-r--r-- 1 grid oinstall 1048576 Aug 9 03:28 /tmp/HC/dsk_3_1.dd -rw-r--r-- 1 grid oinstall 1048576 Aug 9 03:28 /tmp/HC/dsk_3_2.dd -rw-r--r-- 1 grid oinstall 1048576 Aug 9 03:28 /tmp/HC/dsk_3_3.dd -rw-r--r-- 1 grid oinstall 1048576 Aug 9 03:28 /tmp/HC/dsk_3_4.dd ############################################ 4) Compressing dd dumps in the next format: (asm_dd_header_all_.tar) ############################################ /tmp/HC/dsk_1_0.dd /tmp/HC/dsk_1_1.dd /tmp/HC/dsk_2_0.dd /tmp/HC/dsk_2_1.dd /tmp/HC/dsk_2_2.dd /tmp/HC/dsk_3_0.dd /tmp/HC/dsk_3_1.dd /tmp/HC/dsk_3_2.dd /tmp/HC/dsk_3_3.dd /tmp/HC/dsk_3_4.dd ./GetAsmDH.sh: line 81: compress: command not found ls: /tmp/HC/*.Z: No such file or directory [grid@vrh1 ~]$ 4. ??dd ?? ??ocr?votedisk??diskgroup [root@vrh1 ~]# dd if=/dev/zero of=/dev/asm-diskh bs=1024k count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00423853 seconds, 247 MB/s [root@vrh1 ~]# dd if=/dev/zero of=/dev/asm-diskg bs=1024k count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0045179 seconds, 232 MB/s [root@vrh1 ~]# dd if=/dev/zero of=/dev/asm-diski bs=1024k count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00469976 seconds, 223 MB/s [root@vrh1 ~]# dd if=/dev/zero of=/dev/asm-diskj bs=1024k count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00344262 seconds, 305 MB/s [root@vrh1 ~]# dd if=/dev/zero of=/dev/asm-diskk bs=1024k count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0053518 seconds, 196 MB/s 5. ????????????HAS [root@vrh1 ~]# crsctl start has CRS-4123: Oracle High Availability Services has been started. ????ocr?votedisk??diskgroup??,??CSS???????,???????: alertvrh1.log [cssd(5162)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /g01/11.2.0/grid/log/vrh1/cssd/ocssd.log 2012-08-09 03:35:41.207 [cssd(5162)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /g01/11.2.0/grid/log/vrh1/cssd/ocssd.log 2012-08-09 03:35:56.240 [cssd(5162)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /g01/11.2.0/grid/log/vrh1/cssd/ocssd.log 2012-08-09 03:36:11.284 [cssd(5162)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /g01/11.2.0/grid/log/vrh1/cssd/ocssd.log 2012-08-09 03:36:26.305 [cssd(5162)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /g01/11.2.0/grid/log/vrh1/cssd/ocssd.log 2012-08-09 03:36:41.328 ocssd.log 2012-08-09 03:40:26.662: [ CSSD][1078700352]clssnmReadDiscoveryProfile: voting file discovery string(/dev/asm*) 2012-08-09 03:40:26.662: [ CSSD][1078700352]clssnmvDDiscThread: using discovery string /dev/asm* for initial discovery 2012-08-09 03:40:26.662: [ SKGFD][1078700352]Discovery with str:/dev/asm*: 2012-08-09 03:40:26.662: [ SKGFD][1078700352]UFS discovery with :/dev/asm*: 2012-08-09 03:40:26.665: [ SKGFD][1078700352]Fetching UFS disk :/dev/asm-diskf: 2012-08-09 03:40:26.665: [ SKGFD][1078700352]Fetching UFS disk :/dev/asm-diskb: 2012-08-09 03:40:26.665: [ SKGFD][1078700352]Fetching UFS disk :/dev/asm-diskj: 2012-08-09 03:40:26.665: [ SKGFD][1078700352]Fetching UFS disk :/dev/asm-diskh: 2012-08-09 03:40:26.665: [ SKGFD][1078700352]Fetching UFS disk :/dev/asm-diskc: 2012-08-09 03:40:26.665: [ SKGFD][1078700352]Fetching UFS disk :/dev/asm-diskd: 2012-08-09 03:40:26.665: [ SKGFD][1078700352]Fetching UFS disk :/dev/asm-diske: 2012-08-09 03:40:26.665: [ SKGFD][1078700352]Fetching UFS disk :/dev/asm-diskg: 2012-08-09 03:40:26.665: [ SKGFD][1078700352]Fetching UFS disk :/dev/asm-diski: 2012-08-09 03:40:26.665: [ SKGFD][1078700352]Fetching UFS disk :/dev/asm-diskk: 2012-08-09 03:40:26.665: [ SKGFD][1078700352]OSS discovery with :/dev/asm*: 2012-08-09 03:40:26.665: [ SKGFD][1078700352]Handle 0xdf22a0 from lib :UFS:: for disk :/dev/asm-diskf: 2012-08-09 03:40:26.665: [ SKGFD][1078700352]Handle 0xf412a0 from lib :UFS:: for disk :/dev/asm-diskb: 2012-08-09 03:40:26.666: [ SKGFD][1078700352]Handle 0xf3a680 from lib :UFS:: for disk :/dev/asm-diskj: 2012-08-09 03:40:26.666: [ SKGFD][1078700352]Handle 0xf93da0 from lib :UFS:: for disk :/dev/asm-diskh: 2012-08-09 03:40:26.667: [ CSSD][1078700352]clssnmvDiskVerify: Successful discovery of 0 disks 2012-08-09 03:40:26.667: [ CSSD][1078700352]clssnmCompleteInitVFDiscovery: Completing initial voting file discovery 2012-08-09 03:40:26.667: [ CSSD][1078700352]clssnmvFindInitialConfigs: No voting files found 2012-08-09 03:40:26.667: [ CSSD][1078700352](:CSSNM00070:)clssnmCompleteInitVFDiscovery: Voting file not found. Retrying discovery in 15 seconds ?????ocr?votedisk??diskgroup?????: 1. ?-excl -nocrs ????cluster,??????ASM?? ????CRS [root@vrh1 vrh1]# crsctl start crs -excl -nocrs CRS-4123: Oracle High Availability Services has been started. CRS-2672: Attempting to start 'ora.mdnsd' on 'vrh1' CRS-2676: Start of 'ora.mdnsd' on 'vrh1' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'vrh1' CRS-2676: Start of 'ora.gpnpd' on 'vrh1' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'vrh1' CRS-2672: Attempting to start 'ora.gipcd' on 'vrh1' CRS-2676: Start of 'ora.cssdmonitor' on 'vrh1' succeeded CRS-2676: Start of 'ora.gipcd' on 'vrh1' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'vrh1' CRS-2672: Attempting to start 'ora.diskmon' on 'vrh1' CRS-2676: Start of 'ora.diskmon' on 'vrh1' succeeded CRS-2676: Start of 'ora.cssd' on 'vrh1' succeeded CRS-2679: Attempting to clean 'ora.cluster_interconnect.haip' on 'vrh1' CRS-2672: Attempting to start 'ora.ctssd' on 'vrh1' CRS-2681: Clean of 'ora.cluster_interconnect.haip' on 'vrh1' succeeded CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'vrh1' CRS-2676: Start of 'ora.ctssd' on 'vrh1' succeeded CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'vrh1' succeeded CRS-2672: Attempting to start 'ora.asm' on 'vrh1' CRS-2676: Start of 'ora.asm' on 'vrh1' succeeded 2.???ocr?votedisk??diskgroup,??compatible.asm???11.2: [root@vrh1 vrh1]# su - grid [grid@vrh1 ~]$ sqlplus / as sysasm SQL*Plus: Release 11.2.0.3.0 Production on Thu Aug 9 04:16:58 2012 Copyright (c) 1982, 2011, Oracle. All rights reserved. Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production With the Real Application Clusters and Automatic Storage Management options SQL> create diskgroup systemdg high redundancy disk '/dev/asm-diskh','/dev/asm-diskg','/dev/asm-diski','/dev/asm-diskj','/dev/asm-diskk' ATTRIBUTE 'compatible.rdbms' = '11.2', 'compatible.asm' = '11.2'; 3.?ocr backup???ocr??ocrcheck??: [root@vrh1 ~]# ocrconfig -restore /g01/11.2.0/grid/cdata/vrh-cluster/backup00.ocr [root@vrh1 ~]# ocrcheck Status of Oracle Cluster Registry is as follows : Version : 3 Total space (kbytes) : 262120 Used space (kbytes) : 3180 Available space (kbytes) : 258940 ID : 1238458014 Device/File Name : +systemdg Device/File integrity check succeeded Device/File not configured Device/File not configured Device/File not configured Device/File not configured Cluster registry integrity check succeeded Logical corruption check succeeded 4. ????votedisk ,??????????: [grid@vrh1 ~]$ crsctl replace votedisk +SYSTEMDG CRS-4602: Failed 27 to add voting file 2e4e0fe285924f86bf5473d00dcc0388. CRS-4602: Failed 27 to add voting file 4fa54bb0cc5c4fafbf1a9be5479bf389. CRS-4602: Failed 27 to add voting file a109ead9ea4e4f28bfe233188623616a. CRS-4602: Failed 27 to add voting file 042c9fbd71b54f5abfcd3ab3408f3cf3. CRS-4602: Failed 27 to add voting file 7b5a8cd24f954fafbf835ad78615763f. Failed to replace voting disk group with +SYSTEMDG. CRS-4000: Command Replace failed, or completed with errors. ????????ASM???,???ASM: SQL> alter system set asm_diskstring='/dev/asm*'; System altered. SQL> create spfile from memory; File created. SQL> startup force mount; ORA-32004: obsolete or deprecated parameter(s) specified for ASM instance ASM instance started Total System Global Area 283930624 bytes Fixed Size 2227664 bytes Variable Size 256537136 bytes ASM Cache 25165824 bytes ASM diskgroups mounted SQL> show parameter spfile NAME TYPE VALUE ------------------------------------ ----------- ------------------------------ spfile string /g01/11.2.0/grid/dbs/spfile+AS M1.ora [grid@vrh1 trace]$ crsctl replace votedisk +SYSTEMDG CRS-4256: Updating the profile Successful addition of voting disk 85edc0e82d274f78bfc58cdc73b8c68a. Successful addition of voting disk 201ffffc8ba44faabfe2efec2aa75840. Successful addition of voting disk 6f2a25c589964faabf6980f7c5f621ce. Successful addition of voting disk 93eb315648454f25bf3717df1a2c73d5. Successful addition of voting disk 3737240678964f88bfbfbd31d8b3829f. Successfully replaced voting disk group with +SYSTEMDG. CRS-4256: Updating the profile CRS-4266: Voting file(s) successfully replaced 5. ??has??,??cluster????: [root@vrh1 ~]# crsctl check crs CRS-4638: Oracle High Availability Services is online CRS-4537: Cluster Ready Services is online CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online [root@vrh1 ~]# crsctl query css votedisk ## STATE File Universal Id File Name Disk group -- ----- ----------------- --------- --------- 1. ONLINE 85edc0e82d274f78bfc58cdc73b8c68a (/dev/asm-diskh) [SYSTEMDG] 2. ONLINE 201ffffc8ba44faabfe2efec2aa75840 (/dev/asm-diskg) [SYSTEMDG] 3. ONLINE 6f2a25c589964faabf6980f7c5f621ce (/dev/asm-diski) [SYSTEMDG] 4. ONLINE 93eb315648454f25bf3717df1a2c73d5 (/dev/asm-diskj) [SYSTEMDG] 5. ONLINE 3737240678964f88bfbfbd31d8b3829f (/dev/asm-diskk) [SYSTEMDG] Located 5 voting disk(s). [root@vrh1 ~]# crsctl stat res -t -------------------------------------------------------------------------------- NAME TARGET STATE SERVER STATE_DETAILS -------------------------------------------------------------------------------- Local Resources -------------------------------------------------------------------------------- ora.BACKUPDG.dg ONLINE ONLINE vrh1 ora.DATA.dg ONLINE ONLINE vrh1 ora.LISTENER.lsnr ONLINE ONLINE vrh1 ora.LSN_MACLEAN.lsnr ONLINE ONLINE vrh1 ora.SYSTEMDG.dg ONLINE ONLINE vrh1 ora.asm ONLINE ONLINE vrh1 Started ora.gsd OFFLINE OFFLINE vrh1 ora.net1.network ONLINE ONLINE vrh1 ora.ons ONLINE ONLINE vrh1 -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- ora.LISTENER_SCAN1.lsnr http://www.askmaclean.com 1 ONLINE ONLINE vrh1 ora.cvu 1 OFFLINE OFFLINE ora.oc4j 1 OFFLINE OFFLINE ora.scan1.vip 1 ONLINE ONLINE vrh1 ora.vprod.db 1 ONLINE OFFLINE 2 ONLINE OFFLINE ora.vrh1.vip 1 ONLINE ONLINE vrh1 ora.vrh2.vip 1 ONLINE INTERMEDIATE vrh1 FAILED OVER

  Read the article

• ??11.2 RAC??OCR?Votedisk??ASM Diskgroup?????

  - by Liu Maclean(???)

  ????????Oracle Allstarts??????????ocr?votedisk?ASM diskgroup??11gR2 RAC cluster?????????,????«?11gR2 RAC???ASM DISK Path????»??????,??????CRS??????11.2??ASM???????, ????????????”crsctl start crs -excl -nocrs “; ?????????,??ASM????ocr?????votedisk?????,??11.2????ocr?votedisk???ASM?,?ASM???????ocr?votedisk,?????ocr?votedisk????????cluter??????;???????????CRS????,?????diskgroup??????????,?????????????????? ??:?????????????????ASM LUN DISK,???OCR?????,????????4??????????,???????$GI_HOME,?????????;????votedisk?? ????: ??dd????ocr?votedisk??diskgroup header,??diskgroup corruption: 1. ??votedisk? ocr?? [root@vrh1 ~]# crsctl query css votedisk ## STATE File Universal Id File Name Disk group -- ----- ----------------- --------- --------- 1. ONLINE a853d6204bbc4feabfd8c73d4c3b3001 (/dev/asm-diskh) [SYSTEMDG] 2. ONLINE a5b37704c3574f0fbf21d1d9f58c4a6b (/dev/asm-diskg) [SYSTEMDG] 3. ONLINE 36e5c51ff0294fc3bf2a042266650331 (/dev/asm-diski) [SYSTEMDG] 4. ONLINE af337d1512824fe4bf6ad45283517aaa (/dev/asm-diskj) [SYSTEMDG] 5. ONLINE 3c4a349e2e304ff6bf64b2b1c9d9cf5d (/dev/asm-diskk) [SYSTEMDG] Located 5 voting disk(s). su - grid [grid@vrh1 ~]$ ocrconfig -showbackup PROT-26: Oracle Cluster Registry backup locations were retrieved from a local copy vrh1 2012/08/09 01:59:56 /g01/11.2.0/maclean/grid/cdata/vrh-cluster/backup00.ocr vrh1 2012/08/08 21:59:56 /g01/11.2.0/maclean/grid/cdata/vrh-cluster/backup01.ocr vrh1 2012/08/08 17:59:55 /g01/11.2.0/maclean/grid/cdata/vrh-cluster/backup02.ocr vrh1 2012/08/08 05:59:54 /g01/11.2.0/grid/cdata/vrh-cluster/day.ocr vrh1 2012/08/08 05:59:54 /g01/11.2.0/grid/cdata/vrh-cluster/week.ocr PROT-25: Manual backups for the Oracle Cluster Registry are not available 2. ??????????clusterware ,OHASD crsctl stop has -f 3. GetAsmDH.sh ==> GetAsmDH.sh?ASM disk header????? ????????,????????asm header [grid@vrh1 ~]$ ./GetAsmDH.sh ############################################ 1) Collecting Information About the Disks: ############################################ SQL*Plus: Release 11.2.0.3.0 Production on Thu Aug 9 03:28:13 2012 Copyright (c) 1982, 2011, Oracle. All rights reserved. SQL> Connected. SQL> SQL> SQL> SQL> SQL> SQL> SQL> 1 0 /dev/asm-diske 1 1 /dev/asm-diskd 2 0 /dev/asm-diskb 2 1 /dev/asm-diskc 2 2 /dev/asm-diskf 3 0 /dev/asm-diskh 3 1 /dev/asm-diskg 3 2 /dev/asm-diski 3 3 /dev/asm-diskj 3 4 /dev/asm-diskk SQL> SQL> Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production With the Real Application Clusters and Automatic Storage Management options -rw-r--r-- 1 grid oinstall 1048 Aug 9 03:28 /tmp/HC/asmdisks.lst ############################################ 2) Generating asm_diskh.sh script. ############################################ -rwx------ 1 grid oinstall 666 Aug 9 03:28 /tmp/HC/asm_diskh.sh ############################################ 3) Executing asm_diskh.sh script to generate dd dumps. ############################################ -rw-r--r-- 1 grid oinstall 1048576 Aug 9 03:28 /tmp/HC/dsk_1_0.dd -rw-r--r-- 1 grid oinstall 1048576 Aug 9 03:28 /tmp/HC/dsk_1_1.dd -rw-r--r-- 1 grid oinstall 1048576 Aug 9 03:28 /tmp/HC/dsk_2_0.dd -rw-r--r-- 1 grid oinstall 1048576 Aug 9 03:28 /tmp/HC/dsk_2_1.dd -rw-r--r-- 1 grid oinstall 1048576 Aug 9 03:28 /tmp/HC/dsk_2_2.dd -rw-r--r-- 1 grid oinstall 1048576 Aug 9 03:28 /tmp/HC/dsk_3_0.dd -rw-r--r-- 1 grid oinstall 1048576 Aug 9 03:28 /tmp/HC/dsk_3_1.dd -rw-r--r-- 1 grid oinstall 1048576 Aug 9 03:28 /tmp/HC/dsk_3_2.dd -rw-r--r-- 1 grid oinstall 1048576 Aug 9 03:28 /tmp/HC/dsk_3_3.dd -rw-r--r-- 1 grid oinstall 1048576 Aug 9 03:28 /tmp/HC/dsk_3_4.dd ############################################ 4) Compressing dd dumps in the next format: (asm_dd_header_all_.tar) ############################################ /tmp/HC/dsk_1_0.dd /tmp/HC/dsk_1_1.dd /tmp/HC/dsk_2_0.dd /tmp/HC/dsk_2_1.dd /tmp/HC/dsk_2_2.dd /tmp/HC/dsk_3_0.dd /tmp/HC/dsk_3_1.dd /tmp/HC/dsk_3_2.dd /tmp/HC/dsk_3_3.dd /tmp/HC/dsk_3_4.dd ./GetAsmDH.sh: line 81: compress: command not found ls: /tmp/HC/*.Z: No such file or directory [grid@vrh1 ~]$ 4. ??dd ?? ??ocr?votedisk??diskgroup [root@vrh1 ~]# dd if=/dev/zero of=/dev/asm-diskh bs=1024k count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00423853 seconds, 247 MB/s [root@vrh1 ~]# dd if=/dev/zero of=/dev/asm-diskg bs=1024k count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0045179 seconds, 232 MB/s [root@vrh1 ~]# dd if=/dev/zero of=/dev/asm-diski bs=1024k count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00469976 seconds, 223 MB/s [root@vrh1 ~]# dd if=/dev/zero of=/dev/asm-diskj bs=1024k count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00344262 seconds, 305 MB/s [root@vrh1 ~]# dd if=/dev/zero of=/dev/asm-diskk bs=1024k count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0053518 seconds, 196 MB/s 5. ????????????HAS [root@vrh1 ~]# crsctl start has CRS-4123: Oracle High Availability Services has been started. ????ocr?votedisk??diskgroup??,??CSS???????,???????: alertvrh1.log [cssd(5162)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /g01/11.2.0/grid/log/vrh1/cssd/ocssd.log 2012-08-09 03:35:41.207 [cssd(5162)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /g01/11.2.0/grid/log/vrh1/cssd/ocssd.log 2012-08-09 03:35:56.240 [cssd(5162)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /g01/11.2.0/grid/log/vrh1/cssd/ocssd.log 2012-08-09 03:36:11.284 [cssd(5162)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /g01/11.2.0/grid/log/vrh1/cssd/ocssd.log 2012-08-09 03:36:26.305 [cssd(5162)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /g01/11.2.0/grid/log/vrh1/cssd/ocssd.log 2012-08-09 03:36:41.328 ocssd.log 2012-08-09 03:40:26.662: [ CSSD][1078700352]clssnmReadDiscoveryProfile: voting file discovery string(/dev/asm*) 2012-08-09 03:40:26.662: [ CSSD][1078700352]clssnmvDDiscThread: using discovery string /dev/asm* for initial discovery 2012-08-09 03:40:26.662: [ SKGFD][1078700352]Discovery with str:/dev/asm*: 2012-08-09 03:40:26.662: [ SKGFD][1078700352]UFS discovery with :/dev/asm*: 2012-08-09 03:40:26.665: [ SKGFD][1078700352]Fetching UFS disk :/dev/asm-diskf: 2012-08-09 03:40:26.665: [ SKGFD][1078700352]Fetching UFS disk :/dev/asm-diskb: 2012-08-09 03:40:26.665: [ SKGFD][1078700352]Fetching UFS disk :/dev/asm-diskj: 2012-08-09 03:40:26.665: [ SKGFD][1078700352]Fetching UFS disk :/dev/asm-diskh: 2012-08-09 03:40:26.665: [ SKGFD][1078700352]Fetching UFS disk :/dev/asm-diskc: 2012-08-09 03:40:26.665: [ SKGFD][1078700352]Fetching UFS disk :/dev/asm-diskd: 2012-08-09 03:40:26.665: [ SKGFD][1078700352]Fetching UFS disk :/dev/asm-diske: 2012-08-09 03:40:26.665: [ SKGFD][1078700352]Fetching UFS disk :/dev/asm-diskg: 2012-08-09 03:40:26.665: [ SKGFD][1078700352]Fetching UFS disk :/dev/asm-diski: 2012-08-09 03:40:26.665: [ SKGFD][1078700352]Fetching UFS disk :/dev/asm-diskk: 2012-08-09 03:40:26.665: [ SKGFD][1078700352]OSS discovery with :/dev/asm*: 2012-08-09 03:40:26.665: [ SKGFD][1078700352]Handle 0xdf22a0 from lib :UFS:: for disk :/dev/asm-diskf: 2012-08-09 03:40:26.665: [ SKGFD][1078700352]Handle 0xf412a0 from lib :UFS:: for disk :/dev/asm-diskb: 2012-08-09 03:40:26.666: [ SKGFD][1078700352]Handle 0xf3a680 from lib :UFS:: for disk :/dev/asm-diskj: 2012-08-09 03:40:26.666: [ SKGFD][1078700352]Handle 0xf93da0 from lib :UFS:: for disk :/dev/asm-diskh: 2012-08-09 03:40:26.667: [ CSSD][1078700352]clssnmvDiskVerify: Successful discovery of 0 disks 2012-08-09 03:40:26.667: [ CSSD][1078700352]clssnmCompleteInitVFDiscovery: Completing initial voting file discovery 2012-08-09 03:40:26.667: [ CSSD][1078700352]clssnmvFindInitialConfigs: No voting files found 2012-08-09 03:40:26.667: [ CSSD][1078700352](:CSSNM00070:)clssnmCompleteInitVFDiscovery: Voting file not found. Retrying discovery in 15 seconds ?????ocr?votedisk??diskgroup?????: 1. ?-excl -nocrs ????cluster,??????ASM?? ????CRS [root@vrh1 vrh1]# crsctl start crs -excl -nocrs CRS-4123: Oracle High Availability Services has been started. CRS-2672: Attempting to start 'ora.mdnsd' on 'vrh1' CRS-2676: Start of 'ora.mdnsd' on 'vrh1' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'vrh1' CRS-2676: Start of 'ora.gpnpd' on 'vrh1' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'vrh1' CRS-2672: Attempting to start 'ora.gipcd' on 'vrh1' CRS-2676: Start of 'ora.cssdmonitor' on 'vrh1' succeeded CRS-2676: Start of 'ora.gipcd' on 'vrh1' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'vrh1' CRS-2672: Attempting to start 'ora.diskmon' on 'vrh1' CRS-2676: Start of 'ora.diskmon' on 'vrh1' succeeded CRS-2676: Start of 'ora.cssd' on 'vrh1' succeeded CRS-2679: Attempting to clean 'ora.cluster_interconnect.haip' on 'vrh1' CRS-2672: Attempting to start 'ora.ctssd' on 'vrh1' CRS-2681: Clean of 'ora.cluster_interconnect.haip' on 'vrh1' succeeded CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'vrh1' CRS-2676: Start of 'ora.ctssd' on 'vrh1' succeeded CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'vrh1' succeeded CRS-2672: Attempting to start 'ora.asm' on 'vrh1' CRS-2676: Start of 'ora.asm' on 'vrh1' succeeded 2.???ocr?votedisk??diskgroup,??compatible.asm???11.2: [root@vrh1 vrh1]# su - grid [grid@vrh1 ~]$ sqlplus / as sysasm SQL*Plus: Release 11.2.0.3.0 Production on Thu Aug 9 04:16:58 2012 Copyright (c) 1982, 2011, Oracle. All rights reserved. Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production With the Real Application Clusters and Automatic Storage Management options SQL> create diskgroup systemdg high redundancy disk '/dev/asm-diskh','/dev/asm-diskg','/dev/asm-diski','/dev/asm-diskj','/dev/asm-diskk' ATTRIBUTE 'compatible.rdbms' = '11.2', 'compatible.asm' = '11.2'; 3.?ocr backup???ocr??ocrcheck??: [root@vrh1 ~]# ocrconfig -restore /g01/11.2.0/grid/cdata/vrh-cluster/backup00.ocr [root@vrh1 ~]# ocrcheck Status of Oracle Cluster Registry is as follows : Version : 3 Total space (kbytes) : 262120 Used space (kbytes) : 3180 Available space (kbytes) : 258940 ID : 1238458014 Device/File Name : +systemdg Device/File integrity check succeeded Device/File not configured Device/File not configured Device/File not configured Device/File not configured Cluster registry integrity check succeeded Logical corruption check succeeded 4. ????votedisk ,??????????: [grid@vrh1 ~]$ crsctl replace votedisk +SYSTEMDG CRS-4602: Failed 27 to add voting file 2e4e0fe285924f86bf5473d00dcc0388. CRS-4602: Failed 27 to add voting file 4fa54bb0cc5c4fafbf1a9be5479bf389. CRS-4602: Failed 27 to add voting file a109ead9ea4e4f28bfe233188623616a. CRS-4602: Failed 27 to add voting file 042c9fbd71b54f5abfcd3ab3408f3cf3. CRS-4602: Failed 27 to add voting file 7b5a8cd24f954fafbf835ad78615763f. Failed to replace voting disk group with +SYSTEMDG. CRS-4000: Command Replace failed, or completed with errors. ????????ASM???,???ASM: SQL> alter system set asm_diskstring='/dev/asm*'; System altered. SQL> create spfile from memory; File created. SQL> startup force mount; ORA-32004: obsolete or deprecated parameter(s) specified for ASM instance ASM instance started Total System Global Area 283930624 bytes Fixed Size 2227664 bytes Variable Size 256537136 bytes ASM Cache 25165824 bytes ASM diskgroups mounted SQL> show parameter spfile NAME TYPE VALUE ------------------------------------ ----------- ------------------------------ spfile string /g01/11.2.0/grid/dbs/spfile+AS M1.ora [grid@vrh1 trace]$ crsctl replace votedisk +SYSTEMDG CRS-4256: Updating the profile Successful addition of voting disk 85edc0e82d274f78bfc58cdc73b8c68a. Successful addition of voting disk 201ffffc8ba44faabfe2efec2aa75840. Successful addition of voting disk 6f2a25c589964faabf6980f7c5f621ce. Successful addition of voting disk 93eb315648454f25bf3717df1a2c73d5. Successful addition of voting disk 3737240678964f88bfbfbd31d8b3829f. Successfully replaced voting disk group with +SYSTEMDG. CRS-4256: Updating the profile CRS-4266: Voting file(s) successfully replaced 5. ??has??,??cluster????: [root@vrh1 ~]# crsctl check crs CRS-4638: Oracle High Availability Services is online CRS-4537: Cluster Ready Services is online CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online [root@vrh1 ~]# crsctl query css votedisk ## STATE File Universal Id File Name Disk group -- ----- ----------------- --------- --------- 1. ONLINE 85edc0e82d274f78bfc58cdc73b8c68a (/dev/asm-diskh) [SYSTEMDG] 2. ONLINE 201ffffc8ba44faabfe2efec2aa75840 (/dev/asm-diskg) [SYSTEMDG] 3. ONLINE 6f2a25c589964faabf6980f7c5f621ce (/dev/asm-diski) [SYSTEMDG] 4. ONLINE 93eb315648454f25bf3717df1a2c73d5 (/dev/asm-diskj) [SYSTEMDG] 5. ONLINE 3737240678964f88bfbfbd31d8b3829f (/dev/asm-diskk) [SYSTEMDG] Located 5 voting disk(s). [root@vrh1 ~]# crsctl stat res -t -------------------------------------------------------------------------------- NAME TARGET STATE SERVER STATE_DETAILS -------------------------------------------------------------------------------- Local Resources -------------------------------------------------------------------------------- ora.BACKUPDG.dg ONLINE ONLINE vrh1 ora.DATA.dg ONLINE ONLINE vrh1 ora.LISTENER.lsnr ONLINE ONLINE vrh1 ora.LSN_MACLEAN.lsnr ONLINE ONLINE vrh1 ora.SYSTEMDG.dg ONLINE ONLINE vrh1 ora.asm ONLINE ONLINE vrh1 Started ora.gsd OFFLINE OFFLINE vrh1 ora.net1.network ONLINE ONLINE vrh1 ora.ons ONLINE ONLINE vrh1 -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- ora.LISTENER_SCAN1.lsnr http://www.askmaclean.com 1 ONLINE ONLINE vrh1 ora.cvu 1 OFFLINE OFFLINE ora.oc4j 1 OFFLINE OFFLINE ora.scan1.vip 1 ONLINE ONLINE vrh1 ora.vprod.db 1 ONLINE OFFLINE 2 ONLINE OFFLINE ora.vrh1.vip 1 ONLINE ONLINE vrh1 ora.vrh2.vip 1 ONLINE INTERMEDIATE vrh1 FAILED OVER

  Read the article

• Openvpn plugin openvpn-auth-ldap does not bind to Active Directory

  - by Selivanov Pavel

  I'm trying to configure OpenVPN with openvpn-auth-ldap plugin to authorize users via Active Directory LDAP. When I use the same server config without plugin option, and add client config with generated client key and cert, connection is successful, so problem is in the plugin. server.conf: plugin /usr/lib/openvpn/openvpn-auth-ldap.so "/etc/openvpn-test/openvpn-auth-ldap.conf" port 1194 proto tcp dev tun keepalive 10 60 topology subnet server 10.0.2.0 255.255.255.0 tls-server ca ca.crt dh dh1024.pem cert server.crt key server.key #crl-verify crl.pem persist-key persist-tun user nobody group nogroup verb 3 mute 20 openvpn-auth-ldap.conf: <LDAP> URL ldap://dc1.domain:389 TLSEnable no BindDN cn=bot_auth,cn=Users,dc=domain Password bot_auth Timeout 15 FollowReferrals yes </LDAP> <Authorization> BaseDN "cn=Users,dc=domain" SearchFilter "(sAMAccountName=%u)" RequireGroup false # <Group> # BaseDN "ou=groups,dc=mycompany,dc=local" # SearchFilter "(|(cn=developers)(cn=artists))" # MemberAttribute uniqueMember # </Group> </Authorization> Top-level domain in AD is used by historical reasons. Analogue configuration is working for Apache 2.2 in mod-authzn-ldap. User and password are correct. client.conf: remote server_name port 1194 proto tcp client pull remote-cert-tls server dev tun resolv-retry infinite nobind ca ca.crt ; with keys - works fine #cert test.crt #key test.key ; without keys - by password auth-user-pass persist-tun verb 3 mute 20 In server log there is string PLUGIN_INIT: POST /usr/lib/openvpn/openvpn-auth-ldap.so '[/usr/lib/openvpn/openvpn-auth-ldap.so] [/etc/openvpn-test/openvpn-auth-ldap.conf]' which indicates, that plugin failed. I can telnet to dc1.domain:389, so this is not network/firewall problem. Later server says TLS Error: TLS object -> incoming plaintext read error TLS handshake failed - without plugin it tryes to do usal key authentification. server log: Tue Nov 22 03:06:20 2011 OpenVPN 2.1.3 i486-pc-linux-gnu [SSL] [LZO2] [EPOLL] [PKCS11] [MH] [PF_INET6] [eurephia] built on Oct 21 2010 Tue Nov 22 03:06:20 2011 NOTE: OpenVPN 2.1 requires '--script-security 2' or higher to call user-defined scripts or executables Tue Nov 22 03:06:20 2011 PLUGIN_INIT: POST /usr/lib/openvpn/openvpn-auth-ldap.so '[/usr/lib/openvpn/openvpn-auth-ldap.so] [/etc/openvpn-test/openvpn-auth-ldap.conf]' intercepted=PLUGIN_AUTH_USER_PASS_VERIFY|PLUGIN_CLIENT_CONNECT|PLUGIN_CLIENT_DISCONNECT Tue Nov 22 03:06:20 2011 Diffie-Hellman initialized with 1024 bit key Tue Nov 22 03:06:20 2011 /usr/bin/openssl-vulnkey -q -b 1024 -m <modulus omitted> Tue Nov 22 03:06:20 2011 Control Channel Authentication: using 'ta.key' as a OpenVPN static key file Tue Nov 22 03:06:20 2011 Outgoing Control Channel Authentication: Using 160 bit message hash 'SHA1' for HMAC authentication Tue Nov 22 03:06:20 2011 Incoming Control Channel Authentication: Using 160 bit message hash 'SHA1' for HMAC authentication Tue Nov 22 03:06:20 2011 TLS-Auth MTU parms [ L:1543 D:168 EF:68 EB:0 ET:0 EL:0 ] Tue Nov 22 03:06:20 2011 Socket Buffers: R=[87380->131072] S=[16384->131072] Tue Nov 22 03:06:20 2011 TUN/TAP device tun1 opened Tue Nov 22 03:06:20 2011 TUN/TAP TX queue length set to 100 Tue Nov 22 03:06:20 2011 /sbin/ifconfig tun1 10.0.2.1 netmask 255.255.255.0 mtu 1500 broadcast 10.0.2.255 Tue Nov 22 03:06:20 2011 Data Channel MTU parms [ L:1543 D:1450 EF:43 EB:4 ET:0 EL:0 ] Tue Nov 22 03:06:20 2011 GID set to nogroup Tue Nov 22 03:06:20 2011 UID set to nobody Tue Nov 22 03:06:20 2011 Listening for incoming TCP connection on [undef] Tue Nov 22 03:06:20 2011 TCPv4_SERVER link local (bound): [undef] Tue Nov 22 03:06:20 2011 TCPv4_SERVER link remote: [undef] Tue Nov 22 03:06:20 2011 MULTI: multi_init called, r=256 v=256 Tue Nov 22 03:06:20 2011 IFCONFIG POOL: base=10.0.2.2 size=252 Tue Nov 22 03:06:20 2011 MULTI: TCP INIT maxclients=1024 maxevents=1028 Tue Nov 22 03:06:20 2011 Initialization Sequence Completed Tue Nov 22 03:07:10 2011 MULTI: multi_create_instance called Tue Nov 22 03:07:10 2011 Re-using SSL/TLS context Tue Nov 22 03:07:10 2011 Control Channel MTU parms [ L:1543 D:168 EF:68 EB:0 ET:0 EL:0 ] Tue Nov 22 03:07:10 2011 Data Channel MTU parms [ L:1543 D:1450 EF:43 EB:4 ET:0 EL:0 ] Tue Nov 22 03:07:10 2011 Local Options hash (VER=V4): 'c413e92e' Tue Nov 22 03:07:10 2011 Expected Remote Options hash (VER=V4): 'd8421bb0' Tue Nov 22 03:07:10 2011 TCP connection established with [AF_INET]10.0.0.9:47808 Tue Nov 22 03:07:10 2011 TCPv4_SERVER link local: [undef] Tue Nov 22 03:07:10 2011 TCPv4_SERVER link remote: [AF_INET]10.0.0.9:47808 Tue Nov 22 03:07:11 2011 10.0.0.9:47808 TLS: Initial packet from [AF_INET]10.0.0.9:47808, sid=a2cd4052 84b47108 Tue Nov 22 03:07:11 2011 10.0.0.9:47808 TLS_ERROR: BIO read tls_read_plaintext error: error:140890C7:SSL routines:SSL3_GET_CLIENT_CERTIFICATE:peer did not return a certificate Tue Nov 22 03:07:11 2011 10.0.0.9:47808 TLS Error: TLS object -> incoming plaintext read error Tue Nov 22 03:07:11 2011 10.0.0.9:47808 TLS Error: TLS handshake failed Tue Nov 22 03:07:11 2011 10.0.0.9:47808 Fatal TLS error (check_tls_errors_co), restarting Tue Nov 22 03:07:11 2011 10.0.0.9:47808 SIGUSR1[soft,tls-error] received, client-instance restarting Tue Nov 22 03:07:11 2011 TCP/UDP: Closing socket client log: Tue Nov 22 03:06:18 2011 OpenVPN 2.1.3 x86_64-pc-linux-gnu [SSL] [LZO2] [EPOLL] [PKCS11] [MH] [PF_INET6] [eurephia] built on Oct 22 2010 Enter Auth Username:user Enter Auth Password: Tue Nov 22 03:06:25 2011 NOTE: OpenVPN 2.1 requires '--script-security 2' or higher to call user-defined scripts or executables Tue Nov 22 03:06:25 2011 Control Channel Authentication: using 'ta.key' as a OpenVPN static key file Tue Nov 22 03:06:25 2011 Outgoing Control Channel Authentication: Using 160 bit message hash 'SHA1' for HMAC authentication Tue Nov 22 03:06:25 2011 Incoming Control Channel Authentication: Using 160 bit message hash 'SHA1' for HMAC authentication Tue Nov 22 03:06:25 2011 Control Channel MTU parms [ L:1543 D:168 EF:68 EB:0 ET:0 EL:0 ] Tue Nov 22 03:06:25 2011 Socket Buffers: R=[87380->131072] S=[16384->131072] Tue Nov 22 03:06:25 2011 Data Channel MTU parms [ L:1543 D:1450 EF:43 EB:4 ET:0 EL:0 ] Tue Nov 22 03:06:25 2011 Local Options hash (VER=V4): 'd8421bb0' Tue Nov 22 03:06:25 2011 Expected Remote Options hash (VER=V4): 'c413e92e' Tue Nov 22 03:06:25 2011 Attempting to establish TCP connection with [AF_INET]10.0.0.2:1194 [nonblock] Tue Nov 22 03:06:26 2011 TCP connection established with [AF_INET]10.0.0.2:1194 Tue Nov 22 03:06:26 2011 TCPv4_CLIENT link local: [undef] Tue Nov 22 03:06:26 2011 TCPv4_CLIENT link remote: [AF_INET]10.0.0.2:1194 Tue Nov 22 03:06:26 2011 TLS: Initial packet from [AF_INET]10.0.0.2:1194, sid=7a3c2a0f bd35bca7 Tue Nov 22 03:06:26 2011 WARNING: this configuration may cache passwords in memory -- use the auth-nocache option to prevent this Tue Nov 22 03:06:26 2011 VERIFY OK: depth=1, /C=US/ST=CA/L=SanFrancisco/O=Fort-Funston/CN=Fort-Funston_CA/[email protected] Tue Nov 22 03:06:26 2011 Validating certificate key usage Tue Nov 22 03:06:26 2011 ++ Certificate has key usage 00a0, expects 00a0 Tue Nov 22 03:06:26 2011 VERIFY KU OK Tue Nov 22 03:06:26 2011 Validating certificate extended key usage Tue Nov 22 03:06:26 2011 ++ Certificate has EKU (str) TLS Web Server Authentication, expects TLS Web Server Authentication Tue Nov 22 03:06:26 2011 VERIFY EKU OK Tue Nov 22 03:06:26 2011 VERIFY OK: depth=0, /C=US/ST=CA/L=SanFrancisco/O=Fort-Funston/CN=server/[email protected] Tue Nov 22 03:06:26 2011 Connection reset, restarting [0] Tue Nov 22 03:06:26 2011 TCP/UDP: Closing socket Tue Nov 22 03:06:26 2011 SIGUSR1[soft,connection-reset] received, process restarting Tue Nov 22 03:06:26 2011 Restart pause, 5 second(s) ^CTue Nov 22 03:06:27 2011 SIGINT[hard,init_instance] received, process exiting Does anybody know how to get openvpn-auth-ldap wirking?

  Read the article

• Global name not defined

  - by anteater7171

  I wrote a CPU monitoring program in Python. For some reason sometimes the the program will run without any problem. Then other times the program won't even start because of the following error. Traceback (most recent call last): File "", line 244, in run_nodebug File "C:\Python26\CPUR1.7.pyw", line 601, in app = simpleapp_tk(None) File "C:\Python26\CPUR1.7.pyw", line 26, in init self.initialize() File "C:\Python26\CPUR1.7.pyw", line 107, in initialize self.F() File "C:\Python26\CPUR1.7.pyw", line 517, in F S2 = TL.entryVariableS.get() NameError: global name 'TL' is not defined I can't seem to find the problem, maybe someone more experienced may assist me? Here is a snippet of the part giving me trouble: (The second to last line in the snippet is what's giving me trouble) def E(self): if self.selectedM.get() =='Options...': Setup global TL TL = Tkinter.Toplevel(self) menu = Tkinter.Menu(TL) TL.config(menu=menu) filemenu = Tkinter.Menu(menu) menu.add_cascade(label="| Menu |", menu=filemenu) filemenu.add_command(label="Instruction Manual...", command=self.helpmenu) filemenu.add_command(label="About...", command=self.aboutmenu) filemenu.add_separator() filemenu.add_command(label="Exit Options", command=TL.destroy) filemenu.add_command(label="Exit", command=self.destroy) helpmenu = Tkinter.Menu(menu) menu.add_cascade(label="| Help |", menu=helpmenu) helpmenu.add_command(label="Instruction Manual...", command=self.helpmenu) helpmenu.add_separator() helpmenu.add_command(label="Quick Help...", command=self.helpmenu) Title TL.label5 = Tkinter.Label(TL,text="CPU Usage: Options",anchor="center",fg="black",bg="lightgreen",relief="ridge",borderwidth=5,font=('Arial', 18, 'bold')) TL.label5.pack(padx=15,ipadx=5) X Y scale TL.separator = Tkinter.Frame(TL,height=7, bd=1, relief='ridge', bg='grey95') TL.separator.pack(pady=5,padx=5) # TL.sclX = Tkinter.Scale(TL.separator, from_=0, to=1500, orient='horizontal', resolution=1, command=self.A) TL.sclX.grid(column=1,row=0,ipadx=27, sticky='w') TL.label1 = Tkinter.Label(TL.separator,text="X",anchor="s",fg="black",bg="grey95",font=('Arial', 8 ,'bold')) TL.label1.grid(column=0,row=0, pady=1, sticky='S') TL.sclY = Tkinter.Scale(TL.separator, from_=0, to=1500, resolution=1, command=self.A) TL.sclY.grid(column=2,row=1,rowspan=2,sticky='e', padx=4) TL.label3 = Tkinter.Label(TL.separator,text="Y",fg="black",bg="grey95",font=('Arial', 8 ,'bold')) TL.label3.grid(column=2,row=0, padx=10, sticky='e') TL.entryVariable2 = Tkinter.StringVar() TL.entry2 = Tkinter.Entry(TL.separator,textvariable=TL.entryVariable2, fg="grey15",bg="grey90",relief="sunken",insertbackground="black",borderwidth=5,font=('Arial', 10)) TL.entry2.grid(column=1,row=1,ipadx=20, pady=10,sticky='EW') TL.entry2.bind("<Return>", self.B) TL.label2 = Tkinter.Label(TL.separator,text="X:",fg="black",bg="grey95",font=('Arial', 8 ,'bold')) TL.label2.grid(column=0,row=1, ipadx=4, sticky='W') TL.entryVariable1 = Tkinter.StringVar() TL.entry1 = Tkinter.Entry(TL.separator,textvariable=TL.entryVariable1, fg="grey15",bg="grey90",relief="sunken",insertbackground="black",borderwidth=5,font=('Arial', 10)) TL.entry1.grid(column=1,row=2,sticky='EW') TL.entry1.bind("<Return>", self.B) TL.label4 = Tkinter.Label(TL.separator,text="Y:", anchor="center",fg="black",bg="grey95",font=('Arial', 8 ,'bold')) TL.label4.grid(column=0,row=2, ipadx=4, sticky='W') TL.label7 = Tkinter.Label(TL.separator,text="Text Colour:",fg="black",bg="grey95",font=('Arial', 8 ,'bold'),justify='left') TL.label7.grid(column=1,row=3, sticky='W',padx=10,ipady=10,ipadx=30) TL.selectedP = Tkinter.StringVar() TL.opt1 = Tkinter.OptionMenu(TL.separator, TL.selectedP,'Normal', 'White','Black', 'Blue', 'Steel Blue','Green','Light Green','Yellow','Orange' ,'Red',command=self.G) TL.opt1.config(fg="black",bg="grey90",activebackground="grey90",activeforeground="black", anchor="center",relief="raised",direction='right',font=('Arial', 10)) TL.opt1.grid(column=1,row=4,sticky='EW',padx=20,ipadx=20) TL.selectedP.set('Normal') TL.sclS = Tkinter.Scale(TL.separator, from_=10, to=2000, orient='horizontal', resolution=10, command=self.H) TL.sclS.grid(column=1,row=5,ipadx=27, sticky='w') TL.sclS.set(600) TL.entryVariableS = Tkinter.StringVar() TL.entryS = Tkinter.Entry(TL.separator,textvariable=TL.entryVariableS, fg="grey15",bg="grey90",relief="sunken",insertbackground="black",borderwidth=5,font=('Arial', 10)) TL.entryS.grid(column=1,row=6,ipadx=20, pady=10,sticky='EW') TL.entryS.bind("<Return>", self.I) TL.entryVariableS.set(600) # TL.resizable(False,False) TL.title('Options') geomPatt = re.compile(r"(\d+)?x?(\d+)?([+-])(\d+)([+-])(\d+)") s = self.wm_geometry() m = geomPatt.search(s) X = m.group(4) Y = m.group(6) TL.sclY.set(Y) TL.sclX.set(X) if self.selectedM.get() == 'Exit': self.destroy() def F (self): G = round(psutil.cpu_percent(), 1) G1 = str(G) + '%' self.labelVariable.set(G1) if G < 5: self.imageLabel.configure(image=self.image0) if G >= 5: self.imageLabel.configure(image=self.image5) if G >= 10: self.imageLabel.configure(image=self.image10) if G >= 15: self.imageLabel.configure(image=self.image15) if G >= 20: self.imageLabel.configure(image=self.image20) if G >= 25: self.imageLabel.configure(image=self.image25) if G >= 30: self.imageLabel.configure(image=self.image30) if G >= 35: self.imageLabel.configure(image=self.image35) if G >= 40: self.imageLabel.configure(image=self.image40) if G >= 45: self.imageLabel.configure(image=self.image45) if G >= 50: self.imageLabel.configure(image=self.image50) if G >= 55: self.imageLabel.configure(image=self.image55) if G >= 60: self.imageLabel.configure(image=self.image60) if G >= 65: self.imageLabel.configure(image=self.image65) if G >= 70: self.imageLabel.configure(image=self.image70) if G >= 75: self.imageLabel.configure(image=self.image75) if G >= 80: self.imageLabel.configure(image=self.image80) if G >= 85: self.imageLabel.configure(image=self.image85) if G >= 90: self.imageLabel.configure(image=self.image90) if 100> G >= 95: self.imageLabel.configure(image=self.image95) if G == 100: self.imageLabel.configure(image=self.image100) S2 = TL.entryVariableS.get() self.after(int(S2), self.F)

  Read the article

• Finding all the shortest paths between two nodes in unweighted directed graphs using BFS algorithm

  - by andra-isan

  Hi All, I am working on a problem that I need to find all the shortest path between two nodes in a given directed unweighted graph. I have used BFS algorithm to do the job, but unfortunately I can only print one shortest path not all of them, for example if they are 4 paths having lenght 3, my algorithm only prints the first one but I would like it to print all the four shortest paths. I was wondering in the following code, how should I change it so that all the shortest paths between two nodes could be printed out? class graphNode{ public: int id; string name; bool status; double weight;}; map<int, map<int,graphNode>* > graph; int Graph::BFS(graphNode &v, graphNode &w){ queue <int> q; map <int, int> map1; // this is to check if the node has been visited or not. std::string str= ""; map<int,int> inQ; // just to check that we do not insert the same iterm twice in the queue map <int, map<int, graphNode>* >::iterator pos; pos = graph.find(v.id); if(pos == graph.end()) { cout << v.id << " does not exists in the graph " <<endl; return 1; } int parents[graph.size()+1]; // this vector keeps track of the parents for the node parents[v.id] = -1; // there is a direct path between these two words, simply print that path as the shortest path if (findDirectEdge(v.id,w.id) == 1 ){ cout << " Shortest Path: " << v.id << " -> " << w.id << endl; return 1; } //if else{ int gn; map <int, map<int, graphNode>* >::iterator pos; q.push(v.id); inQ.insert(make_pair(v.id, v.id)); while (!q.empty()){ gn = q.front(); q.pop(); map<int, int>::iterator it; cout << " Popping: " << gn <<endl; map1.insert(make_pair(gn,gn)); //backtracing to print all the nodes if gn is the same as our target node such as w.id if (gn == w.id){ int current = w.id; cout << current << " - > "; while (current!=v.id){ current = parents[current]; cout << current << " -> "; } cout <<endl; } if ((pos = graph.find(gn)) == graph.end()) { cout << " pos is empty " <<endl; continue; } map<int, graphNode>* pn = pos->second; map<int, graphNode>::iterator p = pn->begin(); while(p != pn->end()) { map<int, int>::iterator it; //map1 keeps track of the visited nodes it = map1.find(p->first); graphNode gn1= p->second; if (it== map1.end()) { map<int, int>::iterator it1; //if the node already exits in the inQ, we do not insert it twice it1 = inQ.find(p->first); if (it1== inQ.end()){ parents[p->first] = gn; cout << " inserting " << p->first << " into the queue " <<endl; q.push(p->first); // add it to the queue } //if } //if p++; } //while } //while } I do appreciate all your great help Thanks, Andra

  Read the article

• Sorting a file with 55K rows and varying Columns

  - by Prasad

  Hi I want to find a programmatic solution using C++. I have a 900 files each of 27MB size. (just to inform about the enormity ). Each file has 55K rows and Varying columns. But the header indicates the columns I want to sort the rows in an order w.r.t to a Column Value. I wrote the sorting algorithm for this (definitely my newbie attempts, you may say). This algorithm is working for few numbers, but fails for larger numbers. Here is the code for the same: basic functions I defined to use inside the main code: int getNumberOfColumns(const string& aline) { int ncols=0; istringstream ss(aline); string s1; while(ss>>s1) ncols++; return ncols; } vector<string> getWordsFromSentence(const string& aline) { vector<string>words; istringstream ss(aline); string tstr; while(ss>>tstr) words.push_back(tstr); return words; } bool findColumnName(vector<string> vs, const string& colName) { vector<string>::iterator it = find(vs.begin(), vs.end(), colName); if ( it != vs.end()) return true; else return false; } int getIndexForColumnName(vector<string> vs, const string& colName) { if ( !findColumnName(vs,colName) ) return -1; else { vector<string>::iterator it = find(vs.begin(), vs.end(), colName); return it - vs.begin(); } } ////////// I like the Recurssive functions - I tried to create a recursive function ///here. This worked for small values , say 20 rows. But for 55K - core dumps void sort2D(vector<string>vn, vector<string> &srt, int columnIndex) { vector<double> pVals; for ( int i = 0; i < vn.size(); i++) { vector<string>meancols = getWordsFromSentence(vn[i]); pVals.push_back(stringToDouble(meancols[columnIndex])); } srt.push_back(vn[max_element(pVals.begin(), pVals.end())-pVals.begin()]); if (vn.size() > 1 ) { vn.erase(vn.begin()+(max_element(pVals.begin(), pVals.end())-pVals.begin()) ); vector<string> vn2 = vn; //cout<<srt[srt.size() -1 ]<<endl; sort2D(vn2 , srt, columnIndex); } } Now the main code: for ( int i = 0; i < TissueNames.size() -1; i++) { for ( int j = i+1; j < TissueNames.size(); j++) { //string fname = path+"/gse7307_Female_rma"+TissueNames[i]+"_"+TissueNames[j]+".txt"; //string fname2 = sortpath2+"/gse7307_Female_rma"+TissueNames[i]+"_"+TissueNames[j]+"Sorted.txt"; string fname = path+"/gse7307_Male_rma"+TissueNames[i]+"_"+TissueNames[j]+".txt"; string fname2 = sortpath2+"/gse7307_Male_rma"+TissueNames[i]+"_"+TissueNames[j]+"4Columns.txt"; //vector<string>AllLinesInFile; BioInputStream fin(fname); string aline; getline(fin,aline); replace (aline.begin(), aline.end(), '"',' '); string headerline = aline; vector<string> header = getWordsFromSentence(aline); int pindex = getIndexForColumnName(header,"p-raw"); int xcindex = getIndexForColumnName(header,"xC"); int xeindex = getIndexForColumnName(header,"xE"); int prbindex = getIndexForColumnName(header,"X"); string newheaderline = "X\txC\txE\tp-raw"; BioOutputStream fsrt(fname2); fsrt<<newheaderline<<endl; int newpindex=3; while ( getline(fin, aline) ){ replace (aline.begin(), aline.end(), '"',' '); istringstream ss2(aline); string tstr; ss2>>tstr; tstr = ss2.str().substr(tstr.length()+1); vector<string> words = getWordsFromSentence(tstr); string values = words[prbindex]+"\t"+words[xcindex]+"\t"+words[xeindex]+"\t"+words[pindex]; AllLinesInFile.push_back(values); } vector<string>SortedLines; sort2D(AllLinesInFile, SortedLines,newpindex); for ( int si = 0; si < SortedLines.size(); si++) fsrt<<SortedLines[si]<<endl; cout<<"["<<i<<","<<j<<"] = "<<SortedLines.size()<<endl; } } can some one suggest me a better way of doing this? why it is failing for larger values. ? The primary function of interest for this query is Sort2D function. thanks for the time and patience. prasad.

  Read the article

• Python hashable dicts

  - by TokenMacGuy

  As an exercise, and mostly for my own amusement, I'm implementing a backtracking packrat parser. The inspiration for this is i'd like to have a better idea about how hygenic macros would work in an algol-like language (as apposed to the syntax free lisp dialects you normally find them in). Because of this, different passes through the input might see different grammars, so cached parse results are invalid, unless I also store the current version of the grammar along with the cached parse results. (EDIT: a consequence of this use of key-value collections is that they should be immutable, but I don't intend to expose the interface to allow them to be changed, so either mutable or immutable collections are fine) The problem is that python dicts cannot appear as keys to other dicts. Even using a tuple (as I'd be doing anyways) doesn't help. >>> cache = {} >>> rule = {"foo":"bar"} >>> cache[(rule, "baz")] = "quux" Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unhashable type: 'dict' >>> I guess it has to be tuples all the way down. Now the python standard library provides approximately what i'd need, collections.namedtuple has a very different syntax, but can be used as a key. continuing from above session: >>> from collections import namedtuple >>> Rule = namedtuple("Rule",rule.keys()) >>> cache[(Rule(**rule), "baz")] = "quux" >>> cache {(Rule(foo='bar'), 'baz'): 'quux'} Ok. But I have to make a class for each possible combination of keys in the rule I would want to use, which isn't so bad, because each parse rule knows exactly what parameters it uses, so that class can be defined at the same time as the function that parses the rule. But combining the rules together is much more dynamic. In particular, I'd like a simple way to have rules override other rules, but collections.namedtuple has no analogue to dict.update(). Edit: An additional problem with namedtuples is that they are strictly positional. Two tuples that look like they should be different can in fact be the same: >>> you = namedtuple("foo",["bar","baz"]) >>> me = namedtuple("foo",["bar","quux"]) >>> you(bar=1,baz=2) == me(bar=1,quux=2) True >>> bob = namedtuple("foo",["baz","bar"]) >>> you(bar=1,baz=2) == bob(bar=1,baz=2) False tl'dr: How do I get dicts that can be used as keys to other dicts? Having hacked a bit on the answers, here's the more complete solution I'm using. Note that this does a bit extra work to make the resulting dicts vaguely immutable for practical purposes. Of course it's still quite easy to hack around it by calling dict.__setitem__(instance, key, value) but we're all adults here. class hashdict(dict): """ hashable dict implementation, suitable for use as a key into other dicts. >>> h1 = hashdict({"apples": 1, "bananas":2}) >>> h2 = hashdict({"bananas": 3, "mangoes": 5}) >>> h1+h2 hashdict(apples=1, bananas=3, mangoes=5) >>> d1 = {} >>> d1[h1] = "salad" >>> d1[h1] 'salad' >>> d1[h2] Traceback (most recent call last): ... KeyError: hashdict(bananas=3, mangoes=5) based on answers from http://stackoverflow.com/questions/1151658/python-hashable-dicts """ def __key(self): return tuple(sorted(self.items())) def __repr__(self): return "{0}({1})".format(self.__class__.__name__, ", ".join("{0}={1}".format( str(i[0]),repr(i[1])) for i in self.__key())) def __hash__(self): return hash(self.__key()) def __setitem__(self, key, value): raise TypeError("{0} does not support item assignment" .format(self.__class__.__name__)) def __delitem__(self, key): raise TypeError("{0} does not support item assignment" .format(self.__class__.__name__)) def clear(self): raise TypeError("{0} does not support item assignment" .format(self.__class__.__name__)) def pop(self, *args, **kwargs): raise TypeError("{0} does not support item assignment" .format(self.__class__.__name__)) def popitem(self, *args, **kwargs): raise TypeError("{0} does not support item assignment" .format(self.__class__.__name__)) def setdefault(self, *args, **kwargs): raise TypeError("{0} does not support item assignment" .format(self.__class__.__name__)) def update(self, *args, **kwargs): raise TypeError("{0} does not support item assignment" .format(self.__class__.__name__)) def __add__(self, right): result = hashdict(self) dict.update(result, right) return result if __name__ == "__main__": import doctest doctest.testmod()

  Read the article

• Signals and threads - good or bad design decision?

  - by Jens

  I have to write a program that performs highly computationally intensive calculations. The program might run for several days. The calculation can be separated easily in different threads without the need of shared data. I want a GUI or a web service that informs me of the current status. My current design uses BOOST::signals2 and BOOST::thread. It compiles and so far works as expected. If a thread finished one iteration and new data is available it calls a signal which is connected to a slot in the GUI class. My question(s): Is this combination of signals and threads a wise idea? I another forum somebody advised someone else not to "go down this road". Are there potential deadly pitfalls nearby that I failed to see? Is my expectation realistic that it will be "easy" to use my GUI class to provide a web interface or a QT, a VTK or a whatever window? Is there a more clever alternative (like other boost libs) that I overlooked? following code compiles with g++ -Wall -o main -lboost_thread-mt <filename>.cpp code follows: #include <boost/signals2.hpp> #include <boost/thread.hpp> #include <boost/bind.hpp> #include <iostream> #include <iterator> #include <string> using std::cout; using std::cerr; using std::string; /** * Called when a CalcThread finished a new bunch of data. */ boost::signals2::signal<void(string)> signal_new_data; /** * The whole data will be stored here. */ class DataCollector { typedef boost::mutex::scoped_lock scoped_lock; boost::mutex mutex; public: /** * Called by CalcThreads call the to store their data. */ void push(const string &s, const string &caller_name) { scoped_lock lock(mutex); _data.push_back(s); signal_new_data(caller_name); } /** * Output everything collected so far to std::out. */ void out() { typedef std::vector<string>::const_iterator iter; for (iter i = _data.begin(); i != _data.end(); ++i) cout << " " << *i << "\n"; } private: std::vector<string> _data; }; /** * Several of those can calculate stuff. * No data sharing needed. */ struct CalcThread { CalcThread(string name, DataCollector &datcol) : _name(name), _datcol(datcol) { } /** * Expensive algorithms will be implemented here. * @param num_results how many data sets are to be calculated by this thread. */ void operator()(int num_results) { for (int i = 1; i <= num_results; ++i) { std::stringstream s; s << "["; if (i == num_results) s << "LAST "; s << "DATA " << i << " from thread " << _name << "]"; _datcol.push(s.str(), _name); } } private: string _name; DataCollector &_datcol; }; /** * Maybe some VTK or QT or both will be used someday. */ class GuiClass { public: GuiClass(DataCollector &datcol) : _datcol(datcol) { } /** * If the GUI wants to present or at least count the data collected so far. * @param caller_name is the name of the thread whose data is new. */ void slot_data_changed(string caller_name) const { cout << "GuiClass knows: new data from " << caller_name << std::endl; } private: DataCollector & _datcol; }; int main() { DataCollector datcol; GuiClass mc(datcol); signal_new_data.connect(boost::bind(&GuiClass::slot_data_changed, &mc, _1)); CalcThread r1("A", datcol), r2("B", datcol), r3("C", datcol), r4("D", datcol), r5("E", datcol); boost::thread t1(r1, 3); boost::thread t2(r2, 1); boost::thread t3(r3, 2); boost::thread t4(r4, 2); boost::thread t5(r5, 3); t1.join(); t2.join(); t3.join(); t4.join(); t5.join(); datcol.out(); cout << "\nDone" << std::endl; return 0; }

  Read the article

• What does Ruby have that Python doesn't, and vice versa?

  - by Lennart Regebro

  There is a lot of discussions of Python vs Ruby, and I all find them completely unhelpful, because they all turn around why feature X sucks in language Y, or that claim language Y doesn't have X, although in fact it does. I also know exactly why I prefer Python, but that's also subjective, and wouldn't help anybody choosing, as they might not have the same tastes in development as I do. It would therefore be interesting to list the differences, objectively. So no "Python's lambdas sucks". Instead explain what Ruby's lambdas can do that Python's can't. No subjectivity. Example code is good! Don't have several differences in one answer, please. And vote up the ones you know are correct, and down those you know are incorrect (or are subjective). Also, differences in syntax is not interesting. We know Python does with indentation what Ruby does with brackets and ends, and that @ is called self in Python. UPDATE: This is now a community wiki, so we can add the big differences here. Ruby has a class reference in the class body In Ruby you have a reference to the class (self) already in the class body. In Python you don't have a reference to the class until after the class construction is finished. An example: class Kaka puts self end self in this case is the class, and this code would print out "Kaka". There is no way to print out the class name or in other ways access the class from the class definition body in Python. All classes are mutable in Ruby This lets you develop extensions to core classes. Here's an example of a rails extension: class String def starts_with?(other) head = self[0, other.length] head == other end end Ruby has Perl-like scripting features Ruby has first class regexps, $-variables, the awk/perl line by line input loop and other features that make it more suited to writing small shell scripts that munge text files or act as glue code for other programs. Ruby has first class continuations Thanks to the callcc statement. In Python you can create continuations by various techniques, but there is no support built in to the language. Ruby has blocks With the "do" statement you can create a multi-line anonymous function in Ruby, which will be passed in as an argument into the method in front of do, and called from there. In Python you would instead do this either by passing a method or with generators. Ruby: amethod { |here| many=lines+of+code goes(here) } Python: def function(here): many=lines+of+code goes(here) amethod(function) Interestingly, the convenience statement in Ruby for calling a block is called "yield", which in Python will create a generator. Ruby: def themethod yield 5 end themethod do |foo| puts foo end Python: def themethod(): yield 5 for foo in themethod: print foo Although the principles are different, the result is strikingly similar. Python has built-in generators (which are used like Ruby blocks, as noted above) Python has support for generators in the language. In Ruby you could use the generator module that uses continuations to create a generator from a block. Or, you could just use a block/proc/lambda! Moreover, in Ruby 1.9 Fibers are, and can be used as, generators. docs.python.org has this generator example: def reverse(data): for index in range(len(data)-1, -1, -1): yield data[index] Contrast this with the above block examples. Python has flexible name space handling In Ruby, when you import a file with require, all the things defined in that file will end up in your global namespace. This causes namespace pollution. The solution to that is Rubys modules. But if you create a namespace with a module, then you have to use that namespace to access the contained classes. In Python, the file is a module, and you can import its contained names with from themodule import *, thereby polluting the namespace if you want. But you can also import just selected names with from themodule import aname, another or you can simply import themodule and then access the names with themodule.aname. If you want more levels in your namespace you can have packages, which are directories with modules and an __init__.py file. Python has docstrings Docstrings are strings that are attached to modules, functions and methods and can be introspected at runtime. This helps for creating such things as the help command and automatic documentation. def frobnicate(bar): """frobnicate takes a bar and frobnicates it >>> bar = Bar() >>> bar.is_frobnicated() False >>> frobnicate(bar) >>> bar.is_frobnicated() True """ Python has more libraries Python has a vast amount of available modules and bindings for libraries. Python has multiple inheritance Ruby does not ("on purpose" -- see Ruby's website, see here how it's done in Ruby). It does reuse the module concept as a sort of abstract classes. Python has list/dict comprehensions Python: res = [x*x for x in range(1, 10)] Ruby: res = (0..9).map { |x| x * x } Python: >>> (x*x for x in range(10)) <generator object <genexpr> at 0xb7c1ccd4> >>> list(_) [0, 1, 4, 9, 16, 25, 36, 49, 64, 81] Ruby: p = proc { |x| x * x } (0..9).map(&p) Python: >>> {x:str(y*y) for x,y in {1:2, 3:4}.items()} {1: '4', 3: '16'} Ruby: >> Hash[{1=>2, 3=>4}.map{|x,y| [x,(y*y).to_s]}] => {1=>"4", 3=>"16"} Python has decorators Things similar to decorators can be created in Ruby, and it can also be argued that they aren't as necessary as in Python.

  Read the article

• Exception - Illegal Block size during decryption(Android)

  - by Vamsi

  I am writing an application which encrypts and decrypts the user notes based on the user set password. i used the following algorithms for encryption/decryption 1. PBEWithSHA256And256BitAES-CBC-BC 2. PBEWithMD5And128BitAES-CBC-OpenSSL e_Cipher = Cipher.getInstance(PBEWithSHA256And256BitAES-CBC-BC); d_Cipher = Cipher.getInstance(PBEWithSHA256And256BitAES-CBC-BC); e_Cipher.init() d_Cipher.init() encryption is working well, but when trying to decrypt it gives Exception - Illegal Block size after encryption i am converting the cipherText to HEX and storing it in a sqlite database. i am retrieving correct values from the sqlite database during decyption but when calling d_Cipher.dofinal() it throws the Exception. I thought i missed to specify the padding and tried to check what are the other available cipher algorithms but i was unable to found. so request you to please give the some knowledge on what are the cipher algorithms and padding that are supported by Android? if the algorithm which i used can be used for padding, how should i specify the padding mechanism? I am pretty new to Encryption so tried a couple of algorithms which are available in BouncyCastle.java but unsuccessful. As requested here is the code public class CryptoHelper { private static final String TAG = "CryptoHelper"; //private static final String PBEWithSHA256And256BitAES = "PBEWithSHA256And256BitAES-CBC-BC"; //private static final String PBEWithSHA256And256BitAES = "PBEWithMD5And128BitAES-CBC-OpenSSL"; private static final String PBEWithSHA256And256BitAES = "PBEWithMD5And128BitAES-CBC-OpenSSLPBEWITHSHA1AND3-KEYTRIPLEDES-CB"; private static final String randomAlgorithm = "SHA1PRNG"; public static final int SALT_LENGTH = 8; public static final int SALT_GEN_ITER_COUNT = 20; private final static String HEX = "0123456789ABCDEF"; private Cipher e_Cipher; private Cipher d_Cipher; private SecretKey secretKey; private byte salt[]; public CryptoHelper(String password) throws InvalidKeyException, NoSuchAlgorithmException, NoSuchPaddingException, InvalidAlgorithmParameterException, InvalidKeySpecException { char[] cPassword = password.toCharArray(); PBEKeySpec pbeKeySpec = new PBEKeySpec(cPassword); PBEParameterSpec pbeParamSpec = new PBEParameterSpec(salt, SALT_GEN_ITER_COUNT); SecretKeyFactory keyFac = SecretKeyFactory.getInstance(PBEWithSHA256And256BitAES); secretKey = keyFac.generateSecret(pbeKeySpec); SecureRandom saltGen = SecureRandom.getInstance(randomAlgorithm); this.salt = new byte[SALT_LENGTH]; saltGen.nextBytes(this.salt); e_Cipher = Cipher.getInstance(PBEWithSHA256And256BitAES); d_Cipher = Cipher.getInstance(PBEWithSHA256And256BitAES); e_Cipher.init(Cipher.ENCRYPT_MODE, secretKey, pbeParamSpec); d_Cipher.init(Cipher.DECRYPT_MODE, secretKey, pbeParamSpec); } public String encrypt(String cleartext) throws IllegalBlockSizeException, BadPaddingException { byte[] encrypted = e_Cipher.doFinal(cleartext.getBytes()); return convertByteArrayToHex(encrypted); } public String decrypt(String cipherString) throws IllegalBlockSizeException { byte[] plainText = decrypt(convertStringtobyte(cipherString)); return(new String(plainText)); } public byte[] decrypt(byte[] ciphertext) throws IllegalBlockSizeException { byte[] retVal = {(byte)0x00}; try { retVal = d_Cipher.doFinal(ciphertext); } catch (BadPaddingException e) { Log.e(TAG, e.toString()); } return retVal; } public String convertByteArrayToHex(byte[] buf) { if (buf == null) return ""; StringBuffer result = new StringBuffer(2*buf.length); for (int i = 0; i < buf.length; i++) { appendHex(result, buf[i]); } return result.toString(); } private static void appendHex(StringBuffer sb, byte b) { sb.append(HEX.charAt((b>>4)&0x0f)).append(HEX.charAt(b&0x0f)); } private static byte[] convertStringtobyte(String hexString) { int len = hexString.length()/2; byte[] result = new byte[len]; for (int i = 0; i < len; i++) { result[i] = Integer.valueOf(hexString.substring(2*i, 2*i+2), 16).byteValue(); } return result; } public byte[] getSalt() { return salt; } public SecretKey getSecretKey() { return secretKey; } public static SecretKey createSecretKey(char[] password) throws NoSuchAlgorithmException, InvalidKeySpecException { PBEKeySpec pbeKeySpec = new PBEKeySpec(password); SecretKeyFactory keyFac = SecretKeyFactory.getInstance(PBEWithSHA256And256BitAES); return keyFac.generateSecret(pbeKeySpec); } } I will call mCryptoHelper.decrypt(String str) then this results in Illegal block size exception My Env: Android 1.6 on Eclipse

  Read the article

• Super Noob C++ variable help

  - by julian

  Ok, I must preface this by stating that I know so so little about c++ and am hoping someone can just help me out... I have the below code: string GoogleMapControl::CreatePolyLine(RideItem *ride) { std::vector<RideFilePoint> intervalPoints; ostringstream oss; int cp; int intervalTime = 30; // 30 seconds int zone =ride->zoneRange(); if(zone >= 0) { cp = 300; // default cp to 300 watts } else { cp = ride->zones->getCP(zone); } foreach(RideFilePoint* rfp, ride->ride()->dataPoints()) { intervalPoints.push_back(*rfp); if((intervalPoints.back().secs - intervalPoints.front().secs) > intervalTime) { // find the avg power and color code it and create a polyline... AvgPower avgPower = for_each(intervalPoints.begin(), intervalPoints.end(), AvgPower()); // find the color QColor color = GetColor(cp,avgPower); // create the polyline CreateSubPolyLine(intervalPoints,oss,color); intervalPoints.clear(); intervalPoints.push_back(*rfp); } } return oss.str(); } void GoogleMapControl::CreateSubPolyLine(const std::vector<RideFilePoint> &points, std::ostringstream &oss, QColor color) { oss.precision(6); QString colorstr = color.name(); oss.setf(ios::fixed,ios::floatfield); oss << "var polyline = new GPolyline(["; BOOST_FOREACH(RideFilePoint rfp, points) { if (ceil(rfp.lat) != 180 && ceil(rfp.lon) != 180) { oss << "new GLatLng(" << rfp.lat << "," << rfp.lon << ")," << endl; } } oss << "],\"" << colorstr.toStdString() << "\",4);"; oss << "GEvent.addListener(polyline, 'mouseover', function() {" << endl << "var tooltip_text = 'Avg watts:" << avgPower <<" <br> Avg Speed: <br> Color: "<< colorstr.toStdString() <<"';" << endl << "var ss={'weight':8};" << endl << "this.setStrokeStyle(ss);" << endl << "this.overlay = new MapTooltip(this,tooltip_text);" << endl << "map.addOverlay(this.overlay);" << endl << "});" << endl << "GEvent.addListener(polyline, 'mouseout', function() {" << endl << "map.removeOverlay(this.overlay);" << endl << "var ss={'weight':5};" << endl << "this.setStrokeStyle(ss);" << endl << "});" << endl; oss << "map.addOverlay (polyline);" << endl; } And I'm trying to get the avgPower from this part: AvgPower avgPower = for_each(intervalPoints.begin(), intervalPoints.end(), AvgPower()); the first part to cary over to the second part: << "var tooltip_text = 'Avg watts:" << avgPower <<" <br> Avg Speed: <br> Color: "<< colorstr.toStdString() <<"';" << endl But of course I haven't the slightest clue how to do it... anyone feeling generous today? Thanks in advance

  Read the article

• libcurl - unable to download a file

  - by marmistrz

  I'm working on a program which will download lyrics from sites like AZLyrics. I'm using libcurl. It's my code lyricsDownloader.cpp #include "lyricsDownloader.h" #include <curl/curl.h> #include <cstring> #include <iostream> #define DEBUG 1 ///////////////////////////////////////////////////////////////////////////// size_t lyricsDownloader::write_data_to_var(char *ptr, size_t size, size_t nmemb, void *userdata) // this function is a static member function { ostringstream * stream = (ostringstream*) userdata; size_t count = size * nmemb; stream->write(ptr, count); return count; } string AZLyricsDownloader::toProviderCode() const { /*this creates an url*/ } CURLcode AZLyricsDownloader::download() { CURL * handle; CURLcode err; ostringstream buff; handle = curl_easy_init(); if (! handle) return static_cast<CURLcode>(-1); // set verbose if debug on curl_easy_setopt( handle, CURLOPT_VERBOSE, DEBUG ); curl_easy_setopt( handle, CURLOPT_URL, toProviderCode().c_str() ); // set the download url to the generated one curl_easy_setopt(handle, CURLOPT_WRITEDATA, &buff); curl_easy_setopt(handle, CURLOPT_WRITEFUNCTION, &AZLyricsDownloader::write_data_to_var); err = curl_easy_perform(handle); // The segfault should be somewhere here - after calling the function but before it ends cerr << "cleanup\n"; curl_easy_cleanup(handle); // copy the contents to text variable lyrics = buff.str(); return err; } main.cpp #include <QString> #include <QTextEdit> #include <iostream> #include "lyricsDownloader.h" int main(int argc, char *argv[]) { AZLyricsDownloader dl(argv[1], argv[2]); dl.perform(); QTextEdit qtexted(QString::fromStdString(dl.lyrics)); cout << qPrintable(qtexted.toPlainText()); return 0; } When running ./maelyrica Anthrax Madhouse I'm getting this logged from curl * About to connect() to azlyrics.com port 80 (#0) * Trying 174.142.163.250... * connected * Connected to azlyrics.com (174.142.163.250) port 80 (#0) > GET /lyrics/anthrax/madhouse.html HTTP/1.1 Host: azlyrics.com Accept: */* < HTTP/1.1 301 Moved Permanently < Server: nginx/1.0.12 < Date: Thu, 05 Jul 2012 16:59:21 GMT < Content-Type: text/html < Content-Length: 185 < Connection: keep-alive < Location: http://www.azlyrics.com/lyrics/anthrax/madhouse.html < Segmentation fault Strangely, the file is there. The same error is displayed when there's no such page (redirect to azlyrics.com mainpage) What am I doing wrong? Thanks in advance EDIT: I made the function for writing data static, but this changes nothing. Even wget seems to have problems $ wget http://www.azlyrics.com/lyrics/anthrax/madhouse.html --2012-07-06 10:36:05-- http://www.azlyrics.com/lyrics/anthrax/madhouse.html Resolving www.azlyrics.com... 174.142.163.250 Connecting to www.azlyrics.com|174.142.163.250|:80... connected. HTTP request sent, awaiting response... No data received. Retrying. Why does opening the page in a browser work and wget/curl not? EDIT2: After adding this: curl_easy_setopt(handle, CURLOPT_FOLLOWLOCATION, 1); The log is: * About to connect() to azlyrics.com port 80 (#0) * Trying 174.142.163.250... * connected * Connected to azlyrics.com (174.142.163.250) port 80 (#0) > GET /lyrics/anthrax/madhouse.html HTTP/1.1 Host: azlyrics.com Accept: */* < HTTP/1.1 301 Moved Permanently < Server: nginx/1.0.12 < Date: Fri, 06 Jul 2012 09:09:47 GMT < Content-Type: text/html < Content-Length: 185 < Connection: keep-alive < Location: http://www.azlyrics.com/lyrics/anthrax/madhouse.html < * Ignoring the response-body * Connection #0 to host azlyrics.com left intact * Issue another request to this URL: 'http://www.azlyrics.com/lyrics/anthrax/madhouse.html' * About to connect() to www.azlyrics.com port 80 (#1) * Trying 174.142.163.250... * connected * Connected to www.azlyrics.com (174.142.163.250) port 80 (#1) > GET /lyrics/anthrax/madhouse.html HTTP/1.1 Host: www.azlyrics.com Accept: */* < HTTP/1.1 200 OK < Server: nginx/1.0.12 < Date: Fri, 06 Jul 2012 09:09:47 GMT < Content-Type: text/html < Transfer-Encoding: chunked < Connection: keep-alive < Segmentation fault

  Read the article

• pagination panel should remain static

  - by fusion

  i've a search form in which a user enters the keyword and the results are displayed with pagination. everything works fine except for the fact that when the user clicks on the 'Next' button, the pagination panel disappears as well when the page loads to retrieve the data through ajax. how do i make the pagination panel static, while the data is being retrieved? search.html: <form name="myform" class="wrapper"> <input type="text" name="q" id="q" onkeyup="showPage();" class="txt_search"/> <input type="button" name="button" onclick="showPage();" class="button"/> <p> </p> <div id="txtHint"></div> </form> ajax: var url="search.php"; url += "?q="+str+"&page="+page+"&list="; url += "&sid="+Math.random(); xmlHttp.onreadystatechange=stateChanged; xmlHttp.open("GET",url,true); xmlHttp.send(null); function stateChanged(){ if (xmlHttp.readyState==4 || xmlHttp.readyState=="complete"){ document.getElementById("txtHint").innerHTML=xmlHttp.responseText; } //end if } //end function search.php: $self = $_SERVER['PHP_SELF']; $limit = 3; //Number of results per page $adjacents = 2; $numpages=ceil($totalrows/$limit); $query = $query." ORDER BY idQuotes LIMIT " . ($page-1)*$limit . ",$limit"; $result = mysql_query($query, $conn) or die('Error:' .mysql_error()); ?> <div class="search_caption">Search Results</div> <div class="search_div"> <table class="result"> <?php while ($row= mysql_fetch_array($result, MYSQL_ASSOC)) { $cQuote = highlightWords(htmlspecialchars($row['cQuotes']), $search_result); ?> <tr> . . .display results. . . </tr> <?php } ?> </table> </div> <hr> <div class="searchmain"> <?php //Create and print the Navigation bar $nav=""; $next = $page+1; $prev = $page-1; if($page > 1) { $nav .= "<a onclick=\"showPage('','$prev'); return false;\" href=\"$self?page=" . $prev . "&q=" .urlencode($search_result) . "\">< Prev</a>"; $first = "<a onclick=\"showPage('','1'); return false;\" href=\"$self?page=1&q=" .urlencode($search_result) . "\"> << </a>" ; } else { $nav .= "&nbsp;"; $first = "&nbsp;"; } for($i = 1 ; $i <= $numpages ; $i++) { if($i == $page) { $nav .= "<span class=\"no_link\">$i</span>"; }else{ $nav .= "<a onclick=\"showPage('',$i); return false;\" href=\"$self?page=" . $i . "&q=" .urlencode($search_result) . "\">$i</a>"; } } if($page < $numpages) { $nav .= "<a onclick=\"showPage('','$next'); return false;\" href=\"$self?page=" . $next . "&q=" .urlencode($search_result) . "\">Next ></a>"; $last = "<a onclick=\"showPage('','$numpages'); return false;\" href=\"$self?page=$numpages&q=" .urlencode($search_result) . "\"> >> </a>"; } else { $nav .= "&nbsp;"; $last = "&nbsp;"; } echo $first . $nav . $last; ?> </div>

  Read the article

• I can't install using Wubi due to permission denied error

  - by Taksh Sharma

  I can't install ubuntu 11.10 inside my windows 7. It shows permission denied while installation. It gave a log file having the following data: 03-29 20:19 DEBUG TaskList: # Running tasklist... 03-29 20:19 DEBUG TaskList: ## Running select_target_dir... 03-29 20:19 INFO WindowsBackend: Installing into D:\ubuntu 03-29 20:19 DEBUG TaskList: ## Finished select_target_dir 03-29 20:19 DEBUG TaskList: ## Running create_dir_structure... 03-29 20:19 DEBUG CommonBackend: Creating dir D:\ubuntu 03-29 20:19 DEBUG CommonBackend: Creating dir D:\ubuntu\disks 03-29 20:19 DEBUG CommonBackend: Creating dir D:\ubuntu\install 03-29 20:19 DEBUG CommonBackend: Creating dir D:\ubuntu\install\boot 03-29 20:19 DEBUG CommonBackend: Creating dir D:\ubuntu\disks\boot 03-29 20:19 DEBUG CommonBackend: Creating dir D:\ubuntu\disks\boot\grub 03-29 20:19 DEBUG CommonBackend: Creating dir D:\ubuntu\install\boot\grub 03-29 20:19 DEBUG TaskList: ## Finished create_dir_structure 03-29 20:19 DEBUG TaskList: ## Running uncompress_target_dir... 03-29 20:19 DEBUG TaskList: ## Finished uncompress_target_dir 03-29 20:19 DEBUG TaskList: ## Running create_uninstaller... 03-29 20:19 DEBUG WindowsBackend: Copying uninstaller E:\wubi.exe -> D:\ubuntu\uninstall-wubi.exe 03-29 20:19 DEBUG registry: Setting registry key -2147483646 Software\Microsoft\Windows\CurrentVersion\Uninstall\Wubi UninstallString D:\ubuntu\uninstall-wubi.exe 03-29 20:19 DEBUG registry: Setting registry key -2147483646 Software\Microsoft\Windows\CurrentVersion\Uninstall\Wubi InstallationDir D:\ubuntu 03-29 20:19 DEBUG registry: Setting registry key -2147483646 Software\Microsoft\Windows\CurrentVersion\Uninstall\Wubi DisplayName Ubuntu 03-29 20:19 DEBUG registry: Setting registry key -2147483646 Software\Microsoft\Windows\CurrentVersion\Uninstall\Wubi DisplayIcon D:\ubuntu\Ubuntu.ico 03-29 20:19 DEBUG registry: Setting registry key -2147483646 Software\Microsoft\Windows\CurrentVersion\Uninstall\Wubi DisplayVersion 11.10-rev241 03-29 20:19 DEBUG registry: Setting registry key -2147483646 Software\Microsoft\Windows\CurrentVersion\Uninstall\Wubi Publisher Ubuntu 03-29 20:19 DEBUG registry: Setting registry key -2147483646 Software\Microsoft\Windows\CurrentVersion\Uninstall\Wubi URLInfoAbout http://www.ubuntu.com 03-29 20:19 DEBUG registry: Setting registry key -2147483646 Software\Microsoft\Windows\CurrentVersion\Uninstall\Wubi HelpLink http://www.ubuntu.com/support 03-29 20:19 DEBUG TaskList: ## Finished create_uninstaller 03-29 20:19 DEBUG TaskList: ## Running copy_installation_files... 03-29 20:19 DEBUG WindowsBackend: Copying C:\Users\Home\AppData\Local\Temp\pylB911.tmp\data\custom-installation -> D:\ubuntu\install\custom-installation 03-29 20:19 DEBUG WindowsBackend: Copying C:\Users\Home\AppData\Local\Temp\pylB911.tmp\winboot -> D:\ubuntu\winboot 03-29 20:19 DEBUG WindowsBackend: Copying C:\Users\Home\AppData\Local\Temp\pylB911.tmp\data\images\Ubuntu.ico -> D:\ubuntu\Ubuntu.ico 03-29 20:19 DEBUG TaskList: ## Finished copy_installation_files 03-29 20:19 DEBUG TaskList: ## Running get_iso... 03-29 20:19 DEBUG TaskList: New task copy_file 03-29 20:19 DEBUG TaskList: ### Running copy_file... 03-29 20:23 ERROR TaskList: [Errno 13] Permission denied Traceback (most recent call last): File "\lib\wubi\backends\common\tasklist.py", line 197, in __call__ File "\lib\wubi\backends\common\utils.py", line 202, in copy_file IOError: [Errno 13] Permission denied 03-29 20:23 DEBUG TaskList: # Cancelling tasklist 03-29 20:23 DEBUG TaskList: New task check_iso 03-29 20:23 ERROR root: [Errno 13] Permission denied Traceback (most recent call last): File "\lib\wubi\application.py", line 58, in run File "\lib\wubi\application.py", line 130, in select_task File "\lib\wubi\application.py", line 205, in run_cd_menu File "\lib\wubi\application.py", line 120, in select_task File "\lib\wubi\application.py", line 158, in run_installer File "\lib\wubi\backends\common\tasklist.py", line 197, in __call__ File "\lib\wubi\backends\common\utils.py", line 202, in copy_file IOError: [Errno 13] Permission denied 03-29 20:23 ERROR TaskList: 'WindowsBackend' object has no attribute 'iso_path' Traceback (most recent call last): File "\lib\wubi\backends\common\tasklist.py", line 197, in __call__ File "\lib\wubi\backends\common\backend.py", line 579, in get_iso File "\lib\wubi\backends\common\backend.py", line 565, in use_iso AttributeError: 'WindowsBackend' object has no attribute 'iso_path' 03-29 20:23 DEBUG TaskList: # Cancelling tasklist 03-29 20:23 DEBUG TaskList: # Finished tasklist 03-29 20:29 INFO root: === wubi 11.10 rev241 === 03-29 20:29 DEBUG root: Logfile is c:\users\home\appdata\local\temp\wubi-11.10-rev241.log 03-29 20:29 DEBUG root: sys.argv = ['main.pyo', '--exefile="E:\\wubi.exe"'] 03-29 20:29 DEBUG CommonBackend: data_dir=C:\Users\Home\AppData\Local\Temp\pyl3487.tmp\data 03-29 20:29 DEBUG WindowsBackend: 7z=C:\Users\Home\AppData\Local\Temp\pyl3487.tmp\bin\7z.exe 03-29 20:29 DEBUG WindowsBackend: startup_folder=C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Startup 03-29 20:29 DEBUG CommonBackend: Fetching basic info... 03-29 20:29 DEBUG CommonBackend: original_exe=E:\wubi.exe 03-29 20:29 DEBUG CommonBackend: platform=win32 03-29 20:29 DEBUG CommonBackend: osname=nt 03-29 20:29 DEBUG CommonBackend: language=en_IN 03-29 20:29 DEBUG CommonBackend: encoding=cp1252 03-29 20:29 DEBUG WindowsBackend: arch=amd64 03-29 20:29 DEBUG CommonBackend: Parsing isolist=C:\Users\Home\AppData\Local\Temp\pyl3487.tmp\data\isolist.ini 03-29 20:29 DEBUG CommonBackend: Adding distro Xubuntu-i386 03-29 20:29 DEBUG CommonBackend: Adding distro Xubuntu-amd64 03-29 20:29 DEBUG CommonBackend: Adding distro Kubuntu-amd64 03-29 20:29 DEBUG CommonBackend: Adding distro Mythbuntu-i386 03-29 20:29 DEBUG CommonBackend: Adding distro Ubuntu-amd64 03-29 20:29 DEBUG CommonBackend: Adding distro Ubuntu-i386 03-29 20:29 DEBUG CommonBackend: Adding distro Mythbuntu-amd64 03-29 20:29 DEBUG CommonBackend: Adding distro Kubuntu-i386 03-29 20:29 DEBUG WindowsBackend: Fetching host info... 03-29 20:29 DEBUG WindowsBackend: registry_key=Software\Microsoft\Windows\CurrentVersion\Uninstall\Wubi 03-29 20:29 DEBUG WindowsBackend: windows version=vista 03-29 20:29 DEBUG WindowsBackend: windows_version2=Windows 7 Home Basic 03-29 20:29 DEBUG WindowsBackend: windows_sp=None 03-29 20:29 DEBUG WindowsBackend: windows_build=7601 03-29 20:29 DEBUG WindowsBackend: gmt=5 03-29 20:29 DEBUG WindowsBackend: country=IN 03-29 20:29 DEBUG WindowsBackend: timezone=Asia/Calcutta 03-29 20:29 DEBUG WindowsBackend: windows_username=Home 03-29 20:29 DEBUG WindowsBackend: user_full_name=Home 03-29 20:29 DEBUG WindowsBackend: user_directory=C:\Users\Home 03-29 20:29 DEBUG WindowsBackend: windows_language_code=1033 03-29 20:29 DEBUG WindowsBackend: windows_language=English 03-29 20:29 DEBUG WindowsBackend: processor_name=Intel(R) Core(TM) i3 CPU M 370 @ 2.40GHz 03-29 20:29 DEBUG WindowsBackend: bootloader=vista 03-29 20:29 DEBUG WindowsBackend: system_drive=Drive(C: hd 61135.1523438 mb free ntfs) 03-29 20:29 DEBUG WindowsBackend: drive=Drive(C: hd 61135.1523438 mb free ntfs) 03-29 20:29 DEBUG WindowsBackend: drive=Drive(D: hd 12742.5507813 mb free ntfs) 03-29 20:29 DEBUG WindowsBackend: drive=Drive(E: cd 0.0 mb free cdfs) 03-29 20:29 DEBUG WindowsBackend: drive=Drive(F: cd 0.0 mb free ) 03-29 20:29 DEBUG WindowsBackend: drive=Drive(G: hd 93.22265625 mb free fat32) 03-29 20:29 DEBUG WindowsBackend: drive=Drive(Q: hd 0.0 mb free ) 03-29 20:29 DEBUG WindowsBackend: uninstaller_path=D:\ubuntu\uninstall-wubi.exe 03-29 20:29 DEBUG WindowsBackend: previous_target_dir=D:\ubuntu 03-29 20:29 DEBUG WindowsBackend: previous_distro_name=Ubuntu 03-29 20:29 DEBUG WindowsBackend: keyboard_id=67699721 03-29 20:29 DEBUG WindowsBackend: keyboard_layout=us 03-29 20:29 DEBUG WindowsBackend: keyboard_variant= 03-29 20:29 DEBUG CommonBackend: python locale=('en_IN', 'cp1252') 03-29 20:29 DEBUG CommonBackend: locale=en_IN 03-29 20:29 DEBUG WindowsBackend: total_memory_mb=3893.859375 03-29 20:29 DEBUG CommonBackend: Searching ISOs on USB devices 03-29 20:29 DEBUG CommonBackend: Searching for local CDs 03-29 20:29 DEBUG Distro: checking whether C:\Users\Home\AppData\Local\Temp\pyl3487.tmp is a valid Ubuntu CD 03-29 20:29 DEBUG Distro: does not contain C:\Users\Home\AppData\Local\Temp\pyl3487.tmp\casper\filesystem.squashfs 03-29 20:29 DEBUG Distro: checking whether C:\Users\Home\AppData\Local\Temp\pyl3487.tmp is a valid Ubuntu CD 03-29 20:29 DEBUG Distro: does not contain C:\Users\Home\AppData\Local\Temp\pyl3487.tmp\casper\filesystem.squashfs 03-29 20:29 DEBUG Distro: checking whether C:\Users\Home\AppData\Local\Temp\pyl3487.tmp is a valid Kubuntu CD 03-29 20:29 DEBUG Distro: does not contain C:\Users\Home\AppData\Local\Temp\pyl3487.tmp\casper\filesystem.squashfs 03-29 20:29 DEBUG Distro: checking whether C:\Users\Home\AppData\Local\Temp\pyl3487.tmp is a valid Kubuntu CD 03-29 20:29 DEBUG Distro: does not contain C:\Users\Home\AppData\Local\Temp\pyl3487.tmp\casper\filesystem.squashfs 03-29 20:29 DEBUG Distro: checking whether C:\Users\Home\AppData\Local\Temp\pyl3487.tmp is a valid Xubuntu CD 03-29 20:29 DEBUG Distro: does not contain C:\Users\Home\AppData\Local\Temp\pyl3487.tmp\casper\filesystem.squashfs 03-29 20:29 DEBUG Distro: checking whether C:\Users\Home\AppData\Local\Temp\pyl3487.tmp is a valid Xubuntu CD 03-29 20:29 DEBUG Distro: does not contain C:\Users\Home\AppData\Local\Temp\pyl3487.tmp\casper\filesystem.squashfs 03-29 20:29 DEBUG Distro: checking whether C:\Users\Home\AppData\Local\Temp\pyl3487.tmp is a valid Mythbuntu CD 03-29 20:29 DEBUG Distro: does not contain C:\Users\Home\AppData\Local\Temp\pyl3487.tmp\casper\filesystem.squashfs 03-29 20:29 DEBUG Distro: checking whether C:\Users\Home\AppData\Local\Temp\pyl3487.tmp is a valid Mythbuntu CD 03-29 20:29 DEBUG Distro: does not contain C:\Users\Home\AppData\Local\Temp\pyl3487.tmp\casper\filesystem.squashfs 03-29 20:29 DEBUG Distro: checking whether D:\ is a valid Ubuntu CD 03-29 20:29 DEBUG Distro: does not contain D:\casper\filesystem.squashfs 03-29 20:29 DEBUG Distro: checking whether D:\ is a valid Ubuntu CD 03-29 20:29 DEBUG Distro: does not contain D:\casper\filesystem.squashfs 03-29 20:29 DEBUG Distro: checking whether D:\ is a valid Kubuntu CD 03-29 20:29 DEBUG Distro: does not contain D:\casper\filesystem.squashfs 03-29 20:29 DEBUG Distro: checking whether D:\ is a valid Kubuntu CD 03-29 20:29 DEBUG Distro: does not contain D:\casper\filesystem.squashfs 03-29 20:29 DEBUG Distro: checking whether D:\ is a valid Xubuntu CD 03-29 20:29 DEBUG Distro: does not contain D:\casper\filesystem.squashfs 03-29 20:29 DEBUG Distro: checking whether D:\ is a valid Xubuntu CD 03-29 20:29 DEBUG Distro: does not contain D:\casper\filesystem.squashfs 03-29 20:29 DEBUG Distro: checking whether D:\ is a valid Mythbuntu CD 03-29 20:29 DEBUG Distro: does not contain D:\casper\filesystem.squashfs 03-29 20:29 DEBUG Distro: checking whether D:\ is a valid Mythbuntu CD 03-29 20:29 DEBUG Distro: does not contain D:\casper\filesystem.squashfs 03-29 20:29 DEBUG Distro: checking whether E:\ is a valid Ubuntu CD 03-29 20:29 DEBUG Distro: parsing info from str=Ubuntu 11.10 "Oneiric Ocelot" - Release i386 (20111012) 03-29 20:29 DEBUG Distro: parsed info={'name': 'Ubuntu', 'subversion': 'Release', 'version': '11.10', 'build': '20111012', 'codename': 'Oneiric Ocelot', 'arch': 'i386'} 03-29 20:29 INFO Distro: Found a valid CD for Ubuntu: E:\ 03-29 20:29 INFO root: Running the CD menu... 03-29 20:29 DEBUG WindowsFrontend: __init__... 03-29 20:29 DEBUG WindowsFrontend: on_init... 03-29 20:29 INFO WinuiPage: appname=wubi, localedir=C:\Users\Home\AppData\Local\Temp\pyl3487.tmp\translations, languages=['en_IN', 'en'] 03-29 20:29 INFO WinuiPage: appname=wubi, localedir=C:\Users\Home\AppData\Local\Temp\pyl3487.tmp\translations, languages=['en_IN', 'en'] 03-29 20:29 INFO root: CD menu finished 03-29 20:29 INFO root: Already installed, running the uninstaller... 03-29 20:29 INFO root: Running the uninstaller... 03-29 20:29 INFO CommonBackend: This is the uninstaller running 03-29 20:29 INFO WinuiPage: appname=wubi, localedir=C:\Users\Home\AppData\Local\Temp\pyl3487.tmp\translations, languages=['en_IN', 'en'] 03-29 20:29 INFO root: Received settings 03-29 20:29 INFO WinuiPage: appname=wubi, localedir=C:\Users\Home\AppData\Local\Temp\pyl3487.tmp\translations, languages=['en_IN', 'en'] 03-29 20:29 DEBUG TaskList: # Running tasklist... 03-29 20:29 DEBUG TaskList: ## Running Remove bootloader entry... 03-29 20:29 DEBUG WindowsBackend: Could not find bcd id 03-29 20:29 DEBUG WindowsBackend: undo_bootini C: 03-29 20:29 DEBUG WindowsBackend: undo_configsys Drive(C: hd 61135.1523438 mb free ntfs) 03-29 20:29 DEBUG WindowsBackend: undo_bootini D: 03-29 20:29 DEBUG WindowsBackend: undo_configsys Drive(D: hd 12742.5507813 mb free ntfs) 03-29 20:29 DEBUG WindowsBackend: undo_bootini G: 03-29 20:29 DEBUG WindowsBackend: undo_configsys Drive(G: hd 93.22265625 mb free fat32) 03-29 20:29 DEBUG WindowsBackend: undo_bootini Q: 03-29 20:29 DEBUG WindowsBackend: undo_configsys Drive(Q: hd 0.0 mb free ) 03-29 20:29 DEBUG TaskList: ## Finished Remove bootloader entry 03-29 20:29 DEBUG TaskList: ## Running Remove target dir... 03-29 20:29 DEBUG CommonBackend: Deleting D:\ubuntu 03-29 20:29 DEBUG TaskList: ## Finished Remove target dir 03-29 20:29 DEBUG TaskList: ## Running Remove registry key... 03-29 20:29 DEBUG TaskList: ## Finished Remove registry key 03-29 20:29 DEBUG TaskList: # Finished tasklist 03-29 20:29 INFO root: Almost finished uninstalling 03-29 20:29 INFO root: Finished uninstallation 03-29 20:29 DEBUG CommonBackend: Fetching basic info... 03-29 20:29 DEBUG CommonBackend: original_exe=E:\wubi.exe 03-29 20:29 DEBUG CommonBackend: platform=win32 03-29 20:29 DEBUG CommonBackend: osname=nt 03-29 20:29 DEBUG WindowsBackend: arch=amd64 03-29 20:29 DEBUG CommonBackend: Parsing isolist=C:\Users\Home\AppData\Local\Temp\pyl3487.tmp\data\isolist.ini 03-29 20:29 DEBUG CommonBackend: Adding distro Xubuntu-i386 03-29 20:29 DEBUG CommonBackend: Adding distro Xubuntu-amd64 03-29 20:29 DEBUG CommonBackend: Adding distro Kubuntu-amd64 03-29 20:29 DEBUG CommonBackend: Adding distro Mythbuntu-i386 03-29 20:29 DEBUG CommonBackend: Adding distro Ubuntu-amd64 03-29 20:29 DEBUG CommonBackend: Adding distro Ubuntu-i386 03-29 20:29 DEBUG CommonBackend: Adding distro Mythbuntu-amd64 03-29 20:29 DEBUG CommonBackend: Adding distro Kubuntu-i386 03-29 20:29 DEBUG WindowsBackend: Fetching host info... 03-29 20:29 DEBUG WindowsBackend: registry_key=Software\Microsoft\Windows\CurrentVersion\Uninstall\Wubi 03-29 20:29 DEBUG WindowsBackend: windows version=vista 03-29 20:29 DEBUG WindowsBackend: windows_version2=Windows 7 Home Basic 03-29 20:29 DEBUG WindowsBackend: windows_sp=None 03-29 20:29 DEBUG WindowsBackend: windows_build=7601 03-29 20:29 DEBUG WindowsBackend: gmt=5 03-29 20:29 DEBUG WindowsBackend: country=IN 03-29 20:29 DEBUG WindowsBackend: timezone=Asia/Calcutta 03-29 20:29 DEBUG WindowsBackend: windows_username=Home 03-29 20:29 DEBUG WindowsBackend: user_full_name=Home 03-29 20:29 DEBUG WindowsBackend: user_directory=C:\Users\Home 03-29 20:29 DEBUG WindowsBackend: windows_language_code=1033 03-29 20:29 DEBUG WindowsBackend: windows_language=English 03-29 20:29 DEBUG WindowsBackend: processor_name=Intel(R) Core(TM) i3 CPU M 370 @ 2.40GHz 03-29 20:29 DEBUG WindowsBackend: bootloader=vista 03-29 20:29 DEBUG WindowsBackend: system_drive=Drive(C: hd 61134.8632813 mb free ntfs) 03-29 20:29 DEBUG WindowsBackend: drive=Drive(C: hd 61134.8632813 mb free ntfs) 03-29 20:29 DEBUG WindowsBackend: drive=Drive(D: hd 12953.140625 mb free ntfs) 03-29 20:29 DEBUG WindowsBackend: drive=Drive(E: cd 0.0 mb free cdfs) 03-29 20:29 DEBUG WindowsBackend: drive=Drive(F: cd 0.0 mb free ) 03-29 20:29 DEBUG WindowsBackend: drive=Drive(G: hd 93.22265625 mb free fat32) 03-29 20:29 DEBUG WindowsBackend: drive=Drive(Q: hd 0.0 mb free ) 03-29 20:29 DEBUG WindowsBackend: uninstaller_path=None 03-29 20:29 DEBUG WindowsBackend: previous_target_dir=None 03-29 20:29 DEBUG WindowsBackend: previous_distro_name=None 03-29 20:29 DEBUG WindowsBackend: keyboard_id=67699721 03-29 20:29 DEBUG WindowsBackend: keyboard_layout=us 03-29 20:29 DEBUG WindowsBackend: keyboard_variant= 03-29 20:29 DEBUG WindowsBackend: total_memory_mb=3893.859375 03-29 20:29 DEBUG CommonBackend: Searching ISOs on USB devices 03-29 20:29 DEBUG CommonBackend: Searching for local CDs 03-29 20:29 DEBUG Distro: checking whether C:\Users\Home\AppData\Local\Temp\pyl3487.tmp is a valid Ubuntu CD 03-29 20:29 DEBUG Distro: does not contain C:\Users\Home\AppData\Local\Temp\pyl3487.tmp\casper\filesystem.squashfs 03-29 20:29 DEBUG Distro: checking whether C:\Users\Home\AppData\Local\Temp\pyl3487.tmp is a valid Ubuntu CD 03-29 20:29 DEBUG Distro: does not contain C:\Users\Home\AppData\Local\Temp\pyl3487.tmp\casper\filesystem.squashfs 03-29 20:29 DEBUG Distro: checking whether C:\Users\Home\AppData\Local\Temp\pyl3487.tmp is a valid Kubuntu CD 03-29 20:29 DEBUG Distro: does not contain C:\Users\Home\AppData\Local\Temp\pyl3487.tmp\casper\filesystem.squashfs 03-29 20:29 DEBUG Distro: checking whether C:\Users\Home\AppData\Local\Temp\pyl3487.tmp is a valid Kubuntu CD 03-29 20:29 DEBUG Distro: does not contain C:\Users\Home\AppData\Local\Temp\pyl3487.tmp\casper\filesystem.squashfs 03-29 20:29 DEBUG Distro: checking whether C:\Users\Home\AppData\Local\Temp\pyl3487.tmp is a valid Xubuntu CD 03-29 20:29 DEBUG Distro: does not contain C:\Users\Home\AppData\Local\Temp\pyl3487.tmp\casper\filesystem.squashfs 03-29 20:29 DEBUG Distro: checking whether C:\Users\Home\AppData\Local\Temp\pyl3487.tmp is a valid Xubuntu CD 03-29 20:29 DEBUG Distro: does not contain C:\Users\Home\AppData\Local\Temp\pyl3487.tmp\casper\filesystem.squashfs 03-29 20:29 DEBUG Distro: checking whether C:\Users\Home\AppData\Local\Temp\pyl3487.tmp is a valid Mythbuntu CD 03-29 20:29 DEBUG Distro: does not contain C:\Users\Home\AppData\Local\Temp\pyl3487.tmp\casper\filesystem.squashfs 03-29 20:29 DEBUG Distro: checking whether C:\Users\Home\AppData\Local\Temp\pyl3487.tmp is a valid Mythbuntu CD 03-29 20:29 DEBUG Distro: does not contain C:\Users\Home\AppData\Local\Temp\pyl3487.tmp\casper\filesystem.squashfs 03-29 20:29 DEBUG Distro: checking whether D:\ is a valid Ubuntu CD 03-29 20:29 DEBUG Distro: does not contain D:\casper\filesystem.squashfs 03-29 20:29 DEBUG Distro: checking whether D:\ is a valid Ubuntu CD 03-29 20:29 DEBUG Distro: does not contain D:\casper\filesystem.squashfs 03-29 20:29 DEBUG Distro: checking whether D:\ is a valid Kubuntu CD 03-29 20:29 DEBUG Distro: does not contain D:\casper\filesystem.squashfs 03-29 20:29 DEBUG Distro: checking whether D:\ is a valid Kubuntu CD 03-29 20:29 DEBUG Distro: does not contain D:\casper\filesystem.squashfs 03-29 20:29 DEBUG Distro: checking whether D:\ is a valid Xubuntu CD 03-29 20:29 DEBUG Distro: does not contain D:\casper\filesystem.squashfs 03-29 20:29 DEBUG Distro: checking whether D:\ is a valid Xubuntu CD 03-29 20:29 DEBUG Distro: does not contain D:\casper\filesystem.squashfs 03-29 20:29 DEBUG Distro: checking whether D:\ is a valid Mythbuntu CD 03-29 20:29 DEBUG Distro: does not contain D:\casper\filesystem.squashfs 03-29 20:29 DEBUG Distro: checking whether D:\ is a valid Mythbuntu CD 03-29 20:29 DEBUG Distro: does not contain D:\casper\filesystem.squashfs 03-29 20:29 DEBUG Distro: checking whether E:\ is a valid Ubuntu CD 03-29 20:29 INFO Distro: Found a valid CD for Ubuntu: E:\ 03-29 20:29 INFO root: Running the installer... 03-29 20:29 INFO WinuiPage: appname=wubi, localedir=C:\Users\Home\AppData\Local\Temp\pyl3487.tmp\translations, languages=['en_IN', 'en'] 03-29 20:29 INFO WinuiPage: appname=wubi, localedir=C:\Users\Home\AppData\Local\Temp\pyl3487.tmp\translations, languages=['en_IN', 'en'] 03-29 20:30 DEBUG WinuiInstallationPage: target_drive=C:, installation_size=8000MB, distro_name=Ubuntu, language=en_US, locale=en_US.UTF-8, username=taksh 03-29 20:30 INFO root: Received settings 03-29 20:30 INFO WinuiPage: appname=wubi, localedir=C:\Users\Home\AppData\Local\Temp\pyl3487.tmp\translations, languages=['en_US', 'en'] 03-29 20:30 DEBUG TaskList: # Running tasklist... 03-29 20:30 DEBUG TaskList: ## Running select_target_dir... 03-29 20:30 INFO WindowsBackend: Installing into C:\ubuntu 03-29 20:30 DEBUG TaskList: ## Finished select_target_dir 03-29 20:30 DEBUG TaskList: ## Running create_dir_structure... 03-29 20:30 DEBUG CommonBackend: Creating dir C:\ubuntu 03-29 20:30 DEBUG CommonBackend: Creating dir C:\ubuntu\disks 03-29 20:30 DEBUG CommonBackend: Creating dir C:\ubuntu\install 03-29 20:30 DEBUG CommonBackend: Creating dir C:\ubuntu\install\boot 03-29 20:30 DEBUG CommonBackend: Creating dir C:\ubuntu\disks\boot 03-29 20:30 DEBUG CommonBackend: Creating dir C:\ubuntu\disks\boot\grub 03-29 20:30 DEBUG CommonBackend: Creating dir C:\ubuntu\install\boot\grub 03-29 20:30 DEBUG TaskList: ## Finished create_dir_structure 03-29 20:30 DEBUG TaskList: ## Running uncompress_target_dir... 03-29 20:30 DEBUG TaskList: ## Finished uncompress_target_dir 03-29 20:30 DEBUG TaskList: ## Running create_uninstaller... 03-29 20:30 DEBUG WindowsBackend: Copying uninstaller E:\wubi.exe -> C:\ubuntu\uninstall-wubi.exe 03-29 20:30 DEBUG registry: Setting registry key -2147483646 Software\Microsoft\Windows\CurrentVersion\Uninstall\Wubi UninstallString C:\ubuntu\uninstall-wubi.exe 03-29 20:30 DEBUG registry: Setting registry key -2147483646 Software\Microsoft\Windows\CurrentVersion\Uninstall\Wubi InstallationDir C:\ubuntu 03-29 20:30 DEBUG registry: Setting registry key -2147483646 Software\Microsoft\Windows\CurrentVersion\Uninstall\Wubi DisplayName Ubuntu 03-29 20:30 DEBUG registry: Setting registry key -2147483646 Software\Microsoft\Windows\CurrentVersion\Uninstall\Wubi DisplayIcon C:\ubuntu\Ubuntu.ico 03-29 20:30 DEBUG registry: Setting registry key -2147483646 Software\Microsoft\Windows\CurrentVersion\Uninstall\Wubi DisplayVersion 11.10-rev241 03-29 20:30 DEBUG registry: Setting registry key -2147483646 Software\Microsoft\Windows\CurrentVersion\Uninstall\Wubi Publisher Ubuntu 03-29 20:30 DEBUG registry: Setting registry key -2147483646 Software\Microsoft\Windows\CurrentVersion\Uninstall\Wubi URLInfoAbout http://www.ubuntu.com 03-29 20:30 DEBUG registry: Setting registry key -2147483646 Software\Microsoft\Windows\CurrentVersion\Uninstall\Wubi HelpLink http://www.ubuntu.com/support 03-29 20:30 DEBUG TaskList: ## Finished create_uninstaller 03-29 20:30 DEBUG TaskList: ## Running copy_installation_files... 03-29 20:30 DEBUG WindowsBackend: Copying C:\Users\Home\AppData\Local\Temp\pyl3487.tmp\data\custom-installation -> C:\ubuntu\install\custom-installation 03-29 20:30 DEBUG WindowsBackend: Copying C:\Users\Home\AppData\Local\Temp\pyl3487.tmp\winboot -> C:\ubuntu\winboot 03-29 20:30 DEBUG WindowsBackend: Copying C:\Users\Home\AppData\Local\Temp\pyl3487.tmp\data\images\Ubuntu.ico -> C:\ubuntu\Ubuntu.ico 03-29 20:30 DEBUG TaskList: ## Finished copy_installation_files 03-29 20:30 DEBUG TaskList: ## Running get_iso... 03-29 20:30 DEBUG TaskList: New task copy_file 03-29 20:30 DEBUG TaskList: ### Running copy_file... 03-29 20:34 ERROR TaskList: [Errno 13] Permission denied Traceback (most recent call last): File "\lib\wubi\backends\common\tasklist.py", line 197, in __call__ File "\lib\wubi\backends\common\utils.py", line 202, in copy_file IOError: [Errno 13] Permission denied 03-29 20:34 DEBUG TaskList: # Cancelling tasklist 03-29 20:34 DEBUG TaskList: New task check_iso 03-29 20:34 ERROR root: [Errno 13] Permission denied Traceback (most recent call last): File "\lib\wubi\application.py", line 58, in run File "\lib\wubi\application.py", line 130, in select_task File "\lib\wubi\application.py", line 205, in run_cd_menu File "\lib\wubi\application.py", line 120, in select_task File "\lib\wubi\application.py", line 158, in run_installer File "\lib\wubi\backends\common\tasklist.py", line 197, in __call__ File "\lib\wubi\backends\common\utils.py", line 202, in copy_file IOError: [Errno 13] Permission denied 03-29 20:34 ERROR TaskList: 'WindowsBackend' object has no attribute 'iso_path' Traceback (most recent call last): File "\lib\wubi\backends\common\tasklist.py", line 197, in __call__ File "\lib\wubi\backends\common\backend.py", line 579, in get_iso File "\lib\wubi\backends\common\backend.py", line 565, in use_iso AttributeError: 'WindowsBackend' object has no attribute 'iso_path' 03-29 20:34 DEBUG TaskList: # Cancelling tasklist 03-29 20:34 DEBUG TaskList: # Finished tasklist I have no idea what's the problem is. I'm a kind of newbie. I'm using win7 64bit, and installing as an administrator. Please help me out!

  Read the article

• Generic Aggregation of C++ Objects by Attribute When Attribute Name is Unknown at Runtime

  - by stretch

  I'm currently implementing a system with a number of class's representing objects such as client, business, product etc. Standard business logic. As one might expect each class has a number of standard attributes. I have a long list of essentially identical requirements such as: the ability to retrieve all business' whose industry is manufacturing. the ability to retrieve all clients based in London Class business has attribute sector and client has attribute location. Clearly this a relational problem and in pseudo SQL would look something like: SELECT ALL business in business' WHERE sector == manufacturing Unfortunately plugging into a DB is not an option. What I want to do is have a single generic aggregation function whose signature would take the form: vector<generic> genericAggregation(class, attribute, value); Where class is the class of object I want to aggregate, attribute and value being the class attribute and value of interest. In my example I've put vector as return type, but this wouldn't work. Probably better to declare a vector of relevant class type and pass it as an argument. But this isn't the main problem. How can I accept arguments in string form for class, attribute and value and then map these in a generic object aggregation function? Since it's rude not to post code, below is a dummy program which creates a bunch of objects of imaginatively named classes. Included is a specific aggregation function which returns a vector of B objects whose A object is equal to an id specified at the command line e.g. .. $ ./aggregations 5 which returns all B's whose A objects 'i' attribute is equal to 5. See below: #include <iostream> #include <cstring> #include <sstream> #include <vector> using namespace std; //First imaginativly names dummy class class A { private: int i; double d; string s; public: A(){} A(int i, double d, string s) { this->i = i; this->d = d; this->s = s; } ~A(){} int getInt() {return i;} double getDouble() {return d;} string getString() {return s;} }; //second imaginativly named dummy class class B { private: int i; double d; string s; A *a; public: B(int i, double d, string s, A *a) { this->i = i; this->d = d; this->s = s; this->a = a; } ~B(){} int getInt() {return i;} double getDouble() {return d;} string getString() {return s;} A* getA() {return a;} }; //Containers for dummy class objects vector<A> a_vec (10); vector<B> b_vec;//100 //Util function, not important.. string int2string(int number) { stringstream ss; ss << number; return ss.str(); } //Example function that returns a new vector containing on B objects //whose A object i attribute is equal to 'id' vector<B> getBbyA(int id) { vector<B> result; for(int i = 0; i < b_vec.size(); i++) { if(b_vec.at(i).getA()->getInt() == id) { result.push_back(b_vec.at(i)); } } return result; } int main(int argc, char** argv) { //Create some A's and B's, each B has an A... //Each of the 10 A's are associated with 10 B's. for(int i = 0; i < 10; ++i) { A a(i, (double)i, int2string(i)); a_vec.at(i) = a; for(int j = 0; j < 10; j++) { B b((i * 10) + j, (double)j, int2string(i), &a_vec.at(i)); b_vec.push_back(b); } } //Got some objects so lets do some aggregation //Call example aggregation function to return all B objects //whose A object has i attribute equal to argv[1] vector<B> result = getBbyA(atoi(argv[1])); //If some B's were found print them, else don't... if(result.size() != 0) { for(int i = 0; i < result.size(); i++) { cout << result.at(i).getInt() << " " << result.at(i).getA()->getInt() << endl; } } else { cout << "No B's had A's with attribute i equal to " << argv[1] << endl; } return 0; } Compile with: g++ -o aggregations aggregations.cpp If you wish :) Instead of implementing a separate aggregation function (i.e. getBbyA() in the example) I'd like to have a single generic aggregation function which accounts for all possible class attribute pairs such that all aggregation requirements are met.. and in the event additional attributes are added later, or additional aggregation requirements, these will automatically be accounted for. So there's a few issues here but the main one I'm seeking insight into is how to map a runtime argument to a class attribute. I hope I've provided enough detail to adequately describe what I'm trying to do...

  Read the article

• Visual Studio C++ list iterator not decementable

  - by user69514

  I keep getting an error on visual studio that says list iterator not decrementable: line 256 My program works fine on Linux, but visual studio compiler throws this error. Dammit this is why I hate windows. Why can't the world run on Linux? Anyway, do you see what my problem is? #include <iostream> #include <fstream> #include <sstream> #include <list> using namespace std; int main(){ /** create the list **/ list<int> l; /** create input stream to read file **/ ifstream inputstream("numbers.txt"); /** read the numbers and add them to list **/ if( inputstream.is_open() ){ string line; istringstream instream; while( getline(inputstream, line) ){ instream.clear(); instream.str(line); /** get he five int's **/ int one, two, three, four, five; instream >> one >> two >> three >> four >> five; /** add them to the list **/ l.push_back(one); l.push_back(two); l.push_back(three); l.push_back(four); l.push_back(five); }//end while loop }//end if /** close the stream **/ inputstream.close(); /** display the list **/ cout << "List Read:" << endl; list<int>::iterator i; for( i=l.begin(); i != l.end(); ++i){ cout << *i << " "; } cout << endl << endl; /** now sort the list **/ l.sort(); /** display the list **/ cout << "Sorted List (head to tail):" << endl; for( i=l.begin(); i != l.end(); ++i){ cout << *i << " "; } cout << endl; list<int> lReversed; for(i=l.begin(); i != l.end(); ++i){ lReversed.push_front(*i); } cout << "Sorted List (tail to head):" << endl; for(i=lReversed.begin(); i!=lReversed.end(); ++i){ cout << *i << " "; } cout << endl << endl; /** remove first biggest element and display **/ l.pop_back(); cout << "List after removing first biggest element:" << endl; cout << "Sorted List (head to tail):" << endl; for( i=l.begin(); i != l.end(); ++i){ cout << *i << " "; } cout << endl; cout << "Sorted List (tail to head):" << endl; lReversed.pop_front(); for(i=lReversed.begin(); i!=lReversed.end(); ++i){ cout << *i << " "; } cout << endl << endl; /** remove second biggest element and display **/ l.pop_back(); cout << "List after removing second biggest element:" << endl; cout << "Sorted List (head to tail):" << endl; for( i=l.begin(); i != l.end(); ++i){ cout << *i << " "; } cout << endl; lReversed.pop_front(); cout << "Sorted List (tail to head):" << endl; for(i=lReversed.begin(); i!=lReversed.end(); ++i){ cout << *i << " "; } cout << endl << endl; /** remove third biggest element and display **/ l.pop_back(); cout << "List after removing third biggest element:" << endl; cout << "Sorted List (head to tail):" << endl; for( i=l.begin(); i != l.end(); ++i){ cout << *i << " "; } cout << endl; cout << "Sorted List (tail to head):" << endl; lReversed.pop_front(); for(i=lReversed.begin(); i!=lReversed.end(); ++i){ cout << *i << " "; } cout << endl << endl; /** create frequency table **/ const int biggest = 1000; //create array size of biggest element int arr[biggest]; //set everything to zero for(int j=0; j<biggest+1; j++){ arr[j] = 0; } //now update number of occurences for( i=l.begin(); i != l.end(); i++){ arr[*i]++; } //now print the frequency table. only print where occurences greater than zero cout << "Final list frequency table: " << endl; for(int j=0; j<biggest+1; j++){ if( arr[j] > 0 ){ cout << j << ": " << arr[j] << " occurences" << endl; } } return 0; }//end main

  Read the article

• How to Upload a file from client to server using OFBIZ?

  - by SIVAKUMAR.J

  Hi all, Im new to ofbiz.So is my question is have any mistake forgive me for my mistakes.Im new to ofbiz so i did not know some terminologies in ofbiz.Sometimes my question is not clear because of lack of knowledge in ofbiz.So try to understand my question and give me a good solution with respect to my level.Because some solutions are in very high level cannot able to understand for me.So please give the solution with good examples. My problem is i created a project inside the ofbiz/hot-deploy folder namely "productionmgntSystem".Inside the folder "ofbiz\hot-deploy\productionmgntSystem\webapp\productionmgntSystem" i created a .ftl file namely "app_details_1.ftl" .The following are the coding of this file <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"> <title>Insert title here</title> <script TYPE="TEXT/JAVASCRIPT" language=""JAVASCRIPT"> function uploadFile() { //alert("Before calling upload.jsp"); window.location='<@ofbizUrl>testing_service1</@ofbizUrl>' } </script> </head> <!-- <form action="<@ofbizUrl>testing_service1</@ofbizUrl>" enctype="multipart/form-data" name="app_details_frm"> --> <form action="<@ofbizUrl>logout1</@ofbizUrl>" enctype="multipart/form-data" name="app_details_frm"> <center style="height: 299px; "> <table border="0" style="height: 177px; width: 788px"> <tr style="height: 115px; "> <td style="width: 103px; "> <td style="width: 413px; "><h1>APPLICATION DETAILS</h1> <td style="width: 55px; "> </tr> <tr> <td style="width: 125px; ">Application name : </td> <td> <input name="app_name_txt" id="txt_1" value=" " /> </td> </tr> <tr> <td style="width: 125px; ">Excell sheet &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;: </td> <td> <input type="file" name="filename"/> </td> </tr> <tr> <td> <!-- <input type="button" name="logout1_cmd" value="Logout" onclick="logout1()"/> --> <input type="submit" name="logout_cmd" value="logout"/> </td> <td> <!-- <input type="submit" name="upload_cmd" value="Submit" /> --> <input type="button" name="upload1_cmd" value="Upload" onclick="uploadFile()"/> </td> </tr> </table> </center> </form> </html> the following coding is present in the file "ofbiz\hot-deploy\productionmgntSystem\webapp\productionmgntSystem\WEB-INF\controller.xml" ...... ....... ........ <request-map uri="testing_service1"> <security https="true" auth="true"/> <event type="java" path="org.ofbiz.productionmgntSystem.web_app_req.WebServices1" invoke="testingService"/> <response name="ok" type="view" value="ok_view"/> <response name="exception" type="view" value="exception_view"/> </request-map> .......... ............ .......... <view-map name="ok_view" type="ftl" page="ok_view.ftl"/> <view-map name="exception_view" type="ftl" page="exception_view.ftl"/> ................ ............. ............. The following are the coding present in the file "ofbiz\hot-deploy\productionmgntSystem\src\org\ofbiz\productionmgntSystem\web_app_req\WebServices1.java" package org.ofbiz.productionmgntSystem.web_app_req; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; import java.io.DataInputStream; import java.io.FileOutputStream; import java.io.IOException; public class WebServices1 { public static String testingService(HttpServletRequest request, HttpServletResponse response) { //int i=0; String result="ok"; System.out.println("\n\n\t*************************************\n\tInside WebServices1.testingService(HttpServletRequest request, HttpServletResponse response)- Start"); String contentType=request.getContentType(); System.out.println("\n\n\t*************************************\n\tInside WebServices1.testingService(HttpServletRequest request, HttpServletResponse response)- contentType : "+contentType); String str=new String(); // response.setContentType("text/html"); //PrintWriter writer; if ((contentType != null) && (contentType.indexOf("multipart/form-data") >= 0)) { System.out.println("\n\n\t**********************************\n\tInside WebServices1.testingService(HttpServletRequest request, HttpServletResponse response) after if (contentType != null)"); try { // writer=response.getWriter(); System.out.println("\n\n\t**********************************\n\tInside WebServices1.testingService(HttpServletRequest request, HttpServletResponse response) - try Start"); DataInputStream in = new DataInputStream(request.getInputStream()); int formDataLength = request.getContentLength(); byte dataBytes[] = new byte[formDataLength]; int byteRead = 0; int totalBytesRead = 0; //this loop converting the uploaded file into byte code while (totalBytesRead < formDataLength) { byteRead = in.read(dataBytes, totalBytesRead,formDataLength); totalBytesRead += byteRead; } String file = new String(dataBytes); //for saving the file name String saveFile = file.substring(file.indexOf("filename=\"") + 10); saveFile = saveFile.substring(0, saveFile.indexOf("\n")); saveFile = saveFile.substring(saveFile.lastIndexOf("\\")+ 1,saveFile.indexOf("\"")); int lastIndex = contentType.lastIndexOf("="); String boundary = contentType.substring(lastIndex + 1,contentType.length()); int pos; //extracting the index of file pos = file.indexOf("filename=\""); pos = file.indexOf("\n", pos) + 1; pos = file.indexOf("\n", pos) + 1; pos = file.indexOf("\n", pos) + 1; int boundaryLocation = file.indexOf(boundary, pos) - 4; int startPos = ((file.substring(0, pos)).getBytes()).length; int endPos = ((file.substring(0, boundaryLocation)).getBytes()).length; //creating a new file with the same name and writing the content in new file FileOutputStream fileOut = new FileOutputStream("/"+saveFile); fileOut.write(dataBytes, startPos, (endPos - startPos)); fileOut.flush(); fileOut.close(); System.out.println("\n\n\t**********************************\n\tInside WebServices1.testingService(HttpServletRequest request, HttpServletResponse response) - try End"); } catch(IOException ioe) { System.out.println("\n\n\t*********************************\n\tInside WebServices1.testingService(HttpServletRequest request, HttpServletResponse response) - Catch IOException"); //ioe.printStackTrace(); return("exception"); } catch(Exception ex) { System.out.println("\n\n\t*********************************\n\tInside WebServices1.testingService(HttpServletRequest request, HttpServletResponse response) - Catch Exception"); return("exception"); } } else { System.out.println("\n\n\t********************************\n\tInside WebServices1.testingService(HttpServletRequest request, HttpServletResponse response) else part"); result="exception"; } System.out.println("\n\n\t*************************************\n\tInside WebServices1.testingService(HttpServletRequest request, HttpServletResponse response)- End"); return(result); } } I want to upload a file to the server.The file is get from user "<input type="file"..> tag in the "app_details_1.ftl" file & it is updated into the server by using the method "testingService(HttpServletRequest request, HttpServletResponse response)" in the class "WebServices1".But the file is not uploaded. Give me a good solution for uploading a file to the server. Thanks & Regards, Sivakumar.J

  Read the article

• Using R to Analyze G1GC Log Files

  - by user12620111

  Using R to Analyze G1GC Log Files body, td { font-family: sans-serif; background-color: white; font-size: 12px; margin: 8px; } tt, code, pre { font-family: 'DejaVu Sans Mono', 'Droid Sans Mono', 'Lucida Console', Consolas, Monaco, monospace; } h1 { font-size:2.2em; } h2 { font-size:1.8em; } h3 { font-size:1.4em; } h4 { font-size:1.0em; } h5 { font-size:0.9em; } h6 { font-size:0.8em; } a:visited { color: rgb(50%, 0%, 50%); } pre { margin-top: 0; max-width: 95%; border: 1px solid #ccc; white-space: pre-wrap; } pre code { display: block; padding: 0.5em; } code.r, code.cpp { background-color: #F8F8F8; } table, td, th { border: none; } blockquote { color:#666666; margin:0; padding-left: 1em; border-left: 0.5em #EEE solid; } hr { height: 0px; border-bottom: none; border-top-width: thin; border-top-style: dotted; border-top-color: #999999; } @media print { * { background: transparent !important; color: black !important; filter:none !important; -ms-filter: none !important; } body { font-size:12pt; max-width:100%; } a, a:visited { text-decoration: underline; } hr { visibility: hidden; page-break-before: always; } pre, blockquote { padding-right: 1em; page-break-inside: avoid; } tr, img { page-break-inside: avoid; } img { max-width: 100% !important; } @page :left { margin: 15mm 20mm 15mm 10mm; } @page :right { margin: 15mm 10mm 15mm 20mm; } p, h2, h3 { orphans: 3; widows: 3; } h2, h3 { page-break-after: avoid; } } pre .operator, pre .paren { color: rgb(104, 118, 135) } pre .literal { color: rgb(88, 72, 246) } pre .number { color: rgb(0, 0, 205); } pre .comment { color: rgb(76, 136, 107); } pre .keyword { color: rgb(0, 0, 255); } pre .identifier { color: rgb(0, 0, 0); } pre .string { color: rgb(3, 106, 7); } var hljs=new function(){function m(p){return p.replace(/&/gm,"&").replace(/"}while(y.length||w.length){var v=u().splice(0,1)[0];z+=m(x.substr(q,v.offset-q));q=v.offset;if(v.event=="start"){z+=t(v.node);s.push(v.node)}else{if(v.event=="stop"){var p,r=s.length;do{r--;p=s[r];z+=("")}while(p!=v.node);s.splice(r,1);while(r'+M[0]+""}else{r+=M[0]}O=P.lR.lastIndex;M=P.lR.exec(L)}return r+L.substr(O,L.length-O)}function J(L,M){if(M.sL&&e[M.sL]){var r=d(M.sL,L);x+=r.keyword_count;return r.value}else{return F(L,M)}}function I(M,r){var L=M.cN?'':"";if(M.rB){y+=L;M.buffer=""}else{if(M.eB){y+=m(r)+L;M.buffer=""}else{y+=L;M.buffer=r}}D.push(M);A+=M.r}function G(N,M,Q){var R=D[D.length-1];if(Q){y+=J(R.buffer+N,R);return false}var P=q(M,R);if(P){y+=J(R.buffer+N,R);I(P,M);return P.rB}var L=v(D.length-1,M);if(L){var O=R.cN?"":"";if(R.rE){y+=J(R.buffer+N,R)+O}else{if(R.eE){y+=J(R.buffer+N,R)+O+m(M)}else{y+=J(R.buffer+N+M,R)+O}}while(L1){O=D[D.length-2].cN?"":"";y+=O;L--;D.length--}var r=D[D.length-1];D.length--;D[D.length-1].buffer="";if(r.starts){I(r.starts,"")}return R.rE}if(w(M,R)){throw"Illegal"}}var E=e[B];var D=[E.dM];var A=0;var x=0;var y="";try{var s,u=0;E.dM.buffer="";do{s=p(C,u);var t=G(s[0],s[1],s[2]);u+=s[0].length;if(!t){u+=s[1].length}}while(!s[2]);if(D.length1){throw"Illegal"}return{r:A,keyword_count:x,value:y}}catch(H){if(H=="Illegal"){return{r:0,keyword_count:0,value:m(C)}}else{throw H}}}function g(t){var p={keyword_count:0,r:0,value:m(t)};var r=p;for(var q in e){if(!e.hasOwnProperty(q)){continue}var s=d(q,t);s.language=q;if(s.keyword_count+s.rr.keyword_count+r.r){r=s}if(s.keyword_count+s.rp.keyword_count+p.r){r=p;p=s}}if(r.language){p.second_best=r}return p}function i(r,q,p){if(q){r=r.replace(/^((]+|\t)+)/gm,function(t,w,v,u){return w.replace(/\t/g,q)})}if(p){r=r.replace(/\n/g,"")}return r}function n(t,w,r){var x=h(t,r);var v=a(t);var y,s;if(v){y=d(v,x)}else{return}var q=c(t);if(q.length){s=document.createElement("pre");s.innerHTML=y.value;y.value=k(q,c(s),x)}y.value=i(y.value,w,r);var u=t.className;if(!u.match("(\\s|^)(language-)?"+v+"(\\s|$)")){u=u?(u+" "+v):v}if(/MSIE [678]/.test(navigator.userAgent)&&t.tagName=="CODE"&&t.parentNode.tagName=="PRE"){s=t.parentNode;var p=document.createElement("div");p.innerHTML=""+y.value+"";t=p.firstChild.firstChild;p.firstChild.cN=s.cN;s.parentNode.replaceChild(p.firstChild,s)}else{t.innerHTML=y.value}t.className=u;t.result={language:v,kw:y.keyword_count,re:y.r};if(y.second_best){t.second_best={language:y.second_best.language,kw:y.second_best.keyword_count,re:y.second_best.r}}}function o(){if(o.called){return}o.called=true;var r=document.getElementsByTagName("pre");for(var p=0;p|=||=||=|\\?|\\[|\\{|\\(|\\^|\\^=|\\||\\|=|\\|\\||~";this.ER="(?![\\s\\S])";this.BE={b:"\\\\.",r:0};this.ASM={cN:"string",b:"'",e:"'",i:"\\n",c:[this.BE],r:0};this.QSM={cN:"string",b:'"',e:'"',i:"\\n",c:[this.BE],r:0};this.CLCM={cN:"comment",b:"//",e:"$"};this.CBLCLM={cN:"comment",b:"/\\*",e:"\\*/"};this.HCM={cN:"comment",b:"#",e:"$"};this.NM={cN:"number",b:this.NR,r:0};this.CNM={cN:"number",b:this.CNR,r:0};this.BNM={cN:"number",b:this.BNR,r:0};this.inherit=function(r,s){var p={};for(var q in r){p[q]=r[q]}if(s){for(var q in s){p[q]=s[q]}}return p}}();hljs.LANGUAGES.cpp=function(){var a={keyword:{"false":1,"int":1,"float":1,"while":1,"private":1,"char":1,"catch":1,"export":1,virtual:1,operator:2,sizeof:2,dynamic_cast:2,typedef:2,const_cast:2,"const":1,struct:1,"for":1,static_cast:2,union:1,namespace:1,unsigned:1,"long":1,"throw":1,"volatile":2,"static":1,"protected":1,bool:1,template:1,mutable:1,"if":1,"public":1,friend:2,"do":1,"return":1,"goto":1,auto:1,"void":2,"enum":1,"else":1,"break":1,"new":1,extern:1,using:1,"true":1,"class":1,asm:1,"case":1,typeid:1,"short":1,reinterpret_cast:2,"default":1,"double":1,register:1,explicit:1,signed:1,typename:1,"try":1,"this":1,"switch":1,"continue":1,wchar_t:1,inline:1,"delete":1,alignof:1,char16_t:1,char32_t:1,constexpr:1,decltype:1,noexcept:1,nullptr:1,static_assert:1,thread_local:1,restrict:1,_Bool:1,complex:1},built_in:{std:1,string:1,cin:1,cout:1,cerr:1,clog:1,stringstream:1,istringstream:1,ostringstream:1,auto_ptr:1,deque:1,list:1,queue:1,stack:1,vector:1,map:1,set:1,bitset:1,multiset:1,multimap:1,unordered_set:1,unordered_map:1,unordered_multiset:1,unordered_multimap:1,array:1,shared_ptr:1}};return{dM:{k:a,i:"",k:a,r:10,c:["self"]}]}}}();hljs.LANGUAGES.r={dM:{c:[hljs.HCM,{cN:"number",b:"\\b0[xX][0-9a-fA-F]+[Li]?\\b",e:hljs.IMMEDIATE_RE,r:0},{cN:"number",b:"\\b\\d+(?:[eE][+\\-]?\\d*)?L\\b",e:hljs.IMMEDIATE_RE,r:0},{cN:"number",b:"\\b\\d+\\.(?!\\d)(?:i\\b)?",e:hljs.IMMEDIATE_RE,r:1},{cN:"number",b:"\\b\\d+(?:\\.\\d*)?(?:[eE][+\\-]?\\d*)?i?\\b",e:hljs.IMMEDIATE_RE,r:0},{cN:"number",b:"\\.\\d+(?:[eE][+\\-]?\\d*)?i?\\b",e:hljs.IMMEDIATE_RE,r:1},{cN:"keyword",b:"(?:tryCatch|library|setGeneric|setGroupGeneric)\\b",e:hljs.IMMEDIATE_RE,r:10},{cN:"keyword",b:"\\.\\.\\.",e:hljs.IMMEDIATE_RE,r:10},{cN:"keyword",b:"\\.\\.\\d+(?![\\w.])",e:hljs.IMMEDIATE_RE,r:10},{cN:"keyword",b:"\\b(?:function)",e:hljs.IMMEDIATE_RE,r:2},{cN:"keyword",b:"(?:if|in|break|next|repeat|else|for|return|switch|while|try|stop|warning|require|attach|detach|source|setMethod|setClass)\\b",e:hljs.IMMEDIATE_RE,r:1},{cN:"literal",b:"(?:NA|NA_integer_|NA_real_|NA_character_|NA_complex_)\\b",e:hljs.IMMEDIATE_RE,r:10},{cN:"literal",b:"(?:NULL|TRUE|FALSE|T|F|Inf|NaN)\\b",e:hljs.IMMEDIATE_RE,r:1},{cN:"identifier",b:"[a-zA-Z.][a-zA-Z0-9._]*\\b",e:hljs.IMMEDIATE_RE,r:0},{cN:"operator",b:"|=||   Using R to Analyze G1GC Log Files   Using R to Analyze G1GC Log Files Introduction Working in Oracle Platform Integration gives an engineer opportunities to work on a wide array of technologies. My team’s goal is to make Oracle applications run best on the Solaris/SPARC platform. When looking for bottlenecks in a modern applications, one needs to be aware of not only how the CPUs and operating system are executing, but also network, storage, and in some cases, the Java Virtual Machine. I was recently presented with about 1.5 GB of Java Garbage First Garbage Collector log file data. If you’re not familiar with the subject, you might want to review Garbage First Garbage Collector Tuning by Monica Beckwith. The customer had been running Java HotSpot 1.6.0_31 to host a web application server. I was told that the Solaris/SPARC server was running a Java process launched using a commmand line that included the following flags: -d64 -Xms9g -Xmx9g -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:InitiatingHeapOccupancyPercent=80 -XX:PermSize=256m -XX:MaxPermSize=256m -XX:+PrintGC -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGC -XX:+PrintGCDateStamps -XX:+PrintFlagsFinal -XX:+DisableExplicitGC -XX:+UnlockExperimentalVMOptions -XX:ParallelGCThreads=8 Several sources on the internet indicate that if I were to print out the 1.5 GB of log files, it would require enough paper to fill the bed of a pick up truck. Of course, it would be fruitless to try to scan the log files by hand. Tools will be required to summarize the contents of the log files. Others have encountered large Java garbage collection log files. There are existing tools to analyze the log files: IBM’s GC toolkit The chewiebug GCViewer gchisto HPjmeter Instead of using one of the other tools listed, I decide to parse the log files with standard Unix tools, and analyze the data with R. Data Cleansing The log files arrived in two different formats. I guess that the difference is that one set of log files was generated using a more verbose option, maybe -XX:+PrintHeapAtGC, and the other set of log files was generated without that option. Format 1 In some of the log files, the log files with the less verbose format, a single trace, i.e. the report of a singe garbage collection event, looks like this: {Heap before GC invocations=12280 (full 61): garbage-first heap total 9437184K, used 7499918K [0xfffffffd00000000, 0xffffffff40000000, 0xffffffff40000000) region size 4096K, 1 young (4096K), 0 survivors (0K) compacting perm gen total 262144K, used 144077K [0xffffffff40000000, 0xffffffff50000000, 0xffffffff50000000) the space 262144K, 54% used [0xffffffff40000000, 0xffffffff48cb3758, 0xffffffff48cb3800, 0xffffffff50000000) No shared spaces configured. 2014-05-14T07:24:00.988-0700: 60586.353: [GC pause (young) 7324M->7320M(9216M), 0.1567265 secs] Heap after GC invocations=12281 (full 61): garbage-first heap total 9437184K, used 7496533K [0xfffffffd00000000, 0xffffffff40000000, 0xffffffff40000000) region size 4096K, 0 young (0K), 0 survivors (0K) compacting perm gen total 262144K, used 144077K [0xffffffff40000000, 0xffffffff50000000, 0xffffffff50000000) the space 262144K, 54% used [0xffffffff40000000, 0xffffffff48cb3758, 0xffffffff48cb3800, 0xffffffff50000000) No shared spaces configured. } A simple grep can be used to extract a summary: $ grep "\[ GC pause (young" g1gc.log 2014-05-13T13:24:35.091-0700: 3.109: [GC pause (young) 20M->5029K(9216M), 0.0146328 secs] 2014-05-13T13:24:35.440-0700: 3.459: [GC pause (young) 9125K->6077K(9216M), 0.0086723 secs] 2014-05-13T13:24:37.581-0700: 5.599: [GC pause (young) 25M->8470K(9216M), 0.0203820 secs] 2014-05-13T13:24:42.686-0700: 10.704: [GC pause (young) 44M->15M(9216M), 0.0288848 secs] 2014-05-13T13:24:48.941-0700: 16.958: [GC pause (young) 51M->20M(9216M), 0.0491244 secs] 2014-05-13T13:24:56.049-0700: 24.066: [GC pause (young) 92M->26M(9216M), 0.0525368 secs] 2014-05-13T13:25:34.368-0700: 62.383: [GC pause (young) 602M->68M(9216M), 0.1721173 secs] But that format wasn't easily read into R, so I needed to be a bit more tricky. I used the following Unix command to create a summary file that was easy for R to read. $ echo "SecondsSinceLaunch BeforeSize AfterSize TotalSize RealTime" $ grep "\[GC pause (young" g1gc.log | grep -v mark | sed -e 's/[A-SU-z\(\),]/ /g' -e 's/->/ /' -e 's/: / /g' | more SecondsSinceLaunch BeforeSize AfterSize TotalSize RealTime 2014-05-13T13:24:35.091-0700 3.109 20 5029 9216 0.0146328 2014-05-13T13:24:35.440-0700 3.459 9125 6077 9216 0.0086723 2014-05-13T13:24:37.581-0700 5.599 25 8470 9216 0.0203820 2014-05-13T13:24:42.686-0700 10.704 44 15 9216 0.0288848 2014-05-13T13:24:48.941-0700 16.958 51 20 9216 0.0491244 2014-05-13T13:24:56.049-0700 24.066 92 26 9216 0.0525368 2014-05-13T13:25:34.368-0700 62.383 602 68 9216 0.1721173 Format 2 In some of the log files, the log files with the more verbose format, a single trace, i.e. the report of a singe garbage collection event, was more complicated than Format 1. Here is a text file with an example of a single G1GC trace in the second format. As you can see, it is quite complicated. It is nice that there is so much information available, but the level of detail can be overwhelming. I wrote this awk script (download) to summarize each trace on a single line. #!/usr/bin/env awk -f BEGIN { printf("SecondsSinceLaunch IncrementalCount FullCount UserTime SysTime RealTime BeforeSize AfterSize TotalSize\n") } ###################### # Save count data from lines that are at the start of each G1GC trace. # Each trace starts out like this: # {Heap before GC invocations=14 (full 0): # garbage-first heap total 9437184K, used 325496K [0xfffffffd00000000, 0xffffffff40000000, 0xffffffff40000000) ###################### /{Heap.*full/{ gsub ( "\\)" , "" ); nf=split($0,a,"="); split(a[2],b," "); getline; if ( match($0, "first") ) { G1GC=1; IncrementalCount=b[1]; FullCount=substr( b[3], 1, length(b[3])-1 ); } else { G1GC=0; } } ###################### # Pull out time stamps that are in lines with this format: # 2014-05-12T14:02:06.025-0700: 94.312: [GC pause (young), 0.08870154 secs] ###################### /GC pause/ { DateTime=$1; SecondsSinceLaunch=substr($2, 1, length($2)-1); } ###################### # Heap sizes are in lines that look like this: # [ 4842M->4838M(9216M)] ###################### /\[ .*]$/ { gsub ( "\\[" , "" ); gsub ( "\ \]" , "" ); gsub ( "->" , " " ); gsub ( "\\( " , " " ); gsub ( "\ \)" , " " ); split($0,a," "); if ( split(a[1],b,"M") > 1 ) {BeforeSize=b[1]*1024;} if ( split(a[1],b,"K") > 1 ) {BeforeSize=b[1];} if ( split(a[2],b,"M") > 1 ) {AfterSize=b[1]*1024;} if ( split(a[2],b,"K") > 1 ) {AfterSize=b[1];} if ( split(a[3],b,"M") > 1 ) {TotalSize=b[1]*1024;} if ( split(a[3],b,"K") > 1 ) {TotalSize=b[1];} } ###################### # Emit an output line when you find input that looks like this: # [Times: user=1.41 sys=0.08, real=0.24 secs] ###################### /\[Times/ { if (G1GC==1) { gsub ( "," , "" ); split($2,a,"="); UserTime=a[2]; split($3,a,"="); SysTime=a[2]; split($4,a,"="); RealTime=a[2]; print DateTime,SecondsSinceLaunch,IncrementalCount,FullCount,UserTime,SysTime,RealTime,BeforeSize,AfterSize,TotalSize; G1GC=0; } } The resulting summary is about 25X smaller that the original file, but still difficult for a human to digest. SecondsSinceLaunch IncrementalCount FullCount UserTime SysTime RealTime BeforeSize AfterSize TotalSize ... 2014-05-12T18:36:34.669-0700: 3985.744 561 0 0.57 0.06 0.16 1724416 1720320 9437184 2014-05-12T18:36:34.839-0700: 3985.914 562 0 0.51 0.06 0.19 1724416 1720320 9437184 2014-05-12T18:36:35.069-0700: 3986.144 563 0 0.60 0.04 0.27 1724416 1721344 9437184 2014-05-12T18:36:35.354-0700: 3986.429 564 0 0.33 0.04 0.09 1725440 1722368 9437184 2014-05-12T18:36:35.545-0700: 3986.620 565 0 0.58 0.04 0.17 1726464 1722368 9437184 2014-05-12T18:36:35.726-0700: 3986.801 566 0 0.43 0.05 0.12 1726464 1722368 9437184 2014-05-12T18:36:35.856-0700: 3986.930 567 0 0.30 0.04 0.07 1726464 1723392 9437184 2014-05-12T18:36:35.947-0700: 3987.023 568 0 0.61 0.04 0.26 1727488 1723392 9437184 2014-05-12T18:36:36.228-0700: 3987.302 569 0 0.46 0.04 0.16 1731584 1724416 9437184 Reading the Data into R Once the GC log data had been cleansed, either by processing the first format with the shell script, or by processing the second format with the awk script, it was easy to read the data into R. g1gc.df = read.csv("summary.txt", row.names = NULL, stringsAsFactors=FALSE,sep="") str(g1gc.df) ## 'data.frame': 8307 obs. of 10 variables: ## $ row.names : chr "2014-05-12T14:00:32.868-0700:" "2014-05-12T14:00:33.179-0700:" "2014-05-12T14:00:33.677-0700:" "2014-05-12T14:00:35.538-0700:" ... ## $ SecondsSinceLaunch: num 1.16 1.47 1.97 3.83 6.1 ... ## $ IncrementalCount : int 0 1 2 3 4 5 6 7 8 9 ... ## $ FullCount : int 0 0 0 0 0 0 0 0 0 0 ... ## $ UserTime : num 0.11 0.05 0.04 0.21 0.08 0.26 0.31 0.33 0.34 0.56 ... ## $ SysTime : num 0.04 0.01 0.01 0.05 0.01 0.06 0.07 0.06 0.07 0.09 ... ## $ RealTime : num 0.02 0.02 0.01 0.04 0.02 0.04 0.05 0.04 0.04 0.06 ... ## $ BeforeSize : int 8192 5496 5768 22528 24576 43008 34816 53248 55296 93184 ... ## $ AfterSize : int 1400 1672 2557 4907 7072 14336 16384 18432 19456 21504 ... ## $ TotalSize : int 9437184 9437184 9437184 9437184 9437184 9437184 9437184 9437184 9437184 9437184 ... head(g1gc.df) ## row.names SecondsSinceLaunch IncrementalCount ## 1 2014-05-12T14:00:32.868-0700: 1.161 0 ## 2 2014-05-12T14:00:33.179-0700: 1.472 1 ## 3 2014-05-12T14:00:33.677-0700: 1.969 2 ## 4 2014-05-12T14:00:35.538-0700: 3.830 3 ## 5 2014-05-12T14:00:37.811-0700: 6.103 4 ## 6 2014-05-12T14:00:41.428-0700: 9.720 5 ## FullCount UserTime SysTime RealTime BeforeSize AfterSize TotalSize ## 1 0 0.11 0.04 0.02 8192 1400 9437184 ## 2 0 0.05 0.01 0.02 5496 1672 9437184 ## 3 0 0.04 0.01 0.01 5768 2557 9437184 ## 4 0 0.21 0.05 0.04 22528 4907 9437184 ## 5 0 0.08 0.01 0.02 24576 7072 9437184 ## 6 0 0.26 0.06 0.04 43008 14336 9437184 Basic Statistics Once the data has been read into R, simple statistics are very easy to generate. All of the numbers from high school statistics are available via simple commands. For example, generate a summary of every column: summary(g1gc.df) ## row.names SecondsSinceLaunch IncrementalCount FullCount ## Length:8307 Min. : 1 Min. : 0 Min. : 0.0 ## Class :character 1st Qu.: 9977 1st Qu.:2048 1st Qu.: 0.0 ## Mode :character Median :12855 Median :4136 Median : 12.0 ## Mean :12527 Mean :4156 Mean : 31.6 ## 3rd Qu.:15758 3rd Qu.:6262 3rd Qu.: 61.0 ## Max. :55484 Max. :8391 Max. :113.0 ## UserTime SysTime RealTime BeforeSize ## Min. :0.040 Min. :0.0000 Min. : 0.0 Min. : 5476 ## 1st Qu.:0.470 1st Qu.:0.0300 1st Qu.: 0.1 1st Qu.:5137920 ## Median :0.620 Median :0.0300 Median : 0.1 Median :6574080 ## Mean :0.751 Mean :0.0355 Mean : 0.3 Mean :5841855 ## 3rd Qu.:0.920 3rd Qu.:0.0400 3rd Qu.: 0.2 3rd Qu.:7084032 ## Max. :3.370 Max. :1.5600 Max. :488.1 Max. :8696832 ## AfterSize TotalSize ## Min. : 1380 Min. :9437184 ## 1st Qu.:5002752 1st Qu.:9437184 ## Median :6559744 Median :9437184 ## Mean :5785454 Mean :9437184 ## 3rd Qu.:7054336 3rd Qu.:9437184 ## Max. :8482816 Max. :9437184 Q: What is the total amount of User CPU time spent in garbage collection? sum(g1gc.df$UserTime) ## [1] 6236 As you can see, less than two hours of CPU time was spent in garbage collection. Is that too much? To find the percentage of time spent in garbage collection, divide the number above by total_elapsed_time*CPU_count. In this case, there are a lot of CPU’s and it turns out the the overall amount of CPU time spent in garbage collection isn’t a problem when viewed in isolation. When calculating rates, i.e. events per unit time, you need to ask yourself if the rate is homogenous across the time period in the log file. Does the log file include spikes of high activity that should be separately analyzed? Averaging in data from nights and weekends with data from business hours may alias problems. If you have a reason to suspect that the garbage collection rates include peaks and valleys that need independent analysis, see the “Time Series” section, below. Q: How much garbage is collected on each pass? The amount of heap space that is recovered per GC pass is surprisingly low: At least one collection didn’t recover any data. (“Min.=0”) 25% of the passes recovered 3MB or less. (“1st Qu.=3072”) Half of the GC passes recovered 4MB or less. (“Median=4096”) The average amount recovered was 56MB. (“Mean=56390”) 75% of the passes recovered 36MB or less. (“3rd Qu.=36860”) At least one pass recovered 2GB. (“Max.=2121000”) g1gc.df$Delta = g1gc.df$BeforeSize - g1gc.df$AfterSize summary(g1gc.df$Delta) ## Min. 1st Qu. Median Mean 3rd Qu. Max. ## 0 3070 4100 56400 36900 2120000 Q: What is the maximum User CPU time for a single collection? The worst garbage collection (“Max.”) is many standard deviations away from the mean. The data appears to be right skewed. summary(g1gc.df$UserTime) ## Min. 1st Qu. Median Mean 3rd Qu. Max. ## 0.040 0.470 0.620 0.751 0.920 3.370 sd(g1gc.df$UserTime) ## [1] 0.3966 Basic Graphics Once the data is in R, it is trivial to plot the data with formats including dot plots, line charts, bar charts (simple, stacked, grouped), pie charts, boxplots, scatter plots histograms, and kernel density plots. Histogram of User CPU Time per Collection I don't think that this graph requires any explanation. hist(g1gc.df$UserTime, main="User CPU Time per Collection", xlab="Seconds", ylab="Frequency") Box plot to identify outliers When the initial data is viewed with a box plot, you can see the one crazy outlier in the real time per GC. Save this data point for future analysis and drop the outlier so that it’s not throwing off our statistics. Now the box plot shows many outliers, which will be examined later, using times series analysis. Notice that the scale of the x-axis changes drastically once the crazy outlier is removed. par(mfrow=c(2,1)) boxplot(g1gc.df$UserTime,g1gc.df$SysTime,g1gc.df$RealTime, main="Box Plot of Time per GC\n(dominated by a crazy outlier)", names=c("usr","sys","elapsed"), xlab="Seconds per GC", ylab="Time (Seconds)", horizontal = TRUE, outcol="red") crazy.outlier.df=g1gc.df[g1gc.df$RealTime > 400,] g1gc.df=g1gc.df[g1gc.df$RealTime < 400,] boxplot(g1gc.df$UserTime,g1gc.df$SysTime,g1gc.df$RealTime, main="Box Plot of Time per GC\n(crazy outlier excluded)", names=c("usr","sys","elapsed"), xlab="Seconds per GC", ylab="Time (Seconds)", horizontal = TRUE, outcol="red") box(which = "outer", lty = "solid") Here is the crazy outlier for future analysis: crazy.outlier.df ## row.names SecondsSinceLaunch IncrementalCount ## 8233 2014-05-12T23:15:43.903-0700: 20741 8316 ## FullCount UserTime SysTime RealTime BeforeSize AfterSize TotalSize ## 8233 112 0.55 0.42 488.1 8381440 8235008 9437184 ## Delta ## 8233 146432 R Time Series Data To analyze the garbage collection as a time series, I’ll use Z’s Ordered Observations (zoo). “zoo is the creator for an S3 class of indexed totally ordered observations which includes irregular time series.” require(zoo) ## Loading required package: zoo ## ## Attaching package: 'zoo' ## ## The following objects are masked from 'package:base': ## ## as.Date, as.Date.numeric head(g1gc.df[,1]) ## [1] "2014-05-12T14:00:32.868-0700:" "2014-05-12T14:00:33.179-0700:" ## [3] "2014-05-12T14:00:33.677-0700:" "2014-05-12T14:00:35.538-0700:" ## [5] "2014-05-12T14:00:37.811-0700:" "2014-05-12T14:00:41.428-0700:" options("digits.secs"=3) times=as.POSIXct( g1gc.df[,1], format="%Y-%m-%dT%H:%M:%OS%z:") g1gc.z = zoo(g1gc.df[,-c(1)], order.by=times) head(g1gc.z) ## SecondsSinceLaunch IncrementalCount FullCount ## 2014-05-12 17:00:32.868 1.161 0 0 ## 2014-05-12 17:00:33.178 1.472 1 0 ## 2014-05-12 17:00:33.677 1.969 2 0 ## 2014-05-12 17:00:35.538 3.830 3 0 ## 2014-05-12 17:00:37.811 6.103 4 0 ## 2014-05-12 17:00:41.427 9.720 5 0 ## UserTime SysTime RealTime BeforeSize AfterSize ## 2014-05-12 17:00:32.868 0.11 0.04 0.02 8192 1400 ## 2014-05-12 17:00:33.178 0.05 0.01 0.02 5496 1672 ## 2014-05-12 17:00:33.677 0.04 0.01 0.01 5768 2557 ## 2014-05-12 17:00:35.538 0.21 0.05 0.04 22528 4907 ## 2014-05-12 17:00:37.811 0.08 0.01 0.02 24576 7072 ## 2014-05-12 17:00:41.427 0.26 0.06 0.04 43008 14336 ## TotalSize Delta ## 2014-05-12 17:00:32.868 9437184 6792 ## 2014-05-12 17:00:33.178 9437184 3824 ## 2014-05-12 17:00:33.677 9437184 3211 ## 2014-05-12 17:00:35.538 9437184 17621 ## 2014-05-12 17:00:37.811 9437184 17504 ## 2014-05-12 17:00:41.427 9437184 28672 Example of Two Benchmark Runs in One Log File The data in the following graph is from a different log file, not the one of primary interest to this article. I’m including this image because it is an example of idle periods followed by busy periods. It would be uninteresting to average the rate of garbage collection over the entire log file period. More interesting would be the rate of garbage collect in the two busy periods. Are they the same or different? Your production data may be similar, for example, bursts when employees return from lunch and idle times on weekend evenings, etc. Once the data is in an R Time Series, you can analyze isolated time windows. Clipping the Time Series data Flashing back to our test case… Viewing the data as a time series is interesting. You can see that the work intensive time period is between 9:00 PM and 3:00 AM. Lets clip the data to the interesting period:     par(mfrow=c(2,1)) plot(g1gc.z$UserTime, type="h", main="User Time per GC\nTime: Complete Log File", xlab="Time of Day", ylab="CPU Seconds per GC", col="#1b9e77") clipped.g1gc.z=window(g1gc.z, start=as.POSIXct("2014-05-12 21:00:00"), end=as.POSIXct("2014-05-13 03:00:00")) plot(clipped.g1gc.z$UserTime, type="h", main="User Time per GC\nTime: Limited to Benchmark Execution", xlab="Time of Day", ylab="CPU Seconds per GC", col="#1b9e77") box(which = "outer", lty = "solid") Cumulative Incremental and Full GC count Here is the cumulative incremental and full GC count. When the line is very steep, it indicates that the GCs are repeating very quickly. Notice that the scale on the Y axis is different for full vs. incremental. plot(clipped.g1gc.z[,c(2:3)], main="Cumulative Incremental and Full GC count", xlab="Time of Day", col="#1b9e77") GC Analysis of Benchmark Execution using Time Series data In the following series of 3 graphs: The “After Size” show the amount of heap space in use after each garbage collection. Many Java objects are still referenced, i.e. alive, during each garbage collection. This may indicate that the application has a memory leak, or may indicate that the application has a very large memory footprint. Typically, an application's memory footprint plateau's in the early stage of execution. One would expect this graph to have a flat top. The steep decline in the heap space may indicate that the application crashed after 2:00. The second graph shows that the outliers in real execution time, discussed above, occur near 2:00. when the Java heap seems to be quite full. The third graph shows that Full GCs are infrequent during the first few hours of execution. The rate of Full GC's, (the slope of the cummulative Full GC line), changes near midnight.   plot(clipped.g1gc.z[,c("AfterSize","RealTime","FullCount")], xlab="Time of Day", col=c("#1b9e77","red","#1b9e77")) GC Analysis of heap recovered Each GC trace includes the amount of heap space in use before and after the individual GC event. During garbage coolection, unreferenced objects are identified, the space holding the unreferenced objects is freed, and thus, the difference in before and after usage indicates how much space has been freed. The following box plot and bar chart both demonstrate the same point - the amount of heap space freed per garbage colloection is surprisingly low. par(mfrow=c(2,1)) boxplot(as.vector(clipped.g1gc.z$Delta), main="Amount of Heap Recovered per GC Pass", xlab="Size in KB", horizontal = TRUE, col="red") hist(as.vector(clipped.g1gc.z$Delta), main="Amount of Heap Recovered per GC Pass", xlab="Size in KB", breaks=100, col="red") box(which = "outer", lty = "solid") This graph is the most interesting. The dark blue area shows how much heap is occupied by referenced Java objects. This represents memory that holds live data. The red fringe at the top shows how much data was recovered after each garbage collection. barplot(clipped.g1gc.z[,c("AfterSize","Delta")], col=c("#7570b3","#e7298a"), xlab="Time of Day", border=NA) legend("topleft", c("Live Objects","Heap Recovered on GC"), fill=c("#7570b3","#e7298a")) box(which = "outer", lty = "solid") When I discuss the data in the log files with the customer, I will ask for an explaination for the large amount of referenced data resident in the Java heap. There are two are posibilities: There is a memory leak and the amount of space required to hold referenced objects will continue to grow, limited only by the maximum heap size. After the maximum heap size is reached, the JVM will throw an “Out of Memory” exception every time that the application tries to allocate a new object. If this is the case, the aplication needs to be debugged to identify why old objects are referenced when they are no longer needed. The application has a legitimate requirement to keep a large amount of data in memory. The customer may want to further increase the maximum heap size. Another possible solution would be to partition the application across multiple cluster nodes, where each node has responsibility for managing a unique subset of the data. Conclusion In conclusion, R is a very powerful tool for the analysis of Java garbage collection log files. The primary difficulty is data cleansing so that information can be read into an R data frame. Once the data has been read into R, a rich set of tools may be used for thorough evaluation.

  Read the article

< Previous Page | 81 82 83 84 85 86  | Next Page >