Search Results

Search found 2210 results on 89 pages for 'sum'.

Page 33/89 | < Previous Page | 29 30 31 32 33 34 35 36 37 38 39 40  | Next Page >

  • C#/.NET Little Wonders: The Generic Func Delegates

    - by James Michael Hare
    Once again, in this series of posts I look at the parts of the .NET Framework that may seem trivial, but can help improve your code by making it easier to write and maintain. The index of all my past little wonders posts can be found here. Back in one of my three original “Little Wonders” Trilogy of posts, I had listed generic delegates as one of the Little Wonders of .NET.  Later, someone posted a comment saying said that they would love more detail on the generic delegates and their uses, since my original entry just scratched the surface of them. Last week, I began our look at some of the handy generic delegates built into .NET with a description of delegates in general, and the Action family of delegates.  For this week, I’ll launch into a look at the Func family of generic delegates and how they can be used to support generic, reusable algorithms and classes. Quick Delegate Recap Delegates are similar to function pointers in C++ in that they allow you to store a reference to a method.  They can store references to either static or instance methods, and can actually be used to chain several methods together in one delegate. Delegates are very type-safe and can be satisfied with any standard method, anonymous method, or a lambda expression.  They can also be null as well (refers to no method), so care should be taken to make sure that the delegate is not null before you invoke it. Delegates are defined using the keyword delegate, where the delegate’s type name is placed where you would typically place the method name: 1: // This delegate matches any method that takes string, returns nothing 2: public delegate void Log(string message); This delegate defines a delegate type named Log that can be used to store references to any method(s) that satisfies its signature (whether instance, static, lambda expression, etc.). Delegate instances then can be assigned zero (null) or more methods using the operator = which replaces the existing delegate chain, or by using the operator += which adds a method to the end of a delegate chain: 1: // creates a delegate instance named currentLogger defaulted to Console.WriteLine (static method) 2: Log currentLogger = Console.Out.WriteLine; 3:  4: // invokes the delegate, which writes to the console out 5: currentLogger("Hi Standard Out!"); 6:  7: // append a delegate to Console.Error.WriteLine to go to std error 8: currentLogger += Console.Error.WriteLine; 9:  10: // invokes the delegate chain and writes message to std out and std err 11: currentLogger("Hi Standard Out and Error!"); While delegates give us a lot of power, it can be cumbersome to re-create fairly standard delegate definitions repeatedly, for this purpose the generic delegates were introduced in various stages in .NET.  These support various method types with particular signatures. Note: a caveat with generic delegates is that while they can support multiple parameters, they do not match methods that contains ref or out parameters. If you want to a delegate to represent methods that takes ref or out parameters, you will need to create a custom delegate. We’ve got the Func… delegates Just like it’s cousin, the Action delegate family, the Func delegate family gives us a lot of power to use generic delegates to make classes and algorithms more generic.  Using them keeps us from having to define a new delegate type when need to make a class or algorithm generic. Remember that the point of the Action delegate family was to be able to perform an “action” on an item, with no return results.  Thus Action delegates can be used to represent most methods that take 0 to 16 arguments but return void.  You can assign a method The Func delegate family was introduced in .NET 3.5 with the advent of LINQ, and gives us the power to define a function that can be called on 0 to 16 arguments and returns a result.  Thus, the main difference between Action and Func, from a delegate perspective, is that Actions return nothing, but Funcs return a result. The Func family of delegates have signatures as follows: Func<TResult> – matches a method that takes no arguments, and returns value of type TResult. Func<T, TResult> – matches a method that takes an argument of type T, and returns value of type TResult. Func<T1, T2, TResult> – matches a method that takes arguments of type T1 and T2, and returns value of type TResult. Func<T1, T2, …, TResult> – and so on up to 16 arguments, and returns value of type TResult. These are handy because they quickly allow you to be able to specify that a method or class you design will perform a function to produce a result as long as the method you specify meets the signature. For example, let’s say you were designing a generic aggregator, and you wanted to allow the user to define how the values will be aggregated into the result (i.e. Sum, Min, Max, etc…).  To do this, we would ask the user of our class to pass in a method that would take the current total, the next value, and produce a new total.  A class like this could look like: 1: public sealed class Aggregator<TValue, TResult> 2: { 3: // holds method that takes previous result, combines with next value, creates new result 4: private Func<TResult, TValue, TResult> _aggregationMethod; 5:  6: // gets or sets the current result of aggregation 7: public TResult Result { get; private set; } 8:  9: // construct the aggregator given the method to use to aggregate values 10: public Aggregator(Func<TResult, TValue, TResult> aggregationMethod = null) 11: { 12: if (aggregationMethod == null) throw new ArgumentNullException("aggregationMethod"); 13:  14: _aggregationMethod = aggregationMethod; 15: } 16:  17: // method to add next value 18: public void Aggregate(TValue nextValue) 19: { 20: // performs the aggregation method function on the current result and next and sets to current result 21: Result = _aggregationMethod(Result, nextValue); 22: } 23: } Of course, LINQ already has an Aggregate extension method, but that works on a sequence of IEnumerable<T>, whereas this is designed to work more with aggregating single results over time (such as keeping track of a max response time for a service). We could then use this generic aggregator to find the sum of a series of values over time, or the max of a series of values over time (among other things): 1: // creates an aggregator that adds the next to the total to sum the values 2: var sumAggregator = new Aggregator<int, int>((total, next) => total + next); 3:  4: // creates an aggregator (using static method) that returns the max of previous result and next 5: var maxAggregator = new Aggregator<int, int>(Math.Max); So, if we were timing the response time of a web method every time it was called, we could pass that response time to both of these aggregators to get an idea of the total time spent in that web method, and the max time spent in any one call to the web method: 1: // total will be 13 and max 13 2: int responseTime = 13; 3: sumAggregator.Aggregate(responseTime); 4: maxAggregator.Aggregate(responseTime); 5:  6: // total will be 20 and max still 13 7: responseTime = 7; 8: sumAggregator.Aggregate(responseTime); 9: maxAggregator.Aggregate(responseTime); 10:  11: // total will be 40 and max now 20 12: responseTime = 20; 13: sumAggregator.Aggregate(responseTime); 14: maxAggregator.Aggregate(responseTime); The Func delegate family is useful for making generic algorithms and classes, and in particular allows the caller of the method or user of the class to specify a function to be performed in order to generate a result. What is the result of a Func delegate chain? If you remember, we said earlier that you can assign multiple methods to a delegate by using the += operator to chain them.  So how does this affect delegates such as Func that return a value, when applied to something like the code below? 1: Func<int, int, int> combo = null; 2:  3: // What if we wanted to aggregate the sum and max together? 4: combo += (total, next) => total + next; 5: combo += Math.Max; 6:  7: // what is the result? 8: var comboAggregator = new Aggregator<int, int>(combo); Well, in .NET if you chain multiple methods in a delegate, they will all get invoked, but the result of the delegate is the result of the last method invoked in the chain.  Thus, this aggregator would always result in the Math.Max() result.  The other chained method (the sum) gets executed first, but it’s result is thrown away: 1: // result is 13 2: int responseTime = 13; 3: comboAggregator.Aggregate(responseTime); 4:  5: // result is still 13 6: responseTime = 7; 7: comboAggregator.Aggregate(responseTime); 8:  9: // result is now 20 10: responseTime = 20; 11: comboAggregator.Aggregate(responseTime); So remember, you can chain multiple Func (or other delegates that return values) together, but if you do so you will only get the last executed result. Func delegates and co-variance/contra-variance in .NET 4.0 Just like the Action delegate, as of .NET 4.0, the Func delegate family is contra-variant on its arguments.  In addition, it is co-variant on its return type.  To support this, in .NET 4.0 the signatures of the Func delegates changed to: Func<out TResult> – matches a method that takes no arguments, and returns value of type TResult (or a more derived type). Func<in T, out TResult> – matches a method that takes an argument of type T (or a less derived type), and returns value of type TResult(or a more derived type). Func<in T1, in T2, out TResult> – matches a method that takes arguments of type T1 and T2 (or less derived types), and returns value of type TResult (or a more derived type). Func<in T1, in T2, …, out TResult> – and so on up to 16 arguments, and returns value of type TResult (or a more derived type). Notice the addition of the in and out keywords before each of the generic type placeholders.  As we saw last week, the in keyword is used to specify that a generic type can be contra-variant -- it can match the given type or a type that is less derived.  However, the out keyword, is used to specify that a generic type can be co-variant -- it can match the given type or a type that is more derived. On contra-variance, if you are saying you need an function that will accept a string, you can just as easily give it an function that accepts an object.  In other words, if you say “give me an function that will process dogs”, I could pass you a method that will process any animal, because all dogs are animals.  On the co-variance side, if you are saying you need a function that returns an object, you can just as easily pass it a function that returns a string because any string returned from the given method can be accepted by a delegate expecting an object result, since string is more derived.  Once again, in other words, if you say “give me a method that creates an animal”, I can pass you a method that will create a dog, because all dogs are animals. It really all makes sense, you can pass a more specific thing to a less specific parameter, and you can return a more specific thing as a less specific result.  In other words, pay attention to the direction the item travels (parameters go in, results come out).  Keeping that in mind, you can always pass more specific things in and return more specific things out. For example, in the code below, we have a method that takes a Func<object> to generate an object, but we can pass it a Func<string> because the return type of object can obviously accept a return value of string as well: 1: // since Func<object> is co-variant, this will access Func<string>, etc... 2: public static string Sequence(int count, Func<object> generator) 3: { 4: var builder = new StringBuilder(); 5:  6: for (int i=0; i<count; i++) 7: { 8: object value = generator(); 9: builder.Append(value); 10: } 11:  12: return builder.ToString(); 13: } Even though the method above takes a Func<object>, we can pass a Func<string> because the TResult type placeholder is co-variant and accepts types that are more derived as well: 1: // delegate that's typed to return string. 2: Func<string> stringGenerator = () => DateTime.Now.ToString(); 3:  4: // This will work in .NET 4.0, but not in previous versions 5: Sequence(100, stringGenerator); Previous versions of .NET implemented some forms of co-variance and contra-variance before, but .NET 4.0 goes one step further and allows you to pass or assign an Func<A, BResult> to a Func<Y, ZResult> as long as A is less derived (or same) as Y, and BResult is more derived (or same) as ZResult. Sidebar: The Func and the Predicate A method that takes one argument and returns a bool is generally thought of as a predicate.  Predicates are used to examine an item and determine whether that item satisfies a particular condition.  Predicates are typically unary, but you may also have binary and other predicates as well. Predicates are often used to filter results, such as in the LINQ Where() extension method: 1: var numbers = new[] { 1, 2, 4, 13, 8, 10, 27 }; 2:  3: // call Where() using a predicate which determines if the number is even 4: var evens = numbers.Where(num => num % 2 == 0); As of .NET 3.5, predicates are typically represented as Func<T, bool> where T is the type of the item to examine.  Previous to .NET 3.5, there was a Predicate<T> type that tended to be used (which we’ll discuss next week) and is still supported, but most developers recommend using Func<T, bool> now, as it prevents confusion with overloads that accept unary predicates and binary predicates, etc.: 1: // this seems more confusing as an overload set, because of Predicate vs Func 2: public static SomeMethod(Predicate<int> unaryPredicate) { } 3: public static SomeMethod(Func<int, int, bool> binaryPredicate) { } 4:  5: // this seems more consistent as an overload set, since just uses Func 6: public static SomeMethod(Func<int, bool> unaryPredicate) { } 7: public static SomeMethod(Func<int, int, bool> binaryPredicate) { } Also, even though Predicate<T> and Func<T, bool> match the same signatures, they are separate types!  Thus you cannot assign a Predicate<T> instance to a Func<T, bool> instance and vice versa: 1: // the same method, lambda expression, etc can be assigned to both 2: Predicate<int> isEven = i => (i % 2) == 0; 3: Func<int, bool> alsoIsEven = i => (i % 2) == 0; 4:  5: // but the delegate instances cannot be directly assigned, strongly typed! 6: // ERROR: cannot convert type... 7: isEven = alsoIsEven; 8:  9: // however, you can assign by wrapping in a new instance: 10: isEven = new Predicate<int>(alsoIsEven); 11: alsoIsEven = new Func<int, bool>(isEven); So, the general advice that seems to come from most developers is that Predicate<T> is still supported, but we should use Func<T, bool> for consistency in .NET 3.5 and above. Sidebar: Func as a Generator for Unit Testing One area of difficulty in unit testing can be unit testing code that is based on time of day.  We’d still want to unit test our code to make sure the logic is accurate, but we don’t want the results of our unit tests to be dependent on the time they are run. One way (of many) around this is to create an internal generator that will produce the “current” time of day.  This would default to returning result from DateTime.Now (or some other method), but we could inject specific times for our unit testing.  Generators are typically methods that return (generate) a value for use in a class/method. For example, say we are creating a CacheItem<T> class that represents an item in the cache, and we want to make sure the item shows as expired if the age is more than 30 seconds.  Such a class could look like: 1: // responsible for maintaining an item of type T in the cache 2: public sealed class CacheItem<T> 3: { 4: // helper method that returns the current time 5: private static Func<DateTime> _timeGenerator = () => DateTime.Now; 6:  7: // allows internal access to the time generator 8: internal static Func<DateTime> TimeGenerator 9: { 10: get { return _timeGenerator; } 11: set { _timeGenerator = value; } 12: } 13:  14: // time the item was cached 15: public DateTime CachedTime { get; private set; } 16:  17: // the item cached 18: public T Value { get; private set; } 19:  20: // item is expired if older than 30 seconds 21: public bool IsExpired 22: { 23: get { return _timeGenerator() - CachedTime > TimeSpan.FromSeconds(30.0); } 24: } 25:  26: // creates the new cached item, setting cached time to "current" time 27: public CacheItem(T value) 28: { 29: Value = value; 30: CachedTime = _timeGenerator(); 31: } 32: } Then, we can use this construct to unit test our CacheItem<T> without any time dependencies: 1: var baseTime = DateTime.Now; 2:  3: // start with current time stored above (so doesn't drift) 4: CacheItem<int>.TimeGenerator = () => baseTime; 5:  6: var target = new CacheItem<int>(13); 7:  8: // now add 15 seconds, should still be non-expired 9: CacheItem<int>.TimeGenerator = () => baseTime.AddSeconds(15); 10:  11: Assert.IsFalse(target.IsExpired); 12:  13: // now add 31 seconds, should now be expired 14: CacheItem<int>.TimeGenerator = () => baseTime.AddSeconds(31); 15:  16: Assert.IsTrue(target.IsExpired); Now we can unit test for 1 second before, 1 second after, 1 millisecond before, 1 day after, etc.  Func delegates can be a handy tool for this type of value generation to support more testable code.  Summary Generic delegates give us a lot of power to make truly generic algorithms and classes.  The Func family of delegates is a great way to be able to specify functions to calculate a result based on 0-16 arguments.  Stay tuned in the weeks that follow for other generic delegates in the .NET Framework!   Tweet Technorati Tags: .NET, C#, CSharp, Little Wonders, Generics, Func, Delegates

    Read the article

  • JVM process resident set size "equals" max heap size, not current heap size

    - by Volune
    After a few reading about jvm memory (here, here, here, others I forgot...), I am expecting the resident set size of my java process to be roughly equal to the current heap space capacity. That's not what the numbers are saying, it seems to be roughly equal to the max heap space capacity: Resident set size: # echo 0 $(cat /proc/1/smaps | grep Rss | awk '{print $2}' | sed 's#^#+#') | bc 11507912 # ps -C java -O rss | gawk '{ count ++; sum += $2 }; END {count --; print "Number of processes =",count; print "Memory usage per process =",sum/1024/count, "MB"; print "Total memory usage =", sum/1024, "MB" ;};' Number of processes = 1 Memory usage per process = 11237.8 MB Total memory usage = 11237.8 MB Java heap # jmap -heap 1 Attaching to process ID 1, please wait... Debugger attached successfully. Server compiler detected. JVM version is 24.55-b03 using thread-local object allocation. Garbage-First (G1) GC with 18 thread(s) Heap Configuration: MinHeapFreeRatio = 10 MaxHeapFreeRatio = 20 MaxHeapSize = 10737418240 (10240.0MB) NewSize = 1363144 (1.2999954223632812MB) MaxNewSize = 17592186044415 MB OldSize = 5452592 (5.1999969482421875MB) NewRatio = 2 SurvivorRatio = 8 PermSize = 20971520 (20.0MB) MaxPermSize = 85983232 (82.0MB) G1HeapRegionSize = 2097152 (2.0MB) Heap Usage: G1 Heap: regions = 2560 capacity = 5368709120 (5120.0MB) used = 1672045416 (1594.586769104004MB) free = 3696663704 (3525.413230895996MB) 31.144272834062576% used G1 Young Generation: Eden Space: regions = 627 capacity = 3279945728 (3128.0MB) used = 1314914304 (1254.0MB) free = 1965031424 (1874.0MB) 40.089514066496164% used Survivor Space: regions = 49 capacity = 102760448 (98.0MB) used = 102760448 (98.0MB) free = 0 (0.0MB) 100.0% used G1 Old Generation: regions = 147 capacity = 1986002944 (1894.0MB) used = 252273512 (240.5867691040039MB) free = 1733729432 (1653.413230895996MB) 12.702574926293766% used Perm Generation: capacity = 39845888 (38.0MB) used = 38884120 (37.082786560058594MB) free = 961768 (0.9172134399414062MB) 97.58628042120682% used 14654 interned Strings occupying 2188928 bytes. Are my expectations wrong? What should I expect? I need the heap space to be able to grow during spikes (to avoid very slow Full GC), but I would like to have the resident set size as low as possible the rest of the time, to benefit the other processes running on the server. Is there a better way to achieve that? Linux 3.13.0-32-generic x86_64 java version "1.7.0_55" Running in Docker version 1.1.2 Java is running elasticsearch 1.2.0: /usr/bin/java -Xms5g -Xmx10g -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -Xss256k -Djava.awt.headless=true -XX:+UseG1GC -XX:MaxGCPauseMillis=350 -XX:InitiatingHeapOccupancyPercent=45 -XX:+AggressiveOpts -XX:+UseCompressedOops -XX:-OmitStackTraceInFastThrow -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintClassHistogram -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime -Xloggc:/opt/elasticsearch/logs/gc.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/opt elasticsearch/logs/heapdump.hprof -XX:ErrorFile=/opt/elasticsearch/logs/hs_err.log -Des.logger.port=99999 -Des.logger.host=999.999.999.999 -Delasticsearch -Des.foreground=yes -Des.path.home=/opt/elasticsearch -cp :/opt/elasticsearch/lib/elasticsearch-1.2.0.jar:/opt/elasticsearch/lib/*:/opt/elasticsearch/lib/sigar/* org.elasticsearch.bootstrap.Elasticsearch There actually are 5 elasticsearch nodes, each in a different docker container. All have about the same memory usage. Some stats about the index: size: 9.71Gi (19.4Gi) docs: 3,925,398 (4,052,694)

    Read the article

  • Linux - Only first virtual interface can ping external gateway

    - by husvar
    I created 3 virtual interfaces with different mac addresses all linked to the same physical interface. I see that they successfully arp for the gw and they can ping (the request is coming in the packet capture in wireshark). However the ping utility does not count the responses. Does anyone knows the issue? I am running Ubuntu 14.04 in a VmWare. root@ubuntu:~# ip link sh 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether 00:0c:29:bc:fc:8b brd ff:ff:ff:ff:ff:ff root@ubuntu:~# ip addr sh 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:bc:fc:8b brd ff:ff:ff:ff:ff:ff inet6 fe80::20c:29ff:febc:fc8b/64 scope link valid_lft forever preferred_lft forever root@ubuntu:~# ip route sh root@ubuntu:~# ip link add link eth0 eth0.1 addr 00:00:00:00:00:11 type macvlan root@ubuntu:~# ip link add link eth0 eth0.2 addr 00:00:00:00:00:22 type macvlan root@ubuntu:~# ip link add link eth0 eth0.3 addr 00:00:00:00:00:33 type macvlan root@ubuntu:~# ip -4 link sh 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether 00:0c:29:bc:fc:8b brd ff:ff:ff:ff:ff:ff 18: eth0.1@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default link/ether 00:00:00:00:00:11 brd ff:ff:ff:ff:ff:ff 19: eth0.2@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default link/ether 00:00:00:00:00:22 brd ff:ff:ff:ff:ff:ff 20: eth0.3@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default link/ether 00:00:00:00:00:33 brd ff:ff:ff:ff:ff:ff root@ubuntu:~# ip -4 addr sh 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever root@ubuntu:~# ip -4 route sh root@ubuntu:~# dhclient -v eth0.1 Internet Systems Consortium DHCP Client 4.2.4 Copyright 2004-2012 Internet Systems Consortium. All rights reserved. For info, please visit https://www.isc.org/software/dhcp/ Listening on LPF/eth0.1/00:00:00:00:00:11 Sending on LPF/eth0.1/00:00:00:00:00:11 Sending on Socket/fallback DHCPDISCOVER on eth0.1 to 255.255.255.255 port 67 interval 3 (xid=0x568eac05) DHCPREQUEST of 192.168.1.145 on eth0.1 to 255.255.255.255 port 67 (xid=0x568eac05) DHCPOFFER of 192.168.1.145 from 192.168.1.254 DHCPACK of 192.168.1.145 from 192.168.1.254 bound to 192.168.1.145 -- renewal in 1473 seconds. root@ubuntu:~# dhclient -v eth0.2 Internet Systems Consortium DHCP Client 4.2.4 Copyright 2004-2012 Internet Systems Consortium. All rights reserved. For info, please visit https://www.isc.org/software/dhcp/ Listening on LPF/eth0.2/00:00:00:00:00:22 Sending on LPF/eth0.2/00:00:00:00:00:22 Sending on Socket/fallback DHCPDISCOVER on eth0.2 to 255.255.255.255 port 67 interval 3 (xid=0x21e3114e) DHCPREQUEST of 192.168.1.146 on eth0.2 to 255.255.255.255 port 67 (xid=0x21e3114e) DHCPOFFER of 192.168.1.146 from 192.168.1.254 DHCPACK of 192.168.1.146 from 192.168.1.254 bound to 192.168.1.146 -- renewal in 1366 seconds. root@ubuntu:~# dhclient -v eth0.3 Internet Systems Consortium DHCP Client 4.2.4 Copyright 2004-2012 Internet Systems Consortium. All rights reserved. For info, please visit https://www.isc.org/software/dhcp/ Listening on LPF/eth0.3/00:00:00:00:00:33 Sending on LPF/eth0.3/00:00:00:00:00:33 Sending on Socket/fallback DHCPDISCOVER on eth0.3 to 255.255.255.255 port 67 interval 3 (xid=0x11dc5f03) DHCPREQUEST of 192.168.1.147 on eth0.3 to 255.255.255.255 port 67 (xid=0x11dc5f03) DHCPOFFER of 192.168.1.147 from 192.168.1.254 DHCPACK of 192.168.1.147 from 192.168.1.254 bound to 192.168.1.147 -- renewal in 1657 seconds. root@ubuntu:~# ip -4 link sh 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether 00:0c:29:bc:fc:8b brd ff:ff:ff:ff:ff:ff 18: eth0.1@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default link/ether 00:00:00:00:00:11 brd ff:ff:ff:ff:ff:ff 19: eth0.2@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default link/ether 00:00:00:00:00:22 brd ff:ff:ff:ff:ff:ff 20: eth0.3@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default link/ether 00:00:00:00:00:33 brd ff:ff:ff:ff:ff:ff root@ubuntu:~# ip -4 addr sh 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever 18: eth0.1@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default inet 192.168.1.145/24 brd 192.168.1.255 scope global eth0.1 valid_lft forever preferred_lft forever 19: eth0.2@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default inet 192.168.1.146/24 brd 192.168.1.255 scope global eth0.2 valid_lft forever preferred_lft forever 20: eth0.3@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default inet 192.168.1.147/24 brd 192.168.1.255 scope global eth0.3 valid_lft forever preferred_lft forever root@ubuntu:~# ip -4 route sh default via 192.168.1.254 dev eth0.1 192.168.1.0/24 dev eth0.1 proto kernel scope link src 192.168.1.145 192.168.1.0/24 dev eth0.2 proto kernel scope link src 192.168.1.146 192.168.1.0/24 dev eth0.3 proto kernel scope link src 192.168.1.147 root@ubuntu:~# arping -c 5 -I eth0.1 192.168.1.254 ARPING 192.168.1.254 from 192.168.1.145 eth0.1 Unicast reply from 192.168.1.254 [58:98:35:57:a0:70] 6.936ms Unicast reply from 192.168.1.254 [58:98:35:57:a0:70] 2.986ms Unicast reply from 192.168.1.254 [58:98:35:57:a0:70] 0.654ms Unicast reply from 192.168.1.254 [58:98:35:57:a0:70] 5.137ms Unicast reply from 192.168.1.254 [58:98:35:57:a0:70] 2.426ms Sent 5 probes (1 broadcast(s)) Received 5 response(s) root@ubuntu:~# arping -c 5 -I eth0.2 192.168.1.254 ARPING 192.168.1.254 from 192.168.1.146 eth0.2 Unicast reply from 192.168.1.254 [58:98:35:57:a0:70] 5.665ms Unicast reply from 192.168.1.254 [58:98:35:57:a0:70] 3.753ms Unicast reply from 192.168.1.254 [58:98:35:57:a0:70] 16.500ms Unicast reply from 192.168.1.254 [58:98:35:57:a0:70] 3.287ms Unicast reply from 192.168.1.254 [58:98:35:57:a0:70] 32.438ms Sent 5 probes (1 broadcast(s)) Received 5 response(s) root@ubuntu:~# arping -c 5 -I eth0.3 192.168.1.254 ARPING 192.168.1.254 from 192.168.1.147 eth0.3 Unicast reply from 192.168.1.254 [58:98:35:57:a0:70] 4.422ms Unicast reply from 192.168.1.254 [58:98:35:57:a0:70] 2.429ms Unicast reply from 192.168.1.254 [58:98:35:57:a0:70] 2.321ms Unicast reply from 192.168.1.254 [58:98:35:57:a0:70] 40.423ms Unicast reply from 192.168.1.254 [58:98:35:57:a0:70] 2.268ms Sent 5 probes (1 broadcast(s)) Received 5 response(s) root@ubuntu:~# tcpdump -n -i eth0.1 -v & [1] 5317 root@ubuntu:~# ping -c5 -q -I eth0.1 192.168.1.254 PING 192.168.1.254 (192.168.1.254) from 192.168.1.145 eth0.1: 56(84) bytes of data. tcpdump: listening on eth0.1, link-type EN10MB (Ethernet), capture size 65535 bytes 13:18:37.612558 IP (tos 0x0, ttl 64, id 2595, offset 0, flags [DF], proto ICMP (1), length 84) 192.168.1.145 > 192.168.1.254: ICMP echo request, id 5318, seq 2, length 64 13:18:37.618864 IP (tos 0x68, ttl 64, id 14493, offset 0, flags [none], proto ICMP (1), length 84) 192.168.1.254 > 192.168.1.145: ICMP echo reply, id 5318, seq 2, length 64 13:18:37.743650 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.1.87 tell 192.168.1.86, length 46 13:18:38.134997 IP (tos 0x0, ttl 128, id 23547, offset 0, flags [none], proto UDP (17), length 229) 192.168.1.86.138 > 192.168.1.255.138: NBT UDP PACKET(138) 13:18:38.614580 IP (tos 0x0, ttl 64, id 2596, offset 0, flags [DF], proto ICMP (1), length 84) 192.168.1.145 > 192.168.1.254: ICMP echo request, id 5318, seq 3, length 64 13:18:38.793479 IP (tos 0x68, ttl 64, id 14495, offset 0, flags [none], proto ICMP (1), length 84) 192.168.1.254 > 192.168.1.145: ICMP echo reply, id 5318, seq 3, length 64 13:18:39.151282 IP6 (class 0x68, hlim 255, next-header ICMPv6 (58) payload length: 32) fe80::5a98:35ff:fe57:e070 > ff02::1:ff6b:e9b4: [icmp6 sum ok] ICMP6, neighbor solicitation, length 32, who has 2001:818:d812:da00:8ae3:abff:fe6b:e9b4 source link-address option (1), length 8 (1): 58:98:35:57:a0:70 13:18:39.615612 IP (tos 0x0, ttl 64, id 2597, offset 0, flags [DF], proto ICMP (1), length 84) 192.168.1.145 > 192.168.1.254: ICMP echo request, id 5318, seq 4, length 64 13:18:39.746981 IP (tos 0x68, ttl 64, id 14496, offset 0, flags [none], proto ICMP (1), length 84) 192.168.1.254 > 192.168.1.145: ICMP echo reply, id 5318, seq 4, length 64 --- 192.168.1.254 ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 4008ms rtt min/avg/max/mdev = 2.793/67.810/178.934/73.108 ms root@ubuntu:~# killall tcpdump >> /dev/null 2>&1 9 packets captured 12 packets received by filter 0 packets dropped by kernel [1]+ Done tcpdump -n -i eth0.1 -v root@ubuntu:~# tcpdump -n -i eth0.2 -v & [1] 5320 root@ubuntu:~# ping -c5 -q -I eth0.2 192.168.1.254 PING 192.168.1.254 (192.168.1.254) from 192.168.1.146 eth0.2: 56(84) bytes of data. tcpdump: listening on eth0.2, link-type EN10MB (Ethernet), capture size 65535 bytes 13:18:41.536874 ARP, Ethernet (len 6), IPv4 (len 4), Reply 192.168.1.254 is-at 58:98:35:57:a0:70, length 46 13:18:41.536933 IP (tos 0x0, ttl 64, id 2599, offset 0, flags [DF], proto ICMP (1), length 84) 192.168.1.146 > 192.168.1.254: ICMP echo request, id 5321, seq 1, length 64 13:18:41.539255 IP (tos 0x68, ttl 64, id 14507, offset 0, flags [none], proto ICMP (1), length 84) 192.168.1.254 > 192.168.1.146: ICMP echo reply, id 5321, seq 1, length 64 13:18:42.127715 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.1.87 tell 192.168.1.86, length 46 13:18:42.511725 IP (tos 0x0, ttl 64, id 2600, offset 0, flags [DF], proto ICMP (1), length 84) 192.168.1.146 > 192.168.1.254: ICMP echo request, id 5321, seq 2, length 64 13:18:42.514385 IP (tos 0x68, ttl 64, id 14527, offset 0, flags [none], proto ICMP (1), length 84) 192.168.1.254 > 192.168.1.146: ICMP echo reply, id 5321, seq 2, length 64 13:18:42.743856 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.1.87 tell 192.168.1.86, length 46 13:18:43.511727 IP (tos 0x0, ttl 64, id 2601, offset 0, flags [DF], proto ICMP (1), length 84) 192.168.1.146 > 192.168.1.254: ICMP echo request, id 5321, seq 3, length 64 13:18:43.513768 IP (tos 0x68, ttl 64, id 14528, offset 0, flags [none], proto ICMP (1), length 84) 192.168.1.254 > 192.168.1.146: ICMP echo reply, id 5321, seq 3, length 64 13:18:43.637598 IP (tos 0x0, ttl 128, id 23551, offset 0, flags [none], proto UDP (17), length 225) 192.168.1.86.17500 > 255.255.255.255.17500: UDP, length 197 13:18:43.641185 IP (tos 0x0, ttl 128, id 23552, offset 0, flags [none], proto UDP (17), length 225) 192.168.1.86.17500 > 192.168.1.255.17500: UDP, length 197 13:18:43.641201 IP (tos 0x0, ttl 128, id 23553, offset 0, flags [none], proto UDP (17), length 225) 192.168.1.86.17500 > 255.255.255.255.17500: UDP, length 197 13:18:43.743890 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.1.87 tell 192.168.1.86, length 46 13:18:44.510758 IP (tos 0x0, ttl 64, id 2602, offset 0, flags [DF], proto ICMP (1), length 84) 192.168.1.146 > 192.168.1.254: ICMP echo request, id 5321, seq 4, length 64 13:18:44.512892 IP (tos 0x68, ttl 64, id 14538, offset 0, flags [none], proto ICMP (1), length 84) 192.168.1.254 > 192.168.1.146: ICMP echo reply, id 5321, seq 4, length 64 13:18:45.510794 IP (tos 0x0, ttl 64, id 2603, offset 0, flags [DF], proto ICMP (1), length 84) 192.168.1.146 > 192.168.1.254: ICMP echo request, id 5321, seq 5, length 64 13:18:45.519701 IP (tos 0x68, ttl 64, id 14539, offset 0, flags [none], proto ICMP (1), length 84) 192.168.1.254 > 192.168.1.146: ICMP echo reply, id 5321, seq 5, length 64 13:18:49.287554 IP6 (class 0x68, hlim 255, next-header ICMPv6 (58) payload length: 32) fe80::5a98:35ff:fe57:e070 > ff02::1:ff6b:e9b4: [icmp6 sum ok] ICMP6, neighbor solicitation, length 32, who has 2001:818:d812:da00:8ae3:abff:fe6b:e9b4 source link-address option (1), length 8 (1): 58:98:35:57:a0:70 13:18:50.013463 IP (tos 0x0, ttl 255, id 50737, offset 0, flags [DF], proto UDP (17), length 73) 192.168.1.146.5353 > 224.0.0.251.5353: 0 [2q] PTR (QM)? _ipps._tcp.local. PTR (QM)? _ipp._tcp.local. (45) 13:18:50.218874 IP6 (class 0x68, hlim 255, next-header ICMPv6 (58) payload length: 32) fe80::5a98:35ff:fe57:e070 > ff02::1:ff6b:e9b4: [icmp6 sum ok] ICMP6, neighbor solicitation, length 32, who has 2001:818:d812:da00:8ae3:abff:fe6b:e9b4 source link-address option (1), length 8 (1): 58:98:35:57:a0:70 13:18:51.129961 IP6 (class 0x68, hlim 255, next-header ICMPv6 (58) payload length: 32) fe80::5a98:35ff:fe57:e070 > ff02::1:ff6b:e9b4: [icmp6 sum ok] ICMP6, neighbor solicitation, length 32, who has 2001:818:d812:da00:8ae3:abff:fe6b:e9b4 source link-address option (1), length 8 (1): 58:98:35:57:a0:70 13:18:52.197074 IP6 (hlim 255, next-header UDP (17) payload length: 53) 2001:818:d812:da00:200:ff:fe00:22.5353 > ff02::fb.5353: [udp sum ok] 0 [2q] PTR (QM)? _ipps._tcp.local. PTR (QM)? _ipp._tcp.local. (45) 13:18:54.128240 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.1.87 tell 192.168.1.86, length 46 --- 192.168.1.254 ping statistics --- 5 packets transmitted, 0 received, 100% packet loss, time 4000ms root@ubuntu:~# killall tcpdump >> /dev/null 2>&1 13:18:54.657731 IP6 (class 0x68, hlim 255, next-header ICMPv6 (58) payload length: 32) fe80::5a98:35ff:fe57:e070 > ff02::1:ff6b:e9b4: [icmp6 sum ok] ICMP6, neighbor solicitation, length 32, who has 2001:818:d812:da00:8ae3:abff:fe6b:e9b4 source link-address option (1), length 8 (1): 58:98:35:57:a0:70 13:18:54.743174 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.1.87 tell 192.168.1.86, length 46 25 packets captured 26 packets received by filter 0 packets dropped by kernel [1]+ Done tcpdump -n -i eth0.2 -v root@ubuntu:~# tcpdump -n -i eth0.3 icmp & [1] 5324 root@ubuntu:~# ping -c5 -q -I eth0.3 192.168.1.254 PING 192.168.1.254 (192.168.1.254) from 192.168.1.147 eth0.3: 56(84) bytes of data. tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0.3, link-type EN10MB (Ethernet), capture size 65535 bytes 13:18:56.373434 IP 192.168.1.147 > 192.168.1.254: ICMP echo request, id 5325, seq 1, length 64 13:18:57.372116 IP 192.168.1.147 > 192.168.1.254: ICMP echo request, id 5325, seq 2, length 64 13:18:57.381263 IP 192.168.1.254 > 192.168.1.147: ICMP echo reply, id 5325, seq 2, length 64 13:18:58.371141 IP 192.168.1.147 > 192.168.1.254: ICMP echo request, id 5325, seq 3, length 64 13:18:58.373275 IP 192.168.1.254 > 192.168.1.147: ICMP echo reply, id 5325, seq 3, length 64 13:18:59.371165 IP 192.168.1.147 > 192.168.1.254: ICMP echo request, id 5325, seq 4, length 64 13:18:59.373259 IP 192.168.1.254 > 192.168.1.147: ICMP echo reply, id 5325, seq 4, length 64 13:19:00.371211 IP 192.168.1.147 > 192.168.1.254: ICMP echo request, id 5325, seq 5, length 64 13:19:00.373278 IP 192.168.1.254 > 192.168.1.147: ICMP echo reply, id 5325, seq 5, length 64 --- 192.168.1.254 ping statistics --- 5 packets transmitted, 1 received, 80% packet loss, time 4001ms rtt min/avg/max/mdev = 13.666/13.666/13.666/0.000 ms root@ubuntu:~# killall tcpdump >> /dev/null 2>&1 9 packets captured 10 packets received by filter 0 packets dropped by kernel [1]+ Done tcpdump -n -i eth0.3 icmp root@ubuntu:~# arp -n Address HWtype HWaddress Flags Mask Iface 192.168.1.254 ether 58:98:35:57:a0:70 C eth0.1 192.168.1.254 ether 58:98:35:57:a0:70 C eth0.2 192.168.1.254 ether 58:98:35:57:a0:70 C eth0.3

    Read the article

  • How to make an excel formula which totals several agecent rows based on cell values

    - by Yishai
    I have an excel sheet with three columns: date, person and percentage. I would like to put in a data validation that flags cells if the total for a given data/person combination do not equal 100%. Is that possible? In other words, in the custom formula of a data validation, I would like to make the following type of formula. =if(sum( cells with a (date = the date on this row, person = person on this row))=1) Is there a function which will return the cells in a range conditioned on certain values, or will sum the cells. Note that if it is not possible to do two cells, I have no issue adding a cell which combines both values for the purpose of effecting the lookup.

    Read the article

  • Munin Aggregated Graphs Configuration Error

    - by Sparsh Gupta
    I tried making some Munin Aggregated graphs but somehow I am unable to make the configuration work. I think I have followed the instructions but since its not working, I would love some assistance or guidance as to what I am doing wrong. I want to Aggregate (sum) the total number of requests / second all my nginx servers are doing combined together. The configuration looks like [TRAFFIC.AGGREGATED] update no requests.graph_title nGinx requests requests.graph_vlabel nGinx requests per second requests.draw LINE2 requests.graph_args --base 1000 requests.graph_category nginx requests.label req/sec requests.type DERIVE requests.min 0 requests.graph_order output requests.output.sum \ lb1.visualwebsiteoptimizer.com:nginx_request_lb1.visualwebsiteoptimizer.com_request.request \ lb3.visualwebsiteoptimizer.com:nginx_request_lb2.visualwebsiteoptimizer.com_request.request \ lb3.visualwebsiteoptimizer.com:nginx_request_lb3.visualwebsiteoptimizer.com_request.request The munin graph I want to aggregate is http://exchange.munin-monitoring.org/plugins/nginx_request/details Thanks Sparsh Gupta

    Read the article

  • Finding trends in multi-category data in Excel

    - by Miral
    I have an Excel spreadsheet that contains hundreds of rows of data that each represent a single sample in a larger population. Each row is divided into three columns that contain frequency counts of a specific type of thing. Together the three columns summed on a single row represent 100%, though each row will sum to a different value. What I'm most interested in are the proportions of each of these types (ie. percentages of each column relative to the sum of the three columns). I can easily calculate this on a per-row basis, but what I'm really interested in is trying to find an overall trend from the entire population. I don't really spend much time doing data analysis so the only thing I can think of trying is to create those percentage columns and then average them, but I'm sure there must be a better way to visualise this.

    Read the article

  • Showing name of row instead of excel cell name

    - by Kare
    I am having extremely long formulas over an extremely big sheet. At the moment I am tracking the formulas with the Formula Auditing Tool. However, my idea would be to just replace for example in a formula like this: =IF(AND(ROUND($GX19-SUM(0)/$M$12;2)<=0;$AK$7=1);0;$M$12*$M$22/$K$62 My idea would be to replace the excel cell names with the table row names they are in. Like: =IF(AND(ROUND( "Income" -SUM(0)/ "Debt" ;2)<=0; "Percentage" =1);0; "Investment" * "Debt of house" / "Investment costs" Is there any way to achive sth. like that in excel? I appreciate your inputs!!!

    Read the article

  • In Excel, how to group data by date, and then do operations on the data?

    - by Bicou
    Hi, I have Excel 2003. My data is like this: 01/10/2010 0.99 02/10/2010 1.49 02/10/2010 0.99 02/10/2010 0.99 02/10/2010 0.99 03/10/2010 1.49 03/10/2010 1.49 03/10/2010 0.99 etc. In fact it is a list of sales every day. I want to have something like this: 01/10/2010 0.99 02/10/2010 4.46 03/10/2010 3.97 I want to group by date, and sum the column B. I'd like to see the evolution of the sales over time, and display a nice graph about that. I have managed to create pivot tables that almost do the job: they list the number of 0.99 and 1.49 each day, but I can't find a way to simply sum everything and group by date. Thanks for reading.

    Read the article

  • Automate hashing for each file in a folder?

    - by Kennie R.
    I have quite a few FTP folders, and I add a few each month and prefer to leave some sort of method of verifying their integrity, for example the files MD5SUMS, SHA256SUMS, ... which I could create using a script. Take for example: find ./ -type f -exec md5sum $1 {} \; This works fine, but when I run it each time for each shaxxx sum afterwards, it creates a sum of the MD5SUMs file which is really not wanted. Is there a simpler way, or script, or common way of hashing all the files in to their sums file without causing problems like that? I could really use a better option.

    Read the article

  • Point in polygon OR point on polygon using LINQ

    - by wageoghe
    As noted in an earlier question, How to Zip enumerable with itself, I am working on some math algorithms based on lists of points. I am currently working on point in polygon. I have the code for how to do that and have found several good references here on SO, such as this link Hit test. So, I can figure out whether or not a point is in a polygon. As part of determining that, I want to determine if the point is actually on the polygon. This I can also do. If I can do all of that, what is my question you might ask? Can I do it efficiently using LINQ? I can already do something like the following (assuming a Pairwise extension method as described in my earlier question as well as in links to which my question/answers links, and assuming a Position type that has X and Y members). I have not tested much, so the lambda might not be 100% correct. Also, it does not take very small differences into account. public static PointInPolygonLocation PointInPolygon(IEnumerable<Position> pts, Position pt) { int numIntersections = pts.Pairwise( (p1, p2) => { if (p1.Y != p2.Y) { if ((p1.Y >= pt.Y && p2.Y < pt.Y) || (p1.Y < pt.Y && p2.Y >= pt.Y)) { if (p1.X < p1.X && p2.X < pt.X) { return 1; } if (p1.X < pt.X || p2.X < pt.X) { if (((pt.Y - p1.Y) * ((p1.X - p2.X) / (p1.Y - p2.Y)) * p1.X) < pt.X) { return 1; } } } } return 0; }).Sum(); if (numIntersections % 2 == 0) { return PointInPolygonLocation.Outside; } else { return PointInPolygonLocation.Inside; } } This function, PointInPolygon, takes the input Position, pt, iterates over the input sequence of position values, and uses the Jordan Curve method to determine how many times a ray extended from pt to the left intersects the polygon. The lambda expression will yield, into the "zipped" list, 1 for every segment that is crossed, and 0 for the rest. The sum of these values determines if pt is inside or outside of the polygon (odd == inside, even == outside). So far, so good. Now, for any consecutive pairs of position values in the sequence (i.e. in any execution of the lambda), we can also determine if pt is ON the segment p1, p2. If that is the case, we can stop the calculation because we have our answer. Ultimately, my question is this: Can I perform this calculation (maybe using Aggregate?) such that we will only iterate over the sequence no more than 1 time AND can we stop the iteration if we encounter a segment that pt is ON? In other words, if pt is ON the very first segment, there is no need to examine the rest of the segments because we have the answer. It might very well be that this operation (particularly the requirement/desire to possibly stop the iteration early) does not really lend itself well to the LINQ approach. It just occurred to me that maybe the lambda expression could yield a tuple, the intersection value (1 or 0 or maybe true or false) and the "on" value (true or false). Maybe then I could use TakeWhile(anontype.PointOnPolygon == false). If I Sum the tuples and if ON == 1, then the point is ON the polygon. Otherwise, the oddness or evenness of the sum of the other part of the tuple tells if the point is inside or outside.

    Read the article

  • How to Zip one IEnumerable with itself

    - by wageoghe
    I am implementing some math algorithms based on lists of points, like Distance, Area, Centroid, etc. Just like in this post: http://stackoverflow.com/questions/2227828/find-the-distance-required-to-navigate-a-list-of-points-using-linq That post describes how to calculate the total distance of a sequence of points (taken in order) by essentially zipping the sequence "with itself", generating the sequence for Zip by offsetting the start position of the original IEnumerable by 1. So, given the Zip extension in .Net 4.0, assuming Point for the point type, and a reasonable Distance formula, you can make calls like this to generate a sequence of distances from one point to the next and then to sum the distances: var distances = points.Zip(points.Skip(1),Distance); double totalDistance = distances.Sum(); Area and Centroid calculations are similar in that they need to iterate over the sequence, processing each pair of points (points[i] and points[i+1]). I thought of making a generic IEnumerable extension suitable for implementing these (and possibly other) algorithms that operate over sequences, taking two items at a time (points[0] and points[1], points[1] and points[2], ..., points[n-1] and points[n] (or is it n-2 and n-1 ...) and applying a function. My generic iterator would have a similar signature to Zip, but it would not receive a second sequence to zip with as it is really just going to zip with itself. My first try looks like this: public static IEnumerable<TResult> ZipMyself<TSequence, TResult>(this IEnumerable<TSequence> seq, Func<TSequence, TSequence, TResult> resultSelector) { return seq.Zip(seq.Skip(1),resultSelector); } With my generic iterator in place, I can write functions like this: public static double Length(this IEnumerable<Point> points) { return points.ZipMyself(Distance).Sum(); } and call it like this: double d = points.Length(); and double GreensTheorem(Point p1, Point p1) { return p1.X * p2.Y - p1.Y * p2.X; } public static double SignedArea(this IEnumerable<Point> points) { return points.ZipMyself(GreensTheorem).Sum() / 2.0 } public static double Area(this IEnumerable<Point> points) { return Math.Abs(points.SignedArea()); } public static bool IsClockwise(this IEnumerable<Point> points) { return SignedArea(points) < 0; } and call them like this: double a = points.Area(); bool isClockwise = points.IsClockwise(); In this case, is there any reason NOT to implement "ZipMyself" in terms of Zip and Skip(1)? Is there already something in LINQ that automates this (zipping a list with itself) - not that it needs to be made that much easier ;-) Also, is there better name for the extension that might reflect that it is a well-known pattern (if, indeed it is a well-known pattern)? Had a link here for a StackOverflow question about area calculation. It is question 2432428. Also had a link to Wikipedia article on Centroid. Just go to Wikipedia and search for Centroid if interested. Just starting out, so don't have enough rep to post more than one link,

    Read the article

  • Matlab: Optimization by perturbing variable

    - by S_H
    My main script contains following code: %# Grid and model parameters nModel=50; nModel_want=1; nI_grid1=5; Nth=1; nRow.Scale1=5; nCol.Scale1=5; nRow.Scale2=5^2; nCol.Scale2=5^2; theta = 90; % degrees a_minor = 2; % range along minor direction a_major = 5; % range along major direction sill = var(reshape(Deff_matrix_NthModel,nCell.Scale1,1)); % variance of the coarse data matrix of size nRow.Scale1 X nCol.Scale1 %# Covariance computation % Scale 1 for ihRow = 1:nRow.Scale1 for ihCol = 1:nCol.Scale1 [cov.Scale1(ihRow,ihCol),heff.Scale1(ihRow,ihCol)] = general_CovModel(theta, ihCol, ihRow, a_minor, a_major, sill, 'Exp'); end end % Scale 2 for ihRow = 1:nRow.Scale2 for ihCol = 1:nCol.Scale2 [cov.Scale2(ihRow,ihCol),heff.Scale2(ihRow,ihCol)] = general_CovModel(theta, ihCol/(nCol.Scale2/nCol.Scale1), ihRow/(nRow.Scale2/nRow.Scale1), a_minor, a_major, sill/(nRow.Scale2*nCol.Scale2), 'Exp'); end end %# Scale-up of fine scale values by averaging [covAvg.Scale2,var_covAvg.Scale2,varNorm_covAvg.Scale2] = general_AverageProperty(nRow.Scale2/nRow.Scale1,nCol.Scale2/nCol.Scale1,1,nRow.Scale1,nCol.Scale1,1,cov.Scale2,1); I am using two functions, general_CovModel() and general_AverageProperty(), in my main script which are given as following: function [cov,h_eff] = general_CovModel(theta, hx, hy, a_minor, a_major, sill, mod_type) % mod_type should be in strings angle_rad = theta*(pi/180); % theta in degrees, angle_rad in radians R_theta = [sin(angle_rad) cos(angle_rad); -cos(angle_rad) sin(angle_rad)]; h = [hx; hy]; lambda = a_minor/a_major; D_lambda = [lambda 0; 0 1]; h_2prime = D_lambda*R_theta*h; h_eff = sqrt((h_2prime(1)^2)+(h_2prime(2)^2)); if strcmp(mod_type,'Sph')==1 || strcmp(mod_type,'sph') ==1 if h_eff<=a cov = sill - sill.*(1.5*(h_eff/a_minor)-0.5*((h_eff/a_minor)^3)); else cov = sill; end elseif strcmp(mod_type,'Exp')==1 || strcmp(mod_type,'exp') ==1 cov = sill-(sill.*(1-exp(-(3*h_eff)/a_minor))); elseif strcmp(mod_type,'Gauss')==1 || strcmp(mod_type,'gauss') ==1 cov = sill-(sill.*(1-exp(-((3*h_eff)^2/(a_minor^2))))); end and function [PropertyAvg,variance_PropertyAvg,NormVariance_PropertyAvg]=... general_AverageProperty(blocksize_row,blocksize_col,blocksize_t,... nUpscaledRow,nUpscaledCol,nUpscaledT,PropertyArray,omega) % This function computes average of a property and variance of that averaged % property using power averaging PropertyAvg=zeros(nUpscaledRow,nUpscaledCol,nUpscaledT); %# Average of property for k=1:nUpscaledT, for j=1:nUpscaledCol, for i=1:nUpscaledRow, sum=0; for a=1:blocksize_row, for b=1:blocksize_col, for c=1:blocksize_t, sum=sum+(PropertyArray((i-1)*blocksize_row+a,(j-1)*blocksize_col+b,(k-1)*blocksize_t+c).^omega); % add all the property values in 'blocksize_x','blocksize_y','blocksize_t' to one variable end end end PropertyAvg(i,j,k)=(sum/(blocksize_row*blocksize_col*blocksize_t)).^(1/omega); % take average of the summed property end end end %# Variance of averageed property variance_PropertyAvg=var(reshape(PropertyAvg,... nUpscaledRow*nUpscaledCol*nUpscaledT,1),1,1); %# Normalized variance of averageed property NormVariance_PropertyAvg=variance_PropertyAvg./(var(reshape(... PropertyArray,numel(PropertyArray),1),1,1)); Question: Using Matlab, I would like to optimize covAvg.Scale2 such that it matches closely with cov.Scale1 by perturbing/varying any (or all) of the following variables 1) a_minor 2) a_major 3) theta Thanks.

    Read the article

  • Implementing a popularity algorithm in Django

    - by TheLizardKing
    I am creating a site similar to reddit and hacker news that has a database of links and votes. I am implementing hacker news' popularity algorithm and things are going pretty swimmingly until it comes to actually gathering up these links and displaying them. The algorithm is simple: Y Combinator's Hacker News: Popularity = (p - 1) / (t + 2)^1.5` Votes divided by age factor. Where` p : votes (points) from users. t : time since submission in hours. p is subtracted by 1 to negate submitter's vote. Age factor is (time since submission in hours plus two) to the power of 1.5.factor is (time since submission in hours plus two) to the power of 1.5. I asked a very similar question over yonder http://stackoverflow.com/questions/1964395/complex-ordering-in-django but instead of contemplating my options I choose one and tried to make it work because that's how I did it with PHP/MySQL but I now know Django does things a lot differently. My models look something (exactly) like this class Link(models.Model): category = models.ForeignKey(Category) user = models.ForeignKey(User) created = models.DateTimeField(auto_now_add = True) modified = models.DateTimeField(auto_now = True) fame = models.PositiveIntegerField(default = 1) title = models.CharField(max_length = 256) url = models.URLField(max_length = 2048) def __unicode__(self): return self.title class Vote(models.Model): link = models.ForeignKey(Link) user = models.ForeignKey(User) created = models.DateTimeField(auto_now_add = True) modified = models.DateTimeField(auto_now = True) karma_delta = models.SmallIntegerField() def __unicode__(self): return str(self.karma_delta) and my view: def index(request): popular_links = Link.objects.select_related().annotate(karma_total = Sum('vote__karma_delta')) return render_to_response('links/index.html', {'links': popular_links}) Now from my previous question, I am trying to implement the algorithm using the sorting function. An answer from that question seems to think I should put the algorithm in the select and sort then. I am going to paginate these results so I don't think I can do the sorting in python without grabbing everything. Any suggestions on how I could efficiently do this? EDIT This isn't working yet but I think it's a step in the right direction: from django.shortcuts import render_to_response from linkett.apps.links.models import * def index(request): popular_links = Link.objects.select_related() popular_links = popular_links.extra( select = { 'karma_total': 'SUM(vote.karma_delta)', 'popularity': '(karma_total - 1) / POW(2, 1.5)', }, order_by = ['-popularity'] ) return render_to_response('links/index.html', {'links': popular_links}) This errors out into: Caught an exception while rendering: column "karma_total" does not exist LINE 1: SELECT ((karma_total - 1) / POW(2, 1.5)) AS "popularity", (S... EDIT 2 Better error? TemplateSyntaxError: Caught an exception while rendering: missing FROM-clause entry for table "vote" LINE 1: SELECT ((vote.karma_total - 1) / POW(2, 1.5)) AS "popularity... My index.html is simply: {% block content %} {% for link in links %} karma-up {{ link.karma_total }} karma-down {{ link.title }} Posted by {{ link.user }} to {{ link.category }} at {{ link.created }} {% empty %} No Links {% endfor %} {% endblock content %} EDIT 3 So very close! Again, all these answers are great but I am concentrating on a particular one because I feel it works best for my situation. from django.db.models import Sum from django.shortcuts import render_to_response from linkett.apps.links.models import * def index(request): popular_links = Link.objects.select_related().extra( select = { 'popularity': '(SUM(links_vote.karma_delta) - 1) / POW(2, 1.5)', }, tables = ['links_link', 'links_vote'], order_by = ['-popularity'], ) return render_to_response('links/test.html', {'links': popular_links}) Running this I am presented with an error hating on my lack of group by values. Specifically: TemplateSyntaxError at / Caught an exception while rendering: column "links_link.id" must appear in the GROUP BY clause or be used in an aggregate function LINE 1: ...karma_delta) - 1) / POW(2, 1.5)) AS "popularity", "links_lin... Not sure why my links_link.id wouldn't be in my group by but I am not sure how to alter my group by, django usually does that.

    Read the article

  • iPhone: Repeating Rows in Each Section of Grouped UITableview

    - by Rank Beginner
    I'm trying to learn how to use the UITableView in conjunction with a SQLite back end. My issue is that I've gotten the table to populate with the records from the database, however I'm having a problem with the section titles. I am not able to figure out the proper set up for this, and I'm repeating all tasks under each section. The table looks like this. The groups field is where I'm trying to pull the section title from. TaskID groups TaskName sched lastCompleted nextCompleted success 1 Household laundry 3 03/19/2010 03/22/2010 y 1 Automotive Change oil 3 03/20/2010 03/23/2010 y In my viewDidLoad Method, I create an array from each column in the table like below. //Create and initialize arrays from table columns //______________________________________________________________________________________ ids =[[NSMutableArray alloc] init]; tasks =[[NSMutableArray alloc] init]; sched =[[NSMutableArray alloc] init]; lastComplete =[[NSMutableArray alloc] init]; nextComplete =[[NSMutableArray alloc] init]; weight =[[NSMutableArray alloc] init]; success =[[NSMutableArray alloc] init]; group =[[NSMutableArray alloc] init]; // Bind them to the data //______________________________________________________________________________________ NSString *query = [NSString stringWithFormat:@"SELECT * FROM Tasks ORDER BY nextComplete "]; sqlite3_stmt *statement; if (sqlite3_prepare_v2( database, [query UTF8String], -1, &statement, nil) == SQLITE_OK) { while (sqlite3_step(statement) == SQLITE_ROW) { [ids addObject:[NSString stringWithFormat:@"%i",(int*) sqlite3_column_int(statement, 0)]]; [group addObject:[NSString stringWithFormat:@"%s",(char*) sqlite3_column_text(statement, 1)]]; [tasks addObject:[NSString stringWithFormat:@"%s",(char*) sqlite3_column_text(statement, 2)]]; [sched addObject:[NSString stringWithFormat:@"%i",(int*) sqlite3_column_int(statement, 3)]]; [lastComplete addObject:[NSString stringWithFormat:@"%s",(char*) sqlite3_column_text(statement, 4)]]; [nextComplete addObject:[NSString stringWithFormat:@"%s",(char*) sqlite3_column_text(statement, 5)]]; [success addObject:[NSString stringWithFormat:@"%s",(char*) sqlite3_column_text(statement, 6)]]; [weight addObject:[NSString stringWithFormat:@"%i",(int*) sqlite3_column_int(statement, 7)]]; } sqlite3_finalize(statement); } In the table method:cellForRowAtIndexPath, I create controls on the fly and set their text properties to objects in the array. Below is a sample, I can provide more but am already working on a book here... :) /create the task label NSString *tmpMessage; tmpMessage = [NSString stringWithFormat:@"%@ every %@ days, for %@ points",[tasks objectAtIndex:indexPath.row],[sched objectAtIndex:indexPath.row],[weight objectAtIndex:indexPath.row]]; CGRect schedLabelRect = CGRectMake(0, 0, 250, 15); UILabel *lblSched = [[UILabel alloc] initWithFrame:schedLabelRect]; lblSched.textAlignment = UITextAlignmentLeft; lblSched.text = tmpMessage; lblSched.font = [UIFont boldSystemFontOfSize:10]; [cell.contentView addSubview: lblSched]; [lblSched release]; My numberOfSectionsInTableView method looks like this // Figure out how many sections there are by a distinct count of the groups field // The groups are entered by user when creating tasks //______________________________________________________________________________________ NSString *groupquery = [NSString stringWithFormat:@"SELECT COUNT(DISTINCT groups) as Sum FROM Tasks"]; int sum; sqlite3_stmt *statement; if (sqlite3_prepare_v2( database, [groupquery UTF8String], -1, &statement, nil) == SQLITE_OK) { while (sqlite3_step(statement) == SQLITE_ROW) { sum = sqlite3_column_int(statement, 0); } sqlite3_finalize(statement); } if (sum=0) { return 1; } return 2; } I know I'm going wrong here but this is all that's in my numberOfRowsInSection method return [ids count];

    Read the article

  • Covariance and Contravariance in C#

    - by edalorzo
    I will start by saying that I am Java developer learning to program in C#. As such I do comparisons of what I know with what I am learning. I have been playing with C# generics for a few hours now, and I have been able to reproduce the same things I know in Java in C#, with the exception of a couple of examples using covariance and contravariance. The book I am reading is not very good in the subject. I will certainly seek more info on the web, but while I do that, perhaps you can help me find a C# implementation for the following Java code. An example is worth a thousand words, and I was hoping that by looking a good code sample I will be able to assimilate this more rapidly. Covariance In Java I can do something like this: public static double sum(List<? extends Number> numbers) { double summation = 0.0; for(Number number : numbers){ summation += number.doubleValue(); } return summation; } I can use this code as follows: List<Integer> myInts = asList(1,2,3,4,5); List<Double> myDoubles = asList(3.14, 5.5, 78.9); List<Long> myLongs = asList(1L, 2L, 3L); double result = 0.0; result = sum(myInts); result = sum(myDoubles) result = sum(myLongs); Now I did discover that C# supports covariance/contravariance only on interfaces and as long as they have been explicitly declared to do so (out). I think I was not able to reproduce this case, because I could not find a common ancestor of all numbers, but I believe that I could have used IEnumerable to implement such thing if a common ancestor exists. Since IEnumerable is a covariant type. Right? Any thoughts on how to implement the list above? Just point me into the right direction. Is there any common ancestor of all numeric types? Contravariance The contravariance example I tried was the following. In Java I can do this to copy one list into another. public static void copy(List<? extends Number> source, List<? super Number> destiny){ for(Number number : source) { destiny.add(number); } } Then I could use it with contravariant types as follows: List<Object> anything = new ArrayList<Object>(); List<Integer> myInts = asList(1,2,3,4,5); copy(myInts, anything); My basic problem, trying to implement this in C# is that I could not find an interface that was both covariant and contravariant at the same time, as it is case of List in my example above. Maybe it can be done with two different interface in C#. Any thoughts on how to implement this? Thank you very much to everyone for any answers you can contribute. I am pretty sure I will learn a lot from any example you can provide.

    Read the article

  • sql perfomance on new server

    - by Rapunzo
    My database is running on a pc (AMD Phenom x6, intel ssd disk, 8GB DDR3 RAM and windows 7 OS + sql server 2008 R2 sp3 ) and it started working hard, timeout problems and up to 30 seconds long queries after 200 mb of database And I also have an old server pc (IBM x-series 266: 72*3 15k rpm scsi discs with raid5, 4 gb ram and windows server 2003 + sql server 2008 R2 sp3 ) and same query start to give results in 100 seconds.. I tried query analyser tool for tuning my indexed. but not so much improvements. its a big dissapointment for me. because I thought even its an old server pc it should be more powerfull with 15k rpm discs with raid5. what should I do. do I need $10.000 new server to get a good performance for my sql server? cant I use that IBM server? Extra information: there is 50 sql users and its an ERP program. There is my query ALTER FUNCTION [dbo].[fnDispoTerbiye] ( ) RETURNS TABLE AS RETURN ( SELECT MD.dispoNo, SV.sevkNo, M1.musteriAdi AS musteri, SD.tipTurId, TT.tipTur, SD.tipNo, SD.desenNo, SD.varyantNo, SUM(T.topMetre) AS toplamSevkMetre, MD.dispoMetresi, DT.gelisMetresi, ISNULL(DT.fire, 0) AS fire, SV.sevkTarihi, DT.gelisTarihi, SP.mamulTermin, SD.miktar AS siparisMiktari, M.musteriAdi AS boyahane, MD.akisNotu AS islemler, --dbo.fnAkisIslemleri(MD.dispoNo) DT.partiNo, DT.iplikBoyaId, B.tanimAd AS BoyaTuru, MAX(HD.hamEn) AS hamEn, MAX(HD.hamGramaj) AS hamGramaj, TS.mamulEn, TS.mamulGramaj, DT.atkiCekmesi, DT.cozguCekmesi, DT.fiyat, DV.dovizCins, DT.dovizId, (SELECT CASE WHEN DT.dovizId = 2 THEN CAST(round(SUM(T .topMetre) * DT.fiyat * (SELECT TOP 1 satis FROM tblKur WHERE dovizId = 2 ORDER BY tarih DESC), 2) AS numeric(18, 2)) WHEN DT.dovizId = 3 THEN CAST(round(SUM(T .topMetre) * DT.fiyat * (SELECT TOP 1 satis FROM tblKur WHERE dovizId = 3 ORDER BY tarih DESC), 2) AS numeric(18, 2)) WHEN DT.dovizId = 1 THEN CAST(round(SUM(T .topMetre) * DT.fiyat * (SELECT TOP 1 satis FROM tblKur WHERE dovizId = 1 ORDER BY tarih DESC), 2) AS numeric(18, 2)) END AS Expr1) AS ToplamTLfiyat, DT.aciklama, MD.dispoNotu, SD.siparisId, SD.siparisDetayId, DT.sqlUserName, DT.kayitTarihi, O.orguAd, 'Çözgü=(' + (SELECT dbo.fnTipIplikler(SD.tipTurId, SD.tipNo, SD.desenNo, SD.varyantNo, 1) AS Expr1) + ')' + ' Atki=(' + (SELECT dbo.fnTipIplikler(SD.tipTurId, SD.tipNo, SD.desenNo, SD.varyantNo, 2) AS Expr1) + ')' AS iplikAciklama, DT.prosesOk, dbo.[fnYikamaTalimat](SP.siparisId) yikamaTalimati FROM tblDoviz AS DV WITH(NOLOCK) INNER JOIN tblDispoTerbiye AS DT WITH(NOLOCK) INNER JOIN tblTanimlar AS B WITH(NOLOCK) ON DT.iplikBoyaId = B.tanimId AND B.tanimTurId = 2 ON DV.id = DT.dovizId RIGHT OUTER JOIN tblMusteri AS M1 WITH(NOLOCK) INNER JOIN tblSiparisDetay AS SD WITH(NOLOCK) INNER JOIN tblDispo AS MD WITH(NOLOCK) ON SD.siparisDetayId = MD.siparisDetayId INNER JOIN tblTipTur AS TT WITH(NOLOCK) ON SD.tipTurId = TT.tipTurId INNER JOIN tblSiparis AS SP WITH(NOLOCK) ON SD.siparisId = SP.siparisId ON M1.musteriNo = SP.musteriNo INNER JOIN tblTip AS TP WITH(NOLOCK) ON SD.tipTurId = TP.tipTurId AND SD.tipNo = TP.tipNo AND SD.desenNo = TP.desen AND SD.varyantNo = TP.varyant INNER JOIN tblOrgu AS O WITH(NOLOCK) ON TP.orguId = O.orguId INNER JOIN tblMusteri AS M WITH(NOLOCK) INNER JOIN tblSevkiyat AS SV WITH(NOLOCK) ON M.musteriNo = SV.musteriNo INNER JOIN tblSevkDetay AS SVD WITH(NOLOCK) ON SV.sevkNo = SVD.sevkNo ON MD.mamulDispoHamSevkno = SV.sevkNo LEFT OUTER JOIN tblTop AS T WITH(NOLOCK) INNER JOIN tblDispo AS HD WITH(NOLOCK) ON T.dispoNo = HD.dispoNo AND T.dispoTuruId = HD.dispoTuruId ON SVD.dispoTuruId = T.dispoTuruId AND SVD.dispoNo = T.dispoNo AND SVD.topNo = T.topNo AND MD.siparisDetayId = HD.siparisDetayId ON DT.dispoTuruId = MD.dispoTuruId AND DT.dispoNo = MD.dispoNo LEFT OUTER JOIN tblDispoTerbiyeTest AS TS WITH(NOLOCK) ON DT.dispoTuruId = TS.dispoTuruId AND DT.dispoNo = TS.dispoNo --WHERE DT.gelisTarihi IS NULL -- OR DT.gelisTarihi > GETDATE()-30 GROUP BY MD.dispoNo, DT.partiNo, DT.iplikBoyaId, TS.mamulEn, TS.mamulGramaj, DT.gelisMetresi, DT.gelisTarihi, DT.atkiCekmesi, DT.cozguCekmesi, DT.fire, DT.fiyat, DT.aciklama, DT.sqlUserName, DT.kayitTarihi, SD.tipTurId, TT.tipTur, SD.tipNo, SD.desenNo, SD.varyantNo, SD.siparisId, SD.siparisDetayId, B.tanimAd, M.musteriAdi, M.musteriAdi, M1.musteriAdi, O.orguAd, TP.iplikAciklama, SD.miktar, MD.dispoNotu, SP.mamulTermin, DT.dovizId, DV.dovizCins, MD.dispoMetresi, MD.akisNotu, SV.sevkNo, SV.sevkTarihi, DT.prosesOk,SP.siparisId )

    Read the article

  • Java code optimization leads to numerical inaccuracies and errors

    - by rano
    I'm trying to implement a version of the Fuzzy C-Means algorithm in Java and I'm trying to do some optimization by computing just once everything that can be computed just once. This is an iterative algorithm and regarding the updating of a matrix, the clusters x pixels membership matrix U, this is the update rule I want to optimize: where the x are the element of a matrix X (pixels x features) and v belongs to the matrix V (clusters x features). And m is a parameter that ranges from 1.1 to infinity. The distance used is the euclidean norm. If I had to implement this formula in a banal way I'd do: for(int i = 0; i < X.length; i++) { int count = 0; for(int j = 0; j < V.length; j++) { double num = D[i][j]; double sumTerms = 0; for(int k = 0; k < V.length; k++) { double thisDistance = D[i][k]; sumTerms += Math.pow(num / thisDistance, (1.0 / (m - 1.0))); } U[i][j] = (float) (1f / sumTerms); } } In this way some optimization is already done, I precomputed all the possible squared distances between X and V and stored them in a matrix D but that is not enough, since I'm cycling througn the elements of V two times resulting in two nested loops. Looking at the formula the numerator of the fraction is independent of the sum so I can compute numerator and denominator independently and the denominator can be computed just once for each pixel. So I came to a solution like this: int nClusters = V.length; double exp = (1.0 / (m - 1.0)); for(int i = 0; i < X.length; i++) { int count = 0; for(int j = 0; j < nClusters; j++) { double distance = D[i][j]; double denominator = D[i][nClusters]; double numerator = Math.pow(distance, exp); U[i][j] = (float) (1f / (numerator * denominator)); } } Where I precomputed the denominator into an additional column of the matrix D while I was computing the distances: for (int i = 0; i < X.length; i++) { for (int j = 0; j < V.length; j++) { double sum = 0; for (int k = 0; k < nDims; k++) { final double d = X[i][k] - V[j][k]; sum += d * d; } D[i][j] = sum; D[i][B.length] += Math.pow(1 / D[i][j], exp); } } By doing so I encounter numerical differences between the 'banal' computation and the second one that leads to different numerical value in U (not in the first iterates but soon enough). I guess that the problem is that exponentiate very small numbers to high values (the elements of U can range from 0.0 to 1.0 and exp , for m = 1.1, is 10) leads to ver y small values, whereas by dividing the numerator and the denominator and THEN exponentiating the result seems to be better numerically. The problem is it involves much more operations. Am I doing something wrong? Is there a possible solution to get both the code optimized and numerically stable? Any suggestion or criticism will be appreciated.

    Read the article

  • Nested loop traversing arrays

    - by alecco
    There are 2 very big series of elements, the second 100 times bigger than the first. For each element of the first series, there are 0 or more elements on the second series. This can be traversed and processed with 2 nested loops. But the unpredictability of the amount of matching elements for each member of the first array makes things very, very slow. The actual processing of the 2nd series of elements involves logical and (&) and a population count. I couldn't find good optimizations using C but I am considering doing inline asm, doing rep* mov* or similar for each element of the first series and then doing the batch processing of the matching bytes of the second series, perhaps in buffers of 1MB or something. But the code would be get quite messy. Does anybody know of a better way? C preferred but x86 ASM OK too. Many thanks! Sample/demo code with simplified problem, first series are "people" and second series are "events", for clarity's sake. (the original problem is actually 100m and 10,000m entries!) #include <stdio.h> #include <stdint.h> #define PEOPLE 1000000 // 1m struct Person { uint8_t age; // Filtering condition uint8_t cnt; // Number of events for this person in E } P[PEOPLE]; // Each has 0 or more bytes with bit flags #define EVENTS 100000000 // 100m uint8_t P1[EVENTS]; // Property 1 flags uint8_t P2[EVENTS]; // Property 2 flags void init_arrays() { for (int i = 0; i < PEOPLE; i++) { // just some stuff P[i].age = i & 0x07; P[i].cnt = i % 220; // assert( sum < EVENTS ); } for (int i = 0; i < EVENTS; i++) { P1[i] = i % 7; // just some stuff P2[i] = i % 9; // just some other stuff } } int main(int argc, char *argv[]) { uint64_t sum = 0, fcur = 0; int age_filter = 7; // just some init_arrays(); // Init P, P1, P2 for (int64_t p = 0; p < PEOPLE ; p++) if (P[p].age < age_filter) for (int64_t e = 0; e < P[p].cnt ; e++, fcur++) sum += __builtin_popcount( P1[fcur] & P2[fcur] ); else fcur += P[p].cnt; // skip this person's events printf("(dummy %ld %ld)\n", sum, fcur ); return 0; } gcc -O5 -march=native -std=c99 test.c -o test

    Read the article

  • SQL SERVER – Get All the Information of Database using sys.databases

    - by pinaldave
    Earlier I wrote blog article SQL SERVER – Finding Last Backup Time for All Database. In the response of this article I have received very interesting script from SQL Server Expert Matteo as a comment in the blog. He has written script using sys.databases which provides plenty of the information about database. I suggest you can run this on your database and know unknown of your databases as well. SELECT database_id, CONVERT(VARCHAR(25), DB.name) AS dbName, CONVERT(VARCHAR(10), DATABASEPROPERTYEX(name, 'status')) AS [Status], state_desc, (SELECT COUNT(1) FROM sys.master_files WHERE DB_NAME(database_id) = DB.name AND type_desc = 'rows') AS DataFiles, (SELECT SUM((size*8)/1024) FROM sys.master_files WHERE DB_NAME(database_id) = DB.name AND type_desc = 'rows') AS [Data MB], (SELECT COUNT(1) FROM sys.master_files WHERE DB_NAME(database_id) = DB.name AND type_desc = 'log') AS LogFiles, (SELECT SUM((size*8)/1024) FROM sys.master_files WHERE DB_NAME(database_id) = DB.name AND type_desc = 'log') AS [Log MB], user_access_desc AS [User access], recovery_model_desc AS [Recovery model], CASE compatibility_level WHEN 60 THEN '60 (SQL Server 6.0)' WHEN 65 THEN '65 (SQL Server 6.5)' WHEN 70 THEN '70 (SQL Server 7.0)' WHEN 80 THEN '80 (SQL Server 2000)' WHEN 90 THEN '90 (SQL Server 2005)' WHEN 100 THEN '100 (SQL Server 2008)' END AS [compatibility level], CONVERT(VARCHAR(20), create_date, 103) + ' ' + CONVERT(VARCHAR(20), create_date, 108) AS [Creation date], -- last backup ISNULL((SELECT TOP 1 CASE TYPE WHEN 'D' THEN 'Full' WHEN 'I' THEN 'Differential' WHEN 'L' THEN 'Transaction log' END + ' – ' + LTRIM(ISNULL(STR(ABS(DATEDIFF(DAY, GETDATE(),Backup_finish_date))) + ' days ago', 'NEVER')) + ' – ' + CONVERT(VARCHAR(20), backup_start_date, 103) + ' ' + CONVERT(VARCHAR(20), backup_start_date, 108) + ' – ' + CONVERT(VARCHAR(20), backup_finish_date, 103) + ' ' + CONVERT(VARCHAR(20), backup_finish_date, 108) + ' (' + CAST(DATEDIFF(second, BK.backup_start_date, BK.backup_finish_date) AS VARCHAR(4)) + ' ' + 'seconds)' FROM msdb..backupset BK WHERE BK.database_name = DB.name ORDER BY backup_set_id DESC),'-') AS [Last backup], CASE WHEN is_fulltext_enabled = 1 THEN 'Fulltext enabled' ELSE '' END AS [fulltext], CASE WHEN is_auto_close_on = 1 THEN 'autoclose' ELSE '' END AS [autoclose], page_verify_option_desc AS [page verify option], CASE WHEN is_read_only = 1 THEN 'read only' ELSE '' END AS [read only], CASE WHEN is_auto_shrink_on = 1 THEN 'autoshrink' ELSE '' END AS [autoshrink], CASE WHEN is_auto_create_stats_on = 1 THEN 'auto create statistics' ELSE '' END AS [auto create statistics], CASE WHEN is_auto_update_stats_on = 1 THEN 'auto update statistics' ELSE '' END AS [auto update statistics], CASE WHEN is_in_standby = 1 THEN 'standby' ELSE '' END AS [standby], CASE WHEN is_cleanly_shutdown = 1 THEN 'cleanly shutdown' ELSE '' END AS [cleanly shutdown] FROM sys.databases DB ORDER BY dbName, [Last backup] DESC, NAME Please let me know if you find this information useful. Reference: Pinal Dave (http://blog.SQLAuthority.com) Filed under: Pinal Dave, Readers Contribution, SQL, SQL Authority, SQL Query, SQL Scripts, SQL Server, SQL Tips and Tricks, SQLServer, T SQL, Technology

    Read the article

  • Including Overestimates in MSF Agile Burndown Report

    After using the MSF Agile Burndown report for a few weeks in our new TFS 2010 environment, I have to say I am a huge fan.  I especially find the assignment of Work (hours) portion to be very useful in motivating the team to keep their tasks up to date every day.  Here is a view of the report that you get out of the box. However, I have one problem.  Id like the top line to have some more meaning.  Specifically, when it is changing is that an indication of scope creep, mis-estimation or a combination of the two.  So, today I decided to try to build in a view that would show overestimated time.  This would give me a more consistent top line.  My idea was to add another visual area on top of the graph whenever my originally estimated time was greater than the sum of completed and remaining.  This will effectively show me at least when the top line goes down whether it was scope change or over-estimation. Here is the final result. How did I do it?  Step 1: Add Cumulative_Original_Estimate field to the dsBurndown My approach was to follow the pattern where the completed time is included in the burndown chart and add my Overestimated hours.  First I added a field to the dsBurndown to hold the estimated time.         <Field Name="Cumulative_Original_Estimate">           <DataField><?xml version="1.0" encoding="utf-8"?><Field xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xsi:type="Measure" UniqueName="[Measures].[Microsoft_VSTS_Scheduling_OriginalEstimate]" /></DataField>           <rd:TypeName>System.Int32</rd:TypeName>         </Field> Step 2: Add a column to the query SELECT {     [Measures].[DateValue],     [Measures].[Work Item Count],     [Measures].[Microsoft_VSTS_Scheduling_RemainingWork],     [Measures].[Microsoft_VSTS_Scheduling_CompletedWork],     [Measures].[Microsoft_VSTS_Scheduling_OriginalEstimate],     [Measures].[RemainingWorkLine],     [Measures].[CountLine] Step 3: Add a new Item to the QueryDefinition <Item> <ID xsi:type="Measure"> <MeasureName>Microsoft_VSTS_Scheduling_OriginalEstimate</MeasureName> <UniqueName>[Measures].[Microsoft_VSTS_Scheduling_OriginalEstimate]</UniqueName> </ID> <ItemCaption>Cumulative Original Estimate</ItemCaption> <FormattedValue>true</FormattedValue> </Item> Step 4: Add a new ChartMember to DundasChartControl1 The burndown chart is called DundasChartControl1.  I need to add a ChartMember for the estimated time. <ChartMember>   <Label>Cumulative Original Estimate</Label> </ChartMember> Step 5: Add a ChartSeries to show the Overestimated Time <ChartSeries Name="OriginalEstimate">   <Hidden>=IIF(Parameters!YAxis.Value="count",True,False)</Hidden>   <ChartDataPoints>     <ChartDataPoint>       <ChartDataPointValues>         <Y>=IIF(Parameters!YAxis.Value = "hours", IIF(SUM(Fields!Cumulative_Original_Estimate.Value)>SUM(Fields!Cumulative_Completed_Work.Value+Fields!Cumulative_Remaining_Work.Value), SUM(Fields!Cumulative_Original_Estimate.Value-(Fields!Cumulative_Completed_Work.Value+Fields!Cumulative_Remaining_Work.Value)),Nothing),Nothing)</Y>       </ChartDataPointValues>       <ChartDataLabel>         <Style>           <FontFamily>Microsoft Sans Serif</FontFamily>           <FontSize>8pt</FontSize>         </Style>       </ChartDataLabel>       <Style>         <Border>           <Color>#9bdb00</Color>           <Width>0.75pt</Width>         </Border>         <Color>#666666</Color>         <BackgroundGradientEndColor>#666666</BackgroundGradientEndColor>       </Style>       <ChartMarker>         <Style />       </ChartMarker>       <CustomProperties>         <CustomProperty>           <Name>LabelStyle</Name>           <Value>Top</Value>         </CustomProperty>       </CustomProperties>     </ChartDataPoint>   </ChartDataPoints>   <Type>Area</Type>   <Subtype>Stacked</Subtype>   <Style />   <ChartEmptyPoints>     <Style>       <Color>#00ffffff</Color>     </Style>     <ChartMarker>       <Style />     </ChartMarker>     <ChartDataLabel>       <Style />     </ChartDataLabel>   </ChartEmptyPoints>   <LegendName>Default</LegendName>   <ChartItemInLegend>     <LegendText>Overestimated Hours</LegendText>   </ChartItemInLegend>   <ChartAreaName>Default</ChartAreaName>   <ValueAxisName>Primary</ValueAxisName>   <CategoryAxisName>Primary</CategoryAxisName>   <ChartSmartLabel>     <Disabled>true</Disabled>     <MaxMovingDistance>22.5pt</MaxMovingDistance>   </ChartSmartLabel> </ChartSeries> Thats it.  I find the improved report to add some value over the out of the box version.  You can download the updated rdl for the report here.  Did you know that DotNetSlackers also publishes .net articles written by top known .net Authors? We already have over 80 articles in several categories including Silverlight. Take a look: here.

    Read the article

  • Nautilus can't start due to segmentation fault

    - by Dmitriy Sukharev
    Out of the blue I can't start nautilus today. When I try to open any directory it tries to open it, and sometimes I even can see the content of directory, but finally it's closed, after that there are no icons on desktop. When I tried to launch nautilus from terminal, I got: $ nautilus . Initializing nautilus-dropbox 0.7.1 Initializing nautilus-gdu extension Segmentation fault (core dumped) I've tried to move ~/.local/share/gvfs-metadata folder, I don't have nautilus-open-terminal package and don't have file /usr/local/lib/libgtk-3.so.0 Also I can't update system right now. All the time I'm getting the the same hash-sum error: $ sudo apt-get update [sudo] password for dmitriy: Ign http://mirror.mirohost.net precise InRelease Ign http://mirror.mirohost.net precise-updates InRelease Ign http://mirror.mirohost.net precise-security InRelease Hit http://mirror.mirohost.net precise Release.gpg ... Ign http://ppa.launchpad.net precise/main Translation-en Hit http://mirror.mirohost.net precise-security/restricted Translation-en Hit http://mirror.mirohost.net precise-security/universe Translation-en Fetched 1 B in 1s (0 B/s) W: Failed to fetch gzip:/var/lib/apt/lists/partial/mirror.mirohost.net_ubuntu_dists_precise_universe_source_Sources Hash Sum mismatch E: Some index files failed to download. They have been ignored, or old ones used instead. Any ideas how to rescue my system? UPD: In syslog I have the following errors: Jul 7 21:35:02 dmitriy-desktop kernel: [ 58.059141] nautilus[1991]: segfault at 7fc09d9bb700 ip 00007fc0abb5feb6 sp 00007fff6caa4cf8 error 4 in libc-2.15.so[7fc0aba24000+1b3000] Jul 7 21:35:39 dmitriy-desktop kernel: [ 94.356490] update-notifier[3358]: segfault at 7f6507611700 ip 00007f64cc221eb6 sp 00007fffbcc0dd88 error 4 in libc-2.15.so[7f64cc0e6000+1b3000] Jul 7 21:37:45 dmitriy-desktop kernel: [ 220.501859] nautilus[3629]: segfault at 7f9b9744c700 ip 00007f9b7c9c6eb6 sp 00007fff72e990f8 error 4 in libc-2.15.so[7f9b7c88b000+1b3000] UPD2: Ubuntu version is 12.04.

    Read the article

  • how this scaling down for css code is worked?

    - by harris
    this is a code for scaling down for css. i was wondering, how this worked. please someone explain to me part by part. thank you very much. /* ======================================================================== / / Copyright (C) 2000 - 2009 ND-Tech. Co., Ltd. / / All Rights Reserved. / / ======================================================================== / / Project : ScaleDown Created : 31-AUG-2009 / / File : main.c Contact : [email protected] / / ======================================================================== / / You are free to use or modify this code to the following restrictions: / / Acknowledge ND Tech. Co. Ltd. / / Or, put "Parts of code by ND Tech. Co., Ltd." / / Or, leave this header as it is. / / in somewhere in your code. / / ======================================================================== */ include "vm3224k.h" define CE0CTL *(volatile int *)(0x01800008) define CE2CTL *(volatile int *)(0x01800010) define SDCTL *(volatile int *)(0x01800018) define LED *(volatile short *)(0x90080000) // Definitions for async access(change as you wish) define WSU (2<<28) // Write Setup : 0-15 define WST (8<<22) // Write Strobe: 0-63 define WHD (2<<20) // Write Hold : 0-3 define RSU (2<<16) // Read Setup : 0-15 define TA (3<<14) // Turn Around : 0-3 define RST (8<<8) // Read Strobe : 0-63 define RHD (2<<0) // Read Hold : 0-3 define MTYPE (2<<4) /* EDMA Registers */ define PaRAM_OPT 0 // Options define PaRAM_SRC 1 // Source Address define PaRAM_CNT 2 // Frame count, Element count define PaRAM_DST 3 // Destination Address define PaRAM_IDX 4 // Frame index, Element index define PaRAM_RDL 5 // Element count reload, Link address define EDMA_CIPR *(volatile int *)0x01A0FFE4 // EDMA Channel interrupt pending low register define EDMA_CIER *(volatile int *)0x01A0FFE8 // EDMA Channel interrupt enable low register define EDMA_CCER *(volatile int *)0x01A0FFEC // EDMA Channel chain enable register define EDMA_ER *(volatile int *)0x01A0FFF0 // EDMA Event low register define EDMA_EER *(volatile int *)0x01A0FFF4 // EDMA Event enable low register define EDMA_ECR *(volatile int *)0x01A0FFF8 // EDMA Event clear low register define EDMA_ESR *(volatile int *)0x01A0FFFC // EDMA Event set low register define PRI (2<<29) // 1:High priority, 2:Low priority define ESIZE (1<<27) // 0:32bit, 1:16bit, 2:8bit, 3:reserved define DS2 (0<<26) // 1:2-Dimensional define SUM (0<<24) // 0:no update, 1:increment, 2:decrement, 3:by index define DD2 (0<<23) // 1:2-Dimensional define DUM (0<<21) // 0:no update, 1:increment, 2:decrement, 3:by index define TCINT (1<<20) // 0:disable, 1:enable define TCC (8<<16) // 4 bit code define LINK (0<<1) // 0:disable, 1:enable define FS (1<<0) // 0:element, 1:frame define OptionField_0 (PRI|ESIZE|DS2|SUM|DD2|DUM|TCINT|TCC|LINK|FS) define DD2_1 (1<<23) // 1:2-Dimensional define DUM_1 (1<<21) // 0:no update, 1:increment, 2:decrement, 3:by index define TCC_1 (9<<16) // 4 bit code define OptionField_1 (PRI|ESIZE|DS2|SUM|DD2_1|DUM_1|TCINT|TCC_1|LINK|FS) define TCC_2 (10<<16)// 4 bit code define OptionField_2 (PRI|ESIZE|DS2|SUM|DD2|DUM|TCINT|TCC_2|LINK|FS) define DS2_3 (1<<26) // 1:2-Dimensional define SUM_3 (1<<24) // 0:no update, 1:increment, 2:decrement, 3:by index define TCC_3 (11<<16)// 4 bit code define OptionField_3 (PRI|ESIZE|DS2_3|SUM_3|DD2|DUM|TCINT|TCC_3|LINK|FS) pragma DATA_SECTION ( lcd,".sdram" ) pragma DATA_SECTION ( cam,".sdram" ) pragma DATA_SECTION ( rgb,".sdram" ) pragma DATA_SECTION ( u,".sdram" ) extern cregister volatile unsigned int IER; extern cregister volatile unsigned int CSR; short camcode = 0x08000; short lcdcode = 0x00000; short lcd[2][240][320]; short cam[2][240][320]; short rgb[64][32][32]; short bufsel; int *Cevent,*Levent,*CLink,flag=1; unsigned char v[240][160],out_y[120][160]; unsigned char y[240][320],out_u[120][80]; unsigned char u[240][160],out_v[120][80]; void PLL6713() { int i; // CPU Clock Input : 50MHz *(volatile int *)(0x01b7c100) = *(volatile int *)(0x01b7c100) & 0xfffffffe; for(i=0;i<4;i++); *(volatile int *)(0x01b7c100) = *(volatile int *)(0x01b7c100) | 0x08; *(volatile int *)(0x01b7c114) = 0x08001; // 50MHz/2 = 25MHz *(volatile int *)(0x01b7c110) = 0x0c; // 25MHz * 12 = 300MHz *(volatile int *)(0x01b7c118) = 0x08000; // SYSCLK1 = 300MHz/1 = 300MHz *(volatile int *)(0x01b7c11c) = 0x08001; // SYSCLK2 = 300MHz/2 = 150MHz // Peripheral Clock *(volatile int *)(0x01b7c120) = 0x08003; // SYSCLK3 = 300MHz/4 = 75MHz // SDRAM Clock for(i=0;i<4;i++); *(volatile int *)(0x01b7c100) = *(volatile int *)(0x01b7c100) & 0xfffffff7; for(i=0;i<4;i++); *(volatile int *)(0x01b7c100) = *(volatile int *)(0x01b7c100) | 0x01; } unsigned short ybr_565(short y,short u,short v) { int r,g,b; b = y + 1772*(u-128)/1000; if (b<0) b=0; if (b>255) b=255; g = y - (344*(u-128) + 714*(v-128))/1000; if (g<0) g=0; if (g>255) g=255; r = y + 1402*(v-128)/1000; if (r<0) r=0; if (r>255) r=255; return ((r&0x0f8)<<8)|((g&0x0fc)<<3)|((b&0x0f8)>>3); } void yuyv2yuv(char *yuyv,char *y,char *u,char *v) { int i,j,dy,dy1,dy2,s; for (j=s=dy=dy1=dy2=0;j<240;j++) { for (i=0;i<320;i+=2) { u[dy1++] = yuyv[s++]; y[dy++] = yuyv[s++]; v[dy2++] = yuyv[s++]; y[dy++] = yuyv[s++]; } } } interrupt void c_int06(void) { if(EDMA_CIPR&0x800){ EDMA_CIPR = 0xffff; bufsel=(++bufsel&0x01); Cevent[PaRAM_DST] = (int)cam[(bufsel+1)&0x01]; Levent[PaRAM_SRC] = (int)lcd[(bufsel+1)&0x01]; EDMA_ESR = 0x80; flag=1; } } void main() { int i,j,k,y0,y1,v0,u0; bufsel = 0; CSR &= (~0x1); PLL6713(); // Initialize C6713 PLL CE0CTL = 0xffffbf33;// SDRAM Space CE2CTL = (WSU|WST|WHD|RSU|RST|RHD|MTYPE); SDCTL = 0x57115000; vm3224init(); // Initialize vm3224k2 vm3224rate(1); // Set frame rate vm3224bl(15); // Set backlight VM3224CNTL = VM3224CNTL&0xffff | 0x2; // vm3224 interrupt enable for (k=0;k<64;k++) // Create RGB565 lookup table for (i=0;i<32;i++) for (j=0;j<32;j++) rgb[k][i][j] = ybr_565(k<<2,i<<3,j<<3); Cevent = (int *)(0x01a00000 + 24 * 7); Cevent[PaRAM_OPT] = OptionField_0; Cevent[PaRAM_SRC] = (int)&camcode; Cevent[PaRAM_CNT] = 1; Cevent[PaRAM_DST] = (int)&VM3224ADDH; Cevent = (int *)(0x01a00000 + 24 * 8); Cevent[PaRAM_OPT] = OptionField_1; Cevent[PaRAM_SRC] = (int)&VM3224DATA; Cevent[PaRAM_CNT] = (239<<16)|320; Cevent[PaRAM_DST] = (int)cam[bufsel]; Cevent[PaRAM_IDX] = 0; Levent = (int *)(0x01a00000 + 24 * 9); Levent[PaRAM_OPT] = OptionField_2; Levent[PaRAM_SRC] = (int)&lcdcode; Levent[PaRAM_CNT] = 1; Levent[PaRAM_DST] = (int)&VM3224ADDH; Levent = (int *)(0x01a00000 + 24 * 10); Levent[PaRAM_OPT] = OptionField_3; Levent[PaRAM_SRC] = (int)lcd[bufsel]; Levent[PaRAM_CNT] = (239<<16)|320; Levent[PaRAM_DST] = (int)&VM3224DATA; Levent[PaRAM_IDX] = 0; IER = IER | (1<<6)|3; CSR = CSR | 0x1; EDMA_CCER = (1<<8)|(1<<9)|(1<<10); EDMA_CIER = (1<<11); EDMA_CIPR = 0xffff; EDMA_ESR = 0x80; while (1) { if(flag) { // LED = 0; yuyv2yuv((char *)cam[bufsel],(char *)y,(char *)u,(char *)v); for(j=0;j<240;j++) for(i=0;i<320;i++) lcd[bufsel][j][i]=0; for(j=0;j<240;j+=2) for(i=0;i<320;i+=2) out_y[j>>1][i>>1]=(y[j][i]+y[j][i+1]+y[j+1][i]+y[j+1][i+1])>>2; for(j=0;j<240;j+=2) for(i=0;i<160;i+=2) { out_u[j>>1][i>>1]=(u[j][i]+u[j][i+1]+u[j+1][i]+u[j+1][i+1])>>2; out_v[j>>1][i>>1]=(v[j][i]+v[j][i+1]+v[j+1][i]+v[j+1][i+1])>>2; } for (j=0;j<120;j++) for (i=0;i<160;i+=2) { y0 = out_y[j][i]>>2; u0 = out_u[j][i>>1]>>3; v0 = out_v[j][i>>1]>>3; y1 = out_y[j][i+1]>>2; lcd[bufsel][j+60][i+80]=rgb[y0][u0][v0]; lcd[bufsel][j+60][i+81]=rgb[y1][u0][v0]; } flag=0; // LED = 1; } } }

    Read the article

  • Convert ddply {plyr} to Oracle R Enterprise, or use with Embedded R Execution

    - by Mark Hornick
    The plyr package contains a set of tools for partitioning a problem into smaller sub-problems that can be more easily processed. One function within {plyr} is ddply, which allows you to specify subsets of a data.frame and then apply a function to each subset. The result is gathered into a single data.frame. Such a capability is very convenient. The function ddply also has a parallel option that if TRUE, will apply the function in parallel, using the backend provided by foreach. This type of functionality is available through Oracle R Enterprise using the ore.groupApply function. In this blog post, we show a few examples from Sean Anderson's "A quick introduction to plyr" to illustrate the correpsonding functionality using ore.groupApply. To get started, we'll create a demo data set and load the plyr package. set.seed(1) d <- data.frame(year = rep(2000:2014, each = 3),         count = round(runif(45, 0, 20))) dim(d) library(plyr) This first example takes the data frame, partitions it by year, and calculates the coefficient of variation of the count, returning a data frame. # Example 1 res <- ddply(d, "year", function(x) {   mean.count <- mean(x$count)   sd.count <- sd(x$count)   cv <- sd.count/mean.count   data.frame(cv.count = cv)   }) To illustrate the equivalent functionality in Oracle R Enterprise, using embedded R execution, we use the ore.groupApply function on the same data, but pushed to the database, creating an ore.frame. The function ore.push creates a temporary table in the database, returning a proxy object, the ore.frame. D <- ore.push(d) res <- ore.groupApply (D, D$year, function(x) {   mean.count <- mean(x$count)   sd.count <- sd(x$count)   cv <- sd.count/mean.count   data.frame(year=x$year[1], cv.count = cv)   }, FUN.VALUE=data.frame(year=1, cv.count=1)) You'll notice the similarities in the first three arguments. With ore.groupApply, we augment the function to return the specific data.frame we want. We also specify the argument FUN.VALUE, which describes the resulting data.frame. From our previous blog posts, you may recall that by default, ore.groupApply returns an ore.list containing the results of each function invocation. To get a data.frame, we specify the structure of the result. The results in both cases are the same, however the ore.groupApply result is an ore.frame. In this case the data stays in the database until it's actually required. This can result in significant memory and time savings whe data is large. R> class(res) [1] "ore.frame" attr(,"package") [1] "OREbase" R> head(res)    year cv.count 1 2000 0.3984848 2 2001 0.6062178 3 2002 0.2309401 4 2003 0.5773503 5 2004 0.3069680 6 2005 0.3431743 To make the ore.groupApply execute in parallel, you can specify the argument parallel with either TRUE, to use default database parallelism, or to a specific number, which serves as a hint to the database as to how many parallel R engines should be used. The next ddply example uses the summarise function, which creates a new data.frame. In ore.groupApply, the year column is passed in with the data. Since no automatic creation of columns takes place, we explicitly set the year column in the data.frame result to the value of the first row, since all rows received by the function have the same year. # Example 2 ddply(d, "year", summarise, mean.count = mean(count)) res <- ore.groupApply (D, D$year, function(x) {   mean.count <- mean(x$count)   data.frame(year=x$year[1], mean.count = mean.count)   }, FUN.VALUE=data.frame(year=1, mean.count=1)) R> head(res)    year mean.count 1 2000 7.666667 2 2001 13.333333 3 2002 15.000000 4 2003 3.000000 5 2004 12.333333 6 2005 14.666667 Example 3 uses the transform function with ddply, which modifies the existing data.frame. With ore.groupApply, we again construct the data.frame explicilty, which is returned as an ore.frame. # Example 3 ddply(d, "year", transform, total.count = sum(count)) res <- ore.groupApply (D, D$year, function(x) {   total.count <- sum(x$count)   data.frame(year=x$year[1], count=x$count, total.count = total.count)   }, FUN.VALUE=data.frame(year=1, count=1, total.count=1)) > head(res)    year count total.count 1 2000 5 23 2 2000 7 23 3 2000 11 23 4 2001 18 40 5 2001 4 40 6 2001 18 40 In Example 4, the mutate function with ddply enables you to define new columns that build on columns just defined. Since the construction of the data.frame using ore.groupApply is explicit, you always have complete control over when and how to use columns. # Example 4 ddply(d, "year", mutate, mu = mean(count), sigma = sd(count),       cv = sigma/mu) res <- ore.groupApply (D, D$year, function(x) {   mu <- mean(x$count)   sigma <- sd(x$count)   cv <- sigma/mu   data.frame(year=x$year[1], count=x$count, mu=mu, sigma=sigma, cv=cv)   }, FUN.VALUE=data.frame(year=1, count=1, mu=1,sigma=1,cv=1)) R> head(res)    year count mu sigma cv 1 2000 5 7.666667 3.055050 0.3984848 2 2000 7 7.666667 3.055050 0.3984848 3 2000 11 7.666667 3.055050 0.3984848 4 2001 18 13.333333 8.082904 0.6062178 5 2001 4 13.333333 8.082904 0.6062178 6 2001 18 13.333333 8.082904 0.6062178 In Example 5, ddply is used to partition data on multiple columns before constructing the result. Realizing this with ore.groupApply involves creating an index column out of the concatenation of the columns used for partitioning. This example also allows us to illustrate using the ORE transparency layer to subset the data. # Example 5 baseball.dat <- subset(baseball, year > 2000) # data from the plyr package x <- ddply(baseball.dat, c("year", "team"), summarize,            homeruns = sum(hr)) We first push the data set to the database to get an ore.frame. We then add the composite column and perform the subset, using the transparency layer. Since the results from database execution are unordered, we will explicitly sort these results and view the first 6 rows. BB.DAT <- ore.push(baseball) BB.DAT$index <- with(BB.DAT, paste(year, team, sep="+")) BB.DAT2 <- subset(BB.DAT, year > 2000) X <- ore.groupApply (BB.DAT2, BB.DAT2$index, function(x) {   data.frame(year=x$year[1], team=x$team[1], homeruns=sum(x$hr))   }, FUN.VALUE=data.frame(year=1, team="A", homeruns=1), parallel=FALSE) res <- ore.sort(X, by=c("year","team")) R> head(res)    year team homeruns 1 2001 ANA 4 2 2001 ARI 155 3 2001 ATL 63 4 2001 BAL 58 5 2001 BOS 77 6 2001 CHA 63 Our next example is derived from the ggplot function documentation. This illustrates the use of ddply within using the ggplot2 package. We first create a data.frame with demo data and use ddply to create some statistics for each group (gp). We then use ggplot to produce the graph. We can take this same code, push the data.frame df to the database and invoke this on the database server. The graph will be returned to the client window, as depicted below. # Example 6 with ggplot2 library(ggplot2) df <- data.frame(gp = factor(rep(letters[1:3], each = 10)),                  y = rnorm(30)) # Compute sample mean and standard deviation in each group library(plyr) ds <- ddply(df, .(gp), summarise, mean = mean(y), sd = sd(y)) # Set up a skeleton ggplot object and add layers: ggplot() +   geom_point(data = df, aes(x = gp, y = y)) +   geom_point(data = ds, aes(x = gp, y = mean),              colour = 'red', size = 3) +   geom_errorbar(data = ds, aes(x = gp, y = mean,                                ymin = mean - sd, ymax = mean + sd),              colour = 'red', width = 0.4) DF <- ore.push(df) ore.tableApply(DF, function(df) {   library(ggplot2)   library(plyr)   ds <- ddply(df, .(gp), summarise, mean = mean(y), sd = sd(y))   ggplot() +     geom_point(data = df, aes(x = gp, y = y)) +     geom_point(data = ds, aes(x = gp, y = mean),                colour = 'red', size = 3) +     geom_errorbar(data = ds, aes(x = gp, y = mean,                                  ymin = mean - sd, ymax = mean + sd),                   colour = 'red', width = 0.4) }) But let's take this one step further. Suppose we wanted to produce multiple graphs, partitioned on some index column. We replicate the data three times and add some noise to the y values, just to make the graphs a little different. We also create an index column to form our three partitions. Note that we've also specified that this should be executed in parallel, allowing Oracle Database to control and manage the server-side R engines. The result of ore.groupApply is an ore.list that contains the three graphs. Each graph can be viewed by printing the list element. df2 <- rbind(df,df,df) df2$y <- df2$y + rnorm(nrow(df2)) df2$index <- c(rep(1,300), rep(2,300), rep(3,300)) DF2 <- ore.push(df2) res <- ore.groupApply(DF2, DF2$index, function(df) {   df <- df[,1:2]   library(ggplot2)   library(plyr)   ds <- ddply(df, .(gp), summarise, mean = mean(y), sd = sd(y))   ggplot() +     geom_point(data = df, aes(x = gp, y = y)) +     geom_point(data = ds, aes(x = gp, y = mean),                colour = 'red', size = 3) +     geom_errorbar(data = ds, aes(x = gp, y = mean,                                  ymin = mean - sd, ymax = mean + sd),                   colour = 'red', width = 0.4)   }, parallel=TRUE) res[[1]] res[[2]] res[[3]] To recap, we've illustrated how various uses of ddply from the plyr package can be realized in ore.groupApply, which affords the user explicit control over the contents of the data.frame result in a straightforward manner. We've also highlighted how ddply can be used within an ore.groupApply call.

    Read the article

  • SQL Query for Determining SharePoint ACL Sizes

    - by Damon Armstrong
    When a SharePoint Access Control List (ACL) size exceeds more than 64kb for a particular URL, the contents under that URL become unsearchable due to limitations in the SharePoint search engine.  The error most often seen is The Parameter is Incorrect which really helps to pinpoint the problem (its difficult to convey extreme sarcasm here, please note that it is intended).  Exceeding this limit is not unheard of – it can happen when users brute force security into working by continually overriding inherited permissions and assigning user-level access to securable objects. Once you have this issue, determining where you need to focus to fix the problem can be difficult.  Fortunately, there is a query that you can run on a content database that can help identify the issue: SELECT [SiteId],      MIN([ScopeUrl]) AS URL,      SUM(DATALENGTH([Acl]))/1024 as AclSizeKB,      COUNT(*) AS AclEntries FROM [Perms] (NOLOCK) GROUP BY siteid ORDER BY AclSizeKB DESC This query results in a list of ACL sizes and entry counts on a site-by-site basis.  You can also remove grouping to see a more granular breakdown: SELECT [ScopeUrl] AS URL,       SUM(DATALENGTH([Acl]))/1024 as AclSizeKB,      COUNT(*) AS AclEntries FROM [Perms] (NOLOCK) GROUP BY ScopeUrl ORDER BY AclSizeKB DESC

    Read the article

  • Global vs. Local Monthly Searches in Adwords keyword tool

    - by Gregory
    I'm trying to learn how to use a keyword tool in Adwords. Here's what I entered: Country- Russia Language-Russian Desktop and laptop devices And the keyword was ???? ? ??????? (tours to Israel in Russian Cyrillic letters) . As a broad match type... Now... the results that I got were: Global monthly: 60,500 Local monthly: 40,500 If I got it right..."Global monthly" means in this context : worldwide average monthly searches for this search term in ANY language in any Google search site (google.ru, google.com.ua, google.com, google.fr etc.). It's all nice, BUT... Then I made an query for tours to Israel in English in the US...And I got: Global monthly: 60,500 Local monthly: 27,100 That doesn't make any sense to me though! How come the total sum (the global) is actually a smaller number than a combined sum of just TWO countries??? (27,100+40,500=67,60060,500) By "any language" they mean a translation of the term into ANY possible language???Or maybe by "language" Google means the language of searchers' operating system? or their browsers' language?

    Read the article

< Previous Page | 29 30 31 32 33 34 35 36 37 38 39 40  | Next Page >