Search Results

Search found 149 results on 6 pages for 'for comprehension'.

Page 3/6 | < Previous Page | 1 2 3 4 5 6  | Next Page >

• Python: Improving long cumulative sum

  - by Bo102010

  I have a program that operates on a large set of experimental data. The data is stored as a list of objects that are instances of a class with the following attributes: time_point - the time of the sample cluster - the name of the cluster of nodes from which the sample was taken code - the name of the node from which the sample was taken qty1 = the value of the sample for the first quantity qty2 = the value of the sample for the second quantity I need to derive some values from the data set, grouped in three ways - once for the sample as a whole, once for each cluster of nodes, and once for each node. The values I need to derive depend on the (time sorted) cumulative sums of qty1 and qty2: the maximum value of the element-wise sum of the cumulative sums of qty1 and qty2, the time point at which that maximum value occurred, and the values of qty1 and qty2 at that time point. I came up with the following solution: dataset.sort(key=operator.attrgetter('time_point')) # For the whole set sys_qty1 = 0 sys_qty2 = 0 sys_combo = 0 sys_max = 0 # For the cluster grouping cluster_qty1 = defaultdict(int) cluster_qty2 = defaultdict(int) cluster_combo = defaultdict(int) cluster_max = defaultdict(int) cluster_peak = defaultdict(int) # For the node grouping node_qty1 = defaultdict(int) node_qty2 = defaultdict(int) node_combo = defaultdict(int) node_max = defaultdict(int) node_peak = defaultdict(int) for t in dataset: # For the whole system ###################################################### sys_qty1 += t.qty1 sys_qty2 += t.qty2 sys_combo = sys_qty1 + sys_qty2 if sys_combo > sys_max: sys_max = sys_combo # The Peak class is to record the time point and the cumulative quantities system_peak = Peak(time_point=t.time_point, qty1=sys_qty1, qty2=sys_qty2) # For the cluster grouping ################################################## cluster_qty1[t.cluster] += t.qty1 cluster_qty2[t.cluster] += t.qty2 cluster_combo[t.cluster] = cluster_qty1[t.cluster] + cluster_qty2[t.cluster] if cluster_combo[t.cluster] > cluster_max[t.cluster]: cluster_max[t.cluster] = cluster_combo[t.cluster] cluster_peak[t.cluster] = Peak(time_point=t.time_point, qty1=cluster_qty1[t.cluster], qty2=cluster_qty2[t.cluster]) # For the node grouping ##################################################### node_qty1[t.node] += t.qty1 node_qty2[t.node] += t.qty2 node_combo[t.node] = node_qty1[t.node] + node_qty2[t.node] if node_combo[t.node] > node_max[t.node]: node_max[t.node] = node_combo[t.node] node_peak[t.node] = Peak(time_point=t.time_point, qty1=node_qty1[t.node], qty2=node_qty2[t.node]) This produces the correct output, but I'm wondering if it can be made more readable/Pythonic, and/or faster/more scalable. The above is attractive in that it only loops through the (large) dataset once, but unattractive in that I've essentially copied/pasted three copies of the same algorithm. To avoid the copy/paste issues of the above, I tried this also: def find_peaks(level, dataset): def grouping(object, attr_name): if attr_name == 'system': return attr_name else: return object.__dict__[attrname] cuml_qty1 = defaultdict(int) cuml_qty2 = defaultdict(int) cuml_combo = defaultdict(int) level_max = defaultdict(int) level_peak = defaultdict(int) for t in dataset: cuml_qty1[grouping(t, level)] += t.qty1 cuml_qty2[grouping(t, level)] += t.qty2 cuml_combo[grouping(t, level)] = (cuml_qty1[grouping(t, level)] + cuml_qty2[grouping(t, level)]) if cuml_combo[grouping(t, level)] > level_max[grouping(t, level)]: level_max[grouping(t, level)] = cuml_combo[grouping(t, level)] level_peak[grouping(t, level)] = Peak(time_point=t.time_point, qty1=node_qty1[grouping(t, level)], qty2=node_qty2[grouping(t, level)]) return level_peak system_peak = find_peaks('system', dataset) cluster_peak = find_peaks('cluster', dataset) node_peak = find_peaks('node', dataset) For the (non-grouped) system-level calculations, I also came up with this, which is pretty: dataset.sort(key=operator.attrgetter('time_point')) def cuml_sum(seq): rseq = [] t = 0 for i in seq: t += i rseq.append(t) return rseq time_get = operator.attrgetter('time_point') q1_get = operator.attrgetter('qty1') q2_get = operator.attrgetter('qty2') timeline = [time_get(t) for t in dataset] cuml_qty1 = cuml_sum([q1_get(t) for t in dataset]) cuml_qty2 = cuml_sum([q2_get(t) for t in dataset]) cuml_combo = [q1 + q2 for q1, q2 in zip(cuml_qty1, cuml_qty2)] combo_max = max(cuml_combo) time_max = timeline.index(combo_max) q1_at_max = cuml_qty1.index(time_max) q2_at_max = cuml_qty2.index(time_max) However, despite this version's cool use of list comprehensions and zip(), it loops through the dataset three times just for the system-level calculations, and I can't think of a good way to do the cluster-level and node-level calaculations without doing something slow like: timeline = defaultdict(int) cuml_qty1 = defaultdict(int) #...etc. for c in cluster_list: timeline[c] = [time_get(t) for t in dataset if t.cluster == c] cuml_qty1[c] = [q1_get(t) for t in dataset if t.cluster == c] #...etc. Does anyone here at Stack Overflow have suggestions for improvements? The first snippet above runs well for my initial dataset (on the order of a million records), but later datasets will have more records and clusters/nodes, so scalability is a concern. This is my first non-trivial use of Python, and I want to make sure I'm taking proper advantage of the language (this is replacing a very convoluted set of SQL queries, and earlier versions of the Python version were essentially very ineffecient straight transalations of what that did). I don't normally do much programming, so I may be missing something elementary. Many thanks!

  Read the article

• Why is it still so hard to write software?

  - by nornagon

  Writing software, I find, is composed of two parts: the Idea, and the Implementation. The Idea is about thinking: "I have this problem; how do I solve it?" and further, "how do I solve it elegantly?" The answers to these questions are obtainable by thinking about algorithms and architecture. The ideas come partially through analysis and partially through insight and intuition. The Idea is usually the easy part. You talk to your friends and co-workers and you nut it out in a meeting or over coffee. It takes an hour or two, plus revisions as you implement and find new problems. The Implementation phase of software development is so difficult that we joke about it. "Oh," we say, "the rest is a Simple Matter of Code." Because it should be simple, but it never is. We used to write our code on punch cards, and that was hard: mistakes were very difficult to spot, so we had to spend extra effort making sure every line was perfect. Then we had serial terminals: we could see all our code at once, search through it, organise it hierarchically and create things abstracted from raw machine code. First we had assemblers, one level up from machine code. Mnemonics freed us from remembering the machine code. Then we had compilers, which freed us from remembering the instructions. We had virtual machines, which let us step away from machine-specific details. And now we have advanced tools like Eclipse and Xcode that perform analysis on our code to help us write code faster and avoid common pitfalls. But writing code is still hard. Writing code is about understanding large, complex systems, and tools we have today simply don't go very far to help us with that. When I click "find all references" in Eclipse, I get a list of them at the bottom of the window. I click on one, and I'm torn away from what I was looking at, forced to context switch. Java architecture is usually several levels deep, so I have to switch and switch and switch until I find what I'm really looking for -- by which time I've forgotten where I came from. And I do that all day until I've understood a system. It's taxing mentally, and Eclipse doesn't do much that couldn't be done in 1985 with grep, except eat hundreds of megs of RAM. Writing code has barely changed since we were staring at amber on black. We have the theoretical groundwork for much more advanced tools, tools that actually work to help us comprehend and extend the complex systems we work with every day. So why is writing code still so hard?

  Read the article

• Nested generator functions in python

  - by Yuval A

  Consider a tuple v = (a,b,c) and a generator function generate(x) which receives an item from the tuple and generates several options for each item. What is the pythonic way of generating a set of all the possible combinations of the result of generate(x) on each item in the tuple? I could do this: v = (a,b,c) for d in generate(v[0]): for e in generate(v[1]): for f in generate(v[2]): print d,e,f but that's just ugly, plus I need a generic solution.

  Read the article

• Python beginner confused by a complex line of code

  - by Protean

  I understand the gist of the code, that it forms permutations; however, I was wondering if someone could explain exactly what is going on in the return statement. def perm(l): sz = len(l) print (l) if sz <= 1: print ('sz <= 1') return [l] return [p[:i]+[l[0]]+p[i:] for i in range(sz) for p in perm(l[1:])]

  Read the article

• The truth value of an array with more than one element is ambigous when trying to index an array

  - by user1440194

  I am trying to put all elements of rbs into a new array if the elements in var(another numpy array) is =0 and <=.1 . However when I try the following code I get this error: ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() rbs = [ish[4] for ish in realbooks] for book in realbooks: var -= float(str(book[0]).replace(":", "")) bidsred = rbs[(var <= .1) and (var >=0)] any ideas on what I'm doing wrong?

  Read the article

• filtering elements from list of lists in Python?

  - by user248237

  I want to filter elements from a list of lists, and iterate over the elements of each element using a lambda. For example, given the list: a = [[1,2,3],[4,5,6]] suppose that I want to keep only elements where the sum of the list is greater than N. I tried writing: filter(lambda x, y, z: x + y + z >= N, a) but I get the error: <lambda>() takes exactly 3 arguments (1 given) How can I iterate while assigning values of each element to x, y, and z? Something like zip, but for arbitrarily long lists. thanks, p.s. I know I can write this using: filter(lambda x: sum(x)..., a) but that's not the point, imagine that these were not numbers but arbitrary elements and I wanted to assign their values to variable names.

  Read the article

• Apply function to one element of a list in Python

  - by user189637

  I'm looking for a concise and functional style way to apply a function to one element of a tuple and return the new tuple, in Python. For example, for the following input: inp = ("hello", "my", "friend") I would like to be able to get the following output: out = ("hello", "MY", "friend") I came up with two solutions which I'm not satisfied with. One uses a higher-order function. def apply_at(arr, func, i): return arr[0:i] + [func(arr[i])] + arr[i+1:] apply_at(inp, lambda x: x.upper(), 1) One uses list comprehensions (this one assumes the length of the tuple is known). [(a,b.upper(),c) for a,b,c in [inp]][0] Is there a better way? Thanks!

  Read the article

• python - from matrix to dictionary in single line

  - by Sanich

  matrix is a list of lists. I've to return a dictionary of the form {i:(l1[i],l2[i],...,lm[i])} Where the key i is matched with a tuple the i'th elements from each list. Say matrix=[[1,2,3,4],[9,8,7,6],[4,8,2,6]] so the line: >>> dict([(i,tuple(matrix[k][i] for k in xrange(len(matrix)))) for i in xrange(len(matrix[0]))]) does the job pretty well and outputs: {0: (1, 9, 4), 1: (2, 8, 8), 2: (3, 7, 2), 3: (4, 6, 6)} but fails if the matrix is empty: matrix=[]. The output should be: {} How can i deal with this?

  Read the article

• Python: Split, strip, and join in one line

  - by PandemoniumSyndicate

  I'm curious if their is some python magic I may not know to accomplish a bit of frivolity given the line: csvData.append(','.join([line.split(":").strip() for x in L])) I'm attempting to split a line on :, trim whitespace around it, and join on , problem is, since the array is returned from line.split(":"), the for x in L #<== L doesn't exist! causes issues since I have no name for the array returned by line.split(":") So I'm curious if there is a sexy piece of syntax I could use to accomplish this in one shot? Cheers!

  Read the article

• Paginating requests to an API

  - by user332912

  I'm consuming (via urllib/urllib2) an API that returns XML results. The API always returns the total_hit_count for my query, but only allows me to retrieve results in batches of, say, 100 or 1000. The API stipulates I need to specify a start_pos and end_pos for offsetting this, in order to walk through the results. Say the urllib request looks like "http://someservice?query='test'&start_pos=X&end_pos=Y". If I send an initial 'taster' query with lowest data transfer such as http://someservice?query='test'&start_pos=1&end_pos=1 in order to get back a result of, for conjecture, total_hits = 1234, I'd like to work out an approach to most cleanly request those 1234 results in batches of, again say, 100 or 1000 or... This is what I came up with so far, and it seems to work, but I'd like to know if you would have done things differently or if I could improve upon this: hits_per_page=1000 # or 1000 or 200 or whatever, adjustable total_hits = 1234 # retreived with BSoup from 'taster query' base_url = "http://someservice?query='test'" startdoc_positions = [n for n in range(1, total_hits, hits_per_page)] enddoc_positions = [startdoc_position + hits_per_page - 1 for startdoc_position in startdoc_positions] for start, end in zip(startdoc_positions, enddoc_positions): if end total_hits: end = total_hits print "url to request is:\n ", print "%s&start_pos=%s&end_pos=%s" % (base_url, start, end) p.s. I'm a long time consumer of StackOverflow, especially the Python questions, but this is my first question posted. You guys are just brilliant.

  Read the article

• Generate a sequence of Fibonacci number in Scala

  - by qin

  def fibSeq(n: Int): List[Int] = { var ret = scala.collection.mutable.ListBuffer[Int](1, 2) while (ret(ret.length - 1) < n) { val temp = ret(ret.length - 1) + ret(ret.length - 2) if (temp >= n) { return ret.toList } ret += temp } ret.toList } So the above is my code to generate a Fibonacci sequence using Scala to a value n. I am wondering if there is a more elegant way to do this in Scala?

  Read the article

• Tips for debugging list comprehensions?

  - by andylei

  Python list comprehensions are nice, but near impossible to debug. You guys have any good tips / tools for debugging them?

  Read the article

• Python: concatenate generator and item

  - by TarGz

  I have a generator (numbers) and a value (number). I would like to iterate over these as if they were one sequence: i for i in tuple(my_generator) + (my_value,) The problem is, as far as I undestand, this creates 3 tuples only to immediately discard them and also copies items in "my_generator" once. Better approch would be: def con(seq, item): for i in seq: yield seq yield item i for i in con(my_generator, my_value) But I was wondering whether it is possible to do it without that function definition

  Read the article

• In Python, is it better to use list comprehensions or for-each loops?

  - by froadie

  Which of the following is better to use and why? Method 1: for k, v in os.environ.items() print "%s=%s" % (k, v) Method 2: print "\n".join(["%s=%s" % (k, v) for k,v in os.environ.items()]) I tend to lead towards the first as more understandable, but that might just be because I'm new to Python and list comprehensions are still somewhat foreign to me. Is the second way considered more Pythonic? I'm assuming there's no performance difference, but I may be wrong. What would be the advantages and disadvantages of these 2 techniques? (Code taken from Dive into Python)

  Read the article

• finding elements in python association lists efficiently

  - by user248237

  I have a set of lists that look like this: conditions = [ ["condition1", ["sample1", "sample2", "sample3"]], ["condition2", ["sample4", "sample5", "sample6"], ...] how can I do the following things efficiently and elegantly in Python? Find all the elements in a certain condition? e.g. get all the samples in condition2. Right now I can do: for cond in conditions: cond_name, samples = cond if cond_name == requested_cond: return samples but that's clunky. Find the ordered union of a list of conditions? E.g. ordered_union(["condition1", "condition2"], conditions) should return: ["sample1", "sample2", "sample3", "sample4", "sample5", "sample6"] How can I do this efficiently in Python? There are probably clever one liners?

  Read the article

• Is it possible to do a web search filtered for the user's comprehension level? [migrated]

  - by Tim

  Do any of the major search engines support a filter to indicate the comprehension level of the user and thus to filter out content beyond their scope? So for example, if someone wanted to find results on "astronomy", but to only show content appropriate for someone in junior-high or lower. Alternatively, if there isn't such an existing method directly, any ideas on how this might be accomplished with existing tools? I found this paper, which seems to indicate folks have looked at this, just don't know if there is a real world tool to do this.

  Read the article

• How to use Python list comprehension (or such) for retrieving rows when using MySQLdb?

  - by Erik Nygren

  Hey all, I use MySQLdb a lot when dealing with my webserver. I often find myself repeating the lines: row = cursor.fetchone() while row: do_processing(row) row = cursor.fetchone() Somehow this strikes me as somewhat un-pythonic. Is there a better, one-line way to accomplish the same thing, along the lines of inline assignment in C: while (row = do_fetch()) { do_processing(row); } I've tried figuring out the syntax using list comprehensions, but I can't seem to figure it out. Any recommendations? Thanks, Erik

  Read the article

• speed of map() vs. list comprehension vs. numpy vectorized function in python

  - by mcstrother

  I have a function foo(i) that takes an integer and takes a significant amount of time to execute. Will there be a significant performance difference between any of the following ways of initializing 'a': a = [foo(i) for i in xrange(100)] , a = map(foo, range(100)) , and vfoo = numpy.vectorize(foo) vfoo(range(100)) ? (I don't care whether the output is a list or a numpy array). Is there some other better way of doing this? Thanks.

  Read the article

• Can I do a reduce on a list comprehension into two lists, based on two values?

  - by pdknsk

  I've got the following code. sum_review = reduce(add,[book['rw'] for book in books]) sum_rating = reduce(add,[book['rg'] for book in books]) items = len(books) avg_review = sum_review/items avg_rating = sum_rating/items What I'd like is this. sum_review,sum_rating = reduce(add,([book['rw'],[book['rg']) for book in books]) items = len(books) avg_review = sum_review/items avg_rating = sum_rating/items Obviously this doesn't work. How can I solve this redundancy, without a regular loop?

  Read the article

• How do I return a String from a for comprehension in Scala?

  - by Vonn

  Scala Newbie alert: basically I'm trying to do something like this: where I pattern match and return a String. scala> def processList(list: List[String], m: String): String={list foreach (x=> m match{ | case "test" => "we got test" | case "test1"=> "we got test1"})} :10: error: type mismatch; found : Unit required: String def processList(list: List[String], m: String): String={list foreach (x= m match{ I know I can set a var and return it after the for comp... but that doesn't seem to be the Scala way.

  Read the article

• Are short identifiers bad?

  - by Daniel C. Sobral

  Are short identifiers bad? How does identifier length correlate with code comprehension? What other factors (besides code comprehension) might be of consideration when it comes to naming identifiers? Just to try to keep the quality of the answers up, please note that there is some research on the subject already! Edit Curious that everyone either doesn't think length is relevant or tend to prefer larger identifiers, when both links I provided indicate large identifiers are harmful!

  Read the article

• Language Design: Are languages like phyton and coffescript really more comprehendable?

  - by kittensatplay

  the "Verbally Readable !== Quicker Comprehension" arguement on http://ryanflorence.com/2011/case-against-coffeescript/ is really potent and interesting. i and im sure other would be very interested in evidence arguing against this. there's clear evidence for this and i believe it. ppl naturally think in images, not words, so we should be designing languages dissimilar to human language like english, french, whatever. being "readable" is quicker comprehension. most articles on wikipedia are not readable as they are long, boring, dry, sluggish, very very wordy, and because wikipedia documents a ton of info, is not especially helpful when compared to much more helpful sites with more practical, useful, and relevant info. but languages like phyton and coffescript are "verbally readable" in that they are closer to the english language syntax, and programming firstly and mainly in python, im not so sure this is really a good thing. the second interesting argument is that coffeescript is an intermediator so thereby another step between to ends, which may increase chances of bugs. while coffeescript has other practical benefits, this question is focused specifically on evidence showing support for the counter-case of language "readability"

  Read the article

• Find an element in a JavaScript array

  - by Aligned

  Originally posted on: http://geekswithblogs.net/Aligned/archive/2014/08/22/find-an-element-in-a-javascript-array.aspxI needed a C# Dictionary like data structure in JavaScript and then a way to find that object by a key. I had forgotten how to do this, so did some searching and talked to a colleague and came up with this JsFiddle. See the code in my jsFiddle or below: var processingProgressTimeoutIds = []; var file = { name: 'test', timeId: 1 }; var file2 = { name: 'test2', timeId: 2 }; var file3 = { name: 'test3', timeId: 3 }; processingProgressTimeoutIds.push({ name: file.name, timerId: file.id }); processingProgressTimeoutIds.push({ name: file2.name, timerId: file2.id }); processingProgressTimeoutIds.push({ name: file3.name, timerId: file3.id }); console.log(JSON.stringify(processingProgressTimeoutIds)); var keyName = 'test'; var match = processingProgressTimeoutIds.filter(function (item) { return item.name === keyName; })[0]; console.log(JSON.stringify(match)); // optimization var match2 = processingProgressTimeoutIds.some(function (element, index, array) { return element.name === keyName; }); console.log(JSON.stringify(match2)); // if you have the full object var match3 = processingProgressTimeoutIds.indexOf(file); console.log(JSON.stringify(match3)); // http://jsperf.com/array-find-equal – from Dave // indexOf is faster, but I need to find it by the key, so I can’t use it here //ES6 will rock though, array comprehension! – also from Dave // var ys = [x of xs if x == 3]; // var y = ys[0]; Here’s a good blog post on Array comprehension.

  Read the article

• Language Design: Are languages like Python and CoffeeScript really more comprehensible?

  - by kittensatplay

  The "Verbally Readable !== Quicker Comprehension" argument on http://ryanflorence.com/2011/case-against-coffeescript/ is really potent and interesting. I and I'm sure others would be very interested in evidence arguing against this. There's clear evidence for this and I believe it. People naturally think in images, not words, so we should be designing languages that aren't similar to human language like English, French, whatever. Being "readable" is quicker comprehension. Most articles on Wikipedia are not readable as they are long, boring, dry, sluggish and very very wordy. Because Wikipedia documents a ton of info, it is not especially helpful when compared to sites with more practical, useful and relevant info. Languages like Python and CoffeScript are "verbally readable" in that they are closer to English syntax. Having programmed firstly and mainly in Python, I'm not so sure this is really a good thing. The second interesting argument is that CoffeeScript is an intermediator, a step between two ends, which may increase the chance of bugs. While CoffeeScript has other practical benefits, this question specifically requests evidence showing support for the counter-case of language "readability"

  Read the article

• HTML5 et CSS3/WebGL - Coffret de 2 livres, critiqués par Julien Plu

  Je viens de terminer ce coffret de 2 livres, l'un sur HTML5 et CSS3 et l'autre sur WebGL qui étaient tout deux très bon. Voici d'ailleurs ma critique pour ceux que ça intéresserait d'en savoir plus à propos de ce coffret. Ce livre sur le HTML5 et CSS3 s'adresse à toute personne appelée à développer, mettre en place, faire vivre un site Web. En effet, pour débuter mais surtout pour progresser dans la conception de sites, il faut inévitablement passer par une bonne compréhension et une...

  Read the article

< Previous Page | 1 2 3 4 5 6  | Next Page >