Search Results

Search found 33869 results on 1355 pages for 'python install'.

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

Page 168/1355 | < Previous Page | 164 165 166 167 168 169 170 171 172 173 174 175  | Next Page >

  • Python: Created nested dictionary from list of paths

    - by sberry2A
    I have a list of tuples the looks similar to this (simplified here, there are over 14,000 of these tuples with more complicated paths than Obj.part) [ (Obj1.part1, {<SPEC>}), (Obj1.partN, {<SPEC>}), (ObjK.partN, {<SPEC>}) ] Where Obj goes from 1 - 1000, part from 0 - 2000. These "keys" all have a dictionary of specs associated with them which act as a lookup reference for inspecting another binary file. The specs dict contains information such as the bit offset, bit size, and C type of the data pointed to by the path ObjK.partN. For example: Obj4.part500 might have this spec, {'size':32, 'offset':128, 'type':'int'} which would let me know that to access Obj4.part500 in the binary file I must unpack 32 bits from offset 128. So, now I want to take my list of strings and create a nested dictionary which in the simplified case will look like this data = { 'Obj1' : {'part1':{spec}, 'partN':{spec} }, 'ObjK' : {'part1':{spec}, 'partN':{spec} } } To do this I am currently doing two things, 1. I am using a dotdict class to be able to use dot notation for dictionary get / set. That class looks like this: class dotdict(dict): def __getattr__(self, attr): return self.get(attr, None) __setattr__ = dict.__setitem__ __delattr__ = dict.__delitem__ The method for creating the nested "dotdict"s looks like this: def addPath(self, spec, parts, base): if len(parts) > 1: item = base.setdefault(parts[0], dotdict()) self.addPath(spec, parts[1:], item) else: item = base.setdefault(parts[0], spec) return base Then I just do something like: for path, spec in paths: self.lookup = dotdict() self.addPath(spec, path.split("."), self.lookup) So, in the end self.lookup.Obj4.part500 points to the spec. Is there a better (more pythonic) way to do this?

    Read the article

  • python fdb save huge data from database to file

    - by peter
    I have this script SELECT = """ select coalesce (p.ID,'') as id, coalesce (p.name,'') as name, from TABLE as p """ self.cur.execute(SELECT) for row in self.cur.itermap(): xml +=" <item>\n" xml +=" <id>" + id + "</id>\n" xml +=" <name>" + name + "</name>\n" xml +=" </item>\n\n" #save xml to file here f = open... and I need to save data from huge database to file. There are 10 000s (up to 40000) of items in my database and it takes very long time when script runs (1 hour and more) until finish. How can I take data I need from database and save it to file "at once"? (as quick as possible? I don't need xml output because I can process data from output on my server later. I just need to do it as quickly as possible. Any idea?) Many thanks!

    Read the article

  • Optimizing python code performance when importing zipped csv to a mongo collection

    - by mark
    I need to import a zipped csv into a mongo collection, but there is a catch - every record contains a timestamp in Pacific Time, which must be converted to the local time corresponding to the (longitude,latitude) pair found in the same record. The code looks like so: def read_csv_zip(path, timezones): with ZipFile(path) as z, z.open(z.namelist()[0]) as input: csv_rows = csv.reader(input) header = csv_rows.next() check,converters = get_aux_stuff(header) for csv_row in csv_rows: if check(csv_row): row = { converter[0]:converter[1](value) for converter, value in zip(converters, csv_row) if allow_field(converter) } ts = row['ts'] lng, lat = row['loc'] found_tz_entry = timezones.find_one(SON({'loc': {'$within': {'$box': [[lng-tz_lookup_radius, lat-tz_lookup_radius],[lng+tz_lookup_radius, lat+tz_lookup_radius]]}}})) if found_tz_entry: tz_name = found_tz_entry['tz'] local_ts = ts.astimezone(timezone(tz_name)).replace(tzinfo=None) row['tz'] = tz_name else: local_ts = (ts.astimezone(utc) + timedelta(hours = int(lng/15))).replace(tzinfo = None) row['local_ts'] = local_ts yield row def insert_documents(collection, source, batch_size): while True: items = list(itertools.islice(source, batch_size)) if len(items) == 0: break; try: collection.insert(items) except: for item in items: try: collection.insert(item) except Exception as exc: print("Failed to insert record {0} - {1}".format(item['_id'], exc)) def main(zip_path): with Connection() as connection: data = connection.mydb.data timezones = connection.timezones.data insert_documents(data, read_csv_zip(zip_path, timezones), 1000) The code proceeds as follows: Every record read from the csv is checked and converted to a dictionary, where some fields may be skipped, some titles be renamed (from those appearing in the csv header), some values may be converted (to datetime, to integers, to floats. etc ...) For each record read from the csv, a lookup is made into the timezones collection to map the record location to the respective time zone. If the mapping is successful - that timezone is used to convert the record timestamp (pacific time) to the respective local timestamp. If no mapping is found - a rough approximation is calculated. The timezones collection is appropriately indexed, of course - calling explain() confirms it. The process is slow. Naturally, having to query the timezones collection for every record kills the performance. I am looking for advises on how to improve it. Thanks. EDIT The timezones collection contains 8176040 records, each containing four values: > db.data.findOne() { "_id" : 3038814, "loc" : [ 1.48333, 42.5 ], "tz" : "Europe/Andorra" } EDIT2 OK, I have compiled a release build of http://toblerity.github.com/rtree/ and configured the rtree package. Then I have created an rtree dat/idx pair of files corresponding to my timezones collection. So, instead of calling collection.find_one I call index.intersection. Surprisingly, not only there is no improvement, but it works even more slowly now! May be rtree could be fine tuned to load the entire dat/idx pair into RAM (704M), but I do not know how to do it. Until then, it is not an alternative. In general, I think the solution should involve parallelization of the task. EDIT3 Profile output when using collection.find_one: >>> p.sort_stats('cumulative').print_stats(10) Tue Apr 10 14:28:39 2012 ImportDataIntoMongo.profile 64549590 function calls (64549180 primitive calls) in 1231.257 seconds Ordered by: cumulative time List reduced from 730 to 10 due to restriction <10> ncalls tottime percall cumtime percall filename:lineno(function) 1 0.012 0.012 1231.257 1231.257 ImportDataIntoMongo.py:1(<module>) 1 0.001 0.001 1230.959 1230.959 ImportDataIntoMongo.py:187(main) 1 853.558 853.558 853.558 853.558 {raw_input} 1 0.598 0.598 370.510 370.510 ImportDataIntoMongo.py:165(insert_documents) 343407 9.965 0.000 359.034 0.001 ImportDataIntoMongo.py:137(read_csv_zip) 343408 2.927 0.000 287.035 0.001 c:\python27\lib\site-packages\pymongo\collection.py:489(find_one) 343408 1.842 0.000 274.803 0.001 c:\python27\lib\site-packages\pymongo\cursor.py:699(next) 343408 2.542 0.000 271.212 0.001 c:\python27\lib\site-packages\pymongo\cursor.py:644(_refresh) 343408 4.512 0.000 253.673 0.001 c:\python27\lib\site-packages\pymongo\cursor.py:605(__send_message) 343408 0.971 0.000 242.078 0.001 c:\python27\lib\site-packages\pymongo\connection.py:871(_send_message_with_response) Profile output when using index.intersection: >>> p.sort_stats('cumulative').print_stats(10) Wed Apr 11 16:21:31 2012 ImportDataIntoMongo.profile 41542960 function calls (41542536 primitive calls) in 2889.164 seconds Ordered by: cumulative time List reduced from 778 to 10 due to restriction <10> ncalls tottime percall cumtime percall filename:lineno(function) 1 0.028 0.028 2889.164 2889.164 ImportDataIntoMongo.py:1(<module>) 1 0.017 0.017 2888.679 2888.679 ImportDataIntoMongo.py:202(main) 1 2365.526 2365.526 2365.526 2365.526 {raw_input} 1 0.766 0.766 502.817 502.817 ImportDataIntoMongo.py:180(insert_documents) 343407 9.147 0.000 491.433 0.001 ImportDataIntoMongo.py:152(read_csv_zip) 343406 0.571 0.000 391.394 0.001 c:\python27\lib\site-packages\rtree-0.7.0-py2.7.egg\rtree\index.py:384(intersection) 343406 379.957 0.001 390.824 0.001 c:\python27\lib\site-packages\rtree-0.7.0-py2.7.egg\rtree\index.py:435(_intersection_obj) 686513 22.616 0.000 38.705 0.000 c:\python27\lib\site-packages\rtree-0.7.0-py2.7.egg\rtree\index.py:451(_get_objects) 343406 6.134 0.000 33.326 0.000 ImportDataIntoMongo.py:162(<dictcomp>) 346 0.396 0.001 30.665 0.089 c:\python27\lib\site-packages\pymongo\collection.py:240(insert) EDIT4 I have parallelized the code, but the results are still not very encouraging. I am convinced it could be done better. See my own answer to this question for details.

    Read the article

  • Replacing python docstrings

    - by tomaz
    I have written a epytext to reST markup converter, and now I want to convert all the docstrings in my entire library from epytext to reST format. Is there a smart way to read the all the docstrings in a module and write back the replacements? ps: ast module perhaps?

    Read the article

  • Python problem with resize animate GIF

    - by gigimon
    Hello! I'm want to resize animated GIF with save animate. I'm try use PIL and PythonMagickWand (ImageMagick) and with some GIF's get bad frame. When I'm use PIL, it mar frame in read frame. For test, I'm use this code: from PIL import Image im = Image.open('d:/box_opens_closes.gif') im.seek(im.tell()+1) im.seek(im.tell()+1) im.seek(im.tell()+1) im.show() When I'm use MagickWand with this code: wand = NewMagickWand() MagickReadImage(wand, 'd:/Box_opens_closes.gif') MagickSetLastIterator(wand) length = MagickGetIteratorIndex(wand) MagickSetFirstIterator(wand) for i in range(0, length+1): MagickSetIteratorIndex(wand,i) MagickScaleImage(wand, 87, 58) MagickWriteImages(wand, 'path', 1) My GIF where I'm get bad frame this: test gif In GIF editor software, all freme is ok. Where problem? Thx

    Read the article

  • Python: Count lines and differentiate between them

    - by Mister X
    I'm using an application that gives a timed output based on how many times something is done in a minute, and I wish to manually take the output (copy paste) and have my program, and I wish to count how many times each minute it is done. An example output is this: 13:48 An event happened. 13:48 Another event happened. 13:49 A new event happened. 13:49 A random event happened. 13:49 An event happened. So, the program would need to understand that 2 things happened at 13:48, and 3 at 13:49. I'm not sure how the information would be stored, but I need to average them after, to determine an average of how often it happens. Sorry for being so complicated!

    Read the article

  • I want to design a html form in python

    - by VaIbHaV-JaIn
    when user will enter details in the text box on the html from <h1>Please enter new password</h1> <form method="POST" enctype="application/json action="uid"> Password<input name="passwd"type="password" /><br> Retype Password<input name="repasswd" type="password" /><br> <input type="Submit" /> </form> </body> i want to post the data in json format through http post request and also i want to set content-type = application/json

    Read the article

  • how can i randomly print an element from a list in python

    - by lm
    So far i have this, which prints out every word in my list, but i am trying to print only one word at random. Any suggestions? def main(): # open a file wordsf = open('words.txt', 'r') word=random.choice('wordsf') words_count=0 for line in wordsf: word= line.rstrip('\n') print(word) words_count+=1 # close the file wordsf.close()

    Read the article

  • [Tkinter/Python] Different line widths with canvas.create_line?

    - by Sam
    Does anyone have any idea why I get different line widths on the canvas in the following example? from Tkinter import * bigBoxSize = 150 class cFrame(Frame): def __init__(self, master, cwidth=450, cheight=450): Frame.__init__(self, master, relief=RAISED, height=550, width=600, bg = "grey") self.canvasWidth = cwidth self.canvasHeight = cheight self.canvas = Canvas(self, bg="white", width=cwidth, height=cheight, border =0) self.drawGridLines() self.canvas.pack(side=TOP, pady=20, padx=20) def drawGridLines(self, linewidth = 10): self.canvas.create_line(0, 0, self.canvasWidth, 0, width= linewidth ) self.canvas.create_line(0, 0, 0, self.canvasHeight, width= linewidth ) self.canvas.create_line(0, self.canvasHeight, self.canvasWidth + 2, self.canvasHeight, width= linewidth ) self.canvas.create_line(self.canvasWidth, self.canvasHeight, self.canvasWidth, 1, width= linewidth ) self.canvas.create_line(0, bigBoxSize, self.canvasWidth, bigBoxSize, width= linewidth ) self.canvas.create_line(0, bigBoxSize * 2, self.canvasWidth, bigBoxSize * 2, width= linewidth) root = Tk() C = cFrame(root) C.pack() root.mainloop() It's really frustrating me as I have no idea what's happening. If anyone can help me out then that'd be fantastic. Thanks!

    Read the article

  • Exporting dates properly formatted on Google Appengine in Python

    - by Chris M
    I think this is right but google appengine seems to get to a certain point and cop-out; Firstly is this code actually right; and secondly is there away to skip the record if it cant output (like an ignore errors and continue)? class TrackerExporter(bulkloader.Exporter): def __init__(self): bulkloader.Exporter.__init__(self, 'SearchRec', [('__key__', lambda key:key.name(), None), ('WebSite', str, None), ('DateStamp', lambda x: datetime.datetime.strptime(x, '%d-%m-%Y').date(), None), ('IP', str, None), ('UserAgent', str, None)]) Thanks

    Read the article

  • Get the last '/' or '\\' character in Python

    - by wowus
    If I have a string that looks like either ./A/B/c.d OR .\A\B\c.d How do I get just the "./A/B/" part? The direction of the slashes can be the same as they are passed. This problem kinda boils down to: How do I get the last of a specific character in a string? Basically, I want the path of a file without the file part of it.

    Read the article

  • Python unicode search not giving correct answer

    - by user1318912
    I am trying to search hindi words contained one line per file in file-1 and find them in lines in file-2. I have to print the line numbers with the number of words found. This is the code: import codecs hypernyms = codecs.open("hindi_hypernym.txt", "r", "utf-8").readlines() words = codecs.open("hypernyms_en2hi.txt", "r", "utf-8").readlines() count_arr = [] for counter, line in enumerate(hypernyms): count_arr.append(0) for word in words: if line.find(word) >=0: count_arr[counter] +=1 for iterator, count in enumerate(count_arr): if count>0: print iterator, ' ', count This is finding some words, but ignoring some others The input files are: File-1: ???? ??????? File-2: ???????, ????-???? ?????-???, ?????-???, ?????_???, ?????_??? ????_????, ????-????, ???????_???? ????-???? This gives output: 0 1 3 1 Clearly, it is ignoring ??????? and searching for ???? only. I have tried with other inputs as well. It only searches for one word. Any idea how to correct this?

    Read the article

  • filtering elements from list of lists in Python?

    - by user248237
    I want to filter elements from a list of lists, and iterate over the elements of each element using a lambda. For example, given the list: a = [[1,2,3],[4,5,6]] suppose that I want to keep only elements where the sum of the list is greater than N. I tried writing: filter(lambda x, y, z: x + y + z >= N, a) but I get the error: <lambda>() takes exactly 3 arguments (1 given) How can I iterate while assigning values of each element to x, y, and z? Something like zip, but for arbitrarily long lists. thanks, p.s. I know I can write this using: filter(lambda x: sum(x)..., a) but that's not the point, imagine that these were not numbers but arbitrary elements and I wanted to assign their values to variable names.

    Read the article

  • Regex replace (in Python) - a more simple way?

    - by Evan Fosmark
    Any time I want to replace a piece of text that is part of a larger piece of text, I always have to do something like: "(?P<start>some_pattern)(?P<replace>foo)(?P<end>end)" And then concatenate the start group with the new data for replace and then the end group. Is there a better method for this?

    Read the article

  • Python recursion with list returns None

    - by newman
    def foo(a): a.append(1) if len(a) > 10: print a return a else: foo(a) Why this recursive function returns None (see transcript below)? I can't quite understand what I am doing wrong. In [263]: x = [] In [264]: y = foo(x) [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] In [265]: print y None

    Read the article

  • Python - calendar.timegm() vs. time.mktime()

    - by ibz
    I seem to have a hard time getting my head around this. What's the difference between calendar.timegm() and time.mktime()? Say I have a datetime.datetime with no tzinfo attached, shouldn't the two give the same output? Don't they both give the number of seconds between epoch and the date passed as a parameter? And since the date passed has no tzinfo, isn't that number of seconds the same? >>> import calendar >>> import time >>> import datetime >>> d = datetime.datetime(2010, 10, 10) >>> calendar.timegm(d.timetuple()) 1286668800 >>> time.mktime(d.timetuple()) 1286640000.0 >>>

    Read the article

  • Efficient way in Python to remove an element from a comma-separated string

    - by ensnare
    I'm looking for the most efficient way to add an element to a comma-separated string while maintaining alphabetical order for the words: For example: string = 'Apples, Bananas, Grapes, Oranges' subtraction = 'Bananas' result = 'Apples, Grapes, Oranges' Also, a way to do this but while maintaining IDs: string = '1:Apples, 4:Bananas, 6:Grapes, 23:Oranges' subtraction = '4:Bananas' result = '1:Apples, 6:Grapes, 23:Oranges' Sample code is greatly appreciated. Thank you so much.

    Read the article

  • compute mean in python for a generator

    - by nmaxwell
    Hi, I'm doing some statistics work, I have a (large) collection of random numbers to compute the mean of, I'd like to work with generators, because I just need to compute the mean, so I don't need to store the numbers. The problem is that numpy.mean breaks if you pass it a generator. I can write a simple function to do what I want, but I'm wondering if there's a proper, built-in way to do this? It would be nice if I could say "sum(values)/len(values)", but len doesn't work for genetators, and sum already consumed values. here's an example: import numpy def my_mean(values): n = 0 Sum = 0.0 try: while True: Sum += next(values) n += 1 except StopIteration: pass return float(Sum)/n X = [k for k in range(1,7)] Y = (k for k in range(1,7)) print numpy.mean(X) print my_mean(Y) these both give the same, correct, answer, buy my_mean doesn't work for lists, and numpy.mean doesn't work for generators. I really like the idea of working with generators, but details like this seem to spoil things. thanks for any help -nick

    Read the article

  • I have an Errno 13 Permission denied with subprocess in python

    - by wDroter
    The line with the issue is ret=subprocess.call(shlex.split(cmd)) cmd = /usr/share/java -cp pig-hadoop-conf-Simpsons:lib/pig-0.8.1-cdh3u1-core.jar:lib/hadoop-core-0.20.2-cdh3u1.jar org.apache.pig.Main -param func=cat -param from =foo.txt -x mapreduce fsFunc.pig The error is. File "./run_pig.py", line 157, in process ret=subprocess.call(shlex.split(cmd)) File "/usr/lib/python2.7/subprocess.py", line 493, in call return Popen(*popenargs, **kwargs).wait() File "/usr/lib/python2.7/subprocess.py", line 679, in __init__ errread, errwrite) File "/usr/lib/python2.7/subprocess.py", line 1249, in _execute_child raise child_exception OSError: [Errno 13] Permission denied Let me know if any more info is needed. Any help is appreciated. Thanks.

    Read the article

  • OpenMeetings + Python + Suds

    - by user366774
    Trying to integrate openmeetings with django website, but can't understand how properly configure ImportDoctor: (here :// replaced with __ 'cause spam protection) print url http://sovershenstvo.com.ua:5080/openmeetings/services/UserService?wsdl imp = Import('http__schemas.xmlsoap.org/soap/encoding/') imp.filter.add('http__services.axis.openmeetings.org') imp.filter.add('http__basic.beans.hibernate.app.openmeetings.org/xsd') imp.filter.add('http__basic.beans.data.app.openmeetings.org/xsd') imp.filter.add('http__services.axis.openmeetings.org') d = ImportDoctor(imp) client = Client(url, doctor = d) client.service.getSession() Traceback (most recent call last): File "", line 1, in File "/usr/lib/python2.6/site-packages/suds/client.py", line 539, in call return client.invoke(args, kwargs) File "/usr/lib/python2.6/site-packages/suds/client.py", line 598, in invoke result = self.send(msg) File "/usr/lib/python2.6/site-packages/suds/client.py", line 627, in send result = self.succeeded(binding, reply.message) File "/usr/lib/python2.6/site-packages/suds/client.py", line 659, in succeeded r, p = binding.get_reply(self.method, reply) File "/usr/lib/python2.6/site-packages/suds/bindings/binding.py", line 159, in get_reply resolved = rtypes[0].resolve(nobuiltin=True) File "/usr/lib/python2.6/site-packages/suds/xsd/sxbasic.py", line 63, in resolve raise TypeNotFound(qref) suds.TypeNotFound: Type not found: '(Sessiondata, http__basic.beans.hibernate.app.openmeetings.org/xsd, )' what i'm doing wrong? please help and sorry for my english, but you are my last chance to save position :( need webinars at morning (2.26 am now)

    Read the article

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

< Previous Page | 164 165 166 167 168 169 170 171 172 173 174 175  | Next Page >