Search Results

Search found 15224 results on 609 pages for 'parallel python'.

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

Page 18/609 | < Previous Page | 14 15 16 17 18 19 20 21 22 23 24 25  | Next Page >

  • Python - CSV: Large file with rows of different lengths

    - by dassouki
    In short, I have a 20,000,000 line csv file that has different row lengths. This is due to archaic data loggers and proprietary formats. We get the end result as a csv file in the following format. MY goal is to insert this file into a postgres database. How Can I do the following: Keep the first 8 columns and my last 2 columns, to have a consistent CSV file Add a Column to the file. 1, 2, 3, 4, 5, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, img_id.jpg, -50 1, 2, 3, 4, 5, 0,0,0,0,0,0,0,0,0, img_id.jpg, -50 1, 2, 3, 4, 5, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, img_id.jpg, -50 1, 2, 3, 4, 5, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, img_id.jpg, -50 1, 2, 3, 4, 5, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, img_id.jpg, -50 1, 2, 3, 4, 5, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, img_id.jpg, -50 1, 2, 3, 4, 5, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, img_id.jpg, -50 1, 2, 3, 4, 5, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, img_id.jpg, -50 1, 2, 3, 4, 5, 0,0,0,0,0,0, img_id.jpg, -50 1, 2, 3, 4, 5, 0,0,0,0,0,0,0,0,0,0,0 img_id.jpg, -50

    Read the article

  • Using httplib2 in python 3 properly? (Timeout problems)

    - by Sho Minamimoto
    Hey, first time post, I'm really stuck on httplib2. I've been reading up on it from diveintopython3.org, but it mentions nothing about a timeout function. I look up the documentation, but the only thing I see is an ability to put a timeout int but there are no units specified (seconds? milliseconds? What's the default if None?) This is what I have (I also have code to check what the response is and try again, but it's never tried more than once) h = httplib2.Http('.cache', timeout=None) for url in list: response, content = h.request(url) more stuff... So the Http object stays around until some arbitrary time, but I'm downloading a ton of pages from the same server, and after a while, it hangs on getting a page. No errors are thrown, the thing just hangs at a page. So then I try: h = httplib2.Http('.cache', timeout=None) for url in list: try: response, content = h.request(url) except: h = httplib2.Http('.cache', timeout=None) more stuff... But then it recreates another Http object every time (goes down the 'except' path)...I dont understand how to keep getting with the same object, until it expires and I make another. Also, is there a way to set a timeout on an individual request? Thanks for the help!

    Read the article

  • Generating permutation in Python with specific rule

    - by twfx
    Let say a=[A, B, C, D], each element has a weight w, and is set to 1 if selected, 0 if otherwise. I'd like to generate permutation in the below order 1,1,1,1 1,1,1,0 1,1,0,1 1,1,0,0 1,0,1,1 1,0,1,0 1,0,0,1 1,0,0,0 Let's w=[1,2,3,4] for item A,B,C,D ... and max_weight = 4. For each permutation, if the accum weight has exceeded max_weight, stop calculation for that permutation, move to next permutation. For eg. 1,1,1 --> 6 > 4, exceeded, stop, move to next 1,1,1 --> 6 > 4, exceeded, stop, move to next 1,1,0,1 --> 7 > 4 finished, move to next 1,1,0,0 --> 3 finished, move to next 1,0,1,1 --> 8 > 4, finished, stop, move to next 1,0,1,0 --> 4 finished, move to next 1,0,0,1 --> 5 > 4 finished, move to next 1,0,0,0 --> 1 finished, move to next [1,0,1,0] is the best combination which does not exceeded max_weight 4 My questions are What's the algorithm which generate the required permutation? Or any suggestion I could generate the permutation? As the number of element can be up to 10000, and the calculation stop if the accum weight for the branch exceeds max_weight, it is not necessary to generate all permutation first before the calculation. How can the algo in (1) generate permutation on the fly?

    Read the article

  • How is the 'is' keyword implemented in Python?

    - by Srikanth
    ... the is keyword that can be used for equality in strings. >>> s = 'str' >>> s is 'str' True >>> s is 'st' False I tried both __is__() and __eq__() but they didn't work. >>> class MyString: ... def __init__(self): ... self.s = 'string' ... def __is__(self, s): ... return self.s == s ... >>> >>> >>> m = MyString() >>> m is 'ss' False >>> m is 'string' # <--- Expected to work False >>> >>> class MyString: ... def __init__(self): ... self.s = 'string' ... def __eq__(self, s): ... return self.s == s ... >>> >>> m = MyString() >>> m is 'ss' False >>> m is 'string' # <--- Expected to work, but again failed False >>> Thanks for your help!

    Read the article

  • How to build sqlite for Python 2.4?

    - by Verrtex
    I would like to use pysqlite interface between Python and sdlite database. I have already Python and SQLite on my computer. But I have troubles with installation of pysqlite. During the installation I get the following error message: error: command 'gcc' failed with exit status 1 As far as I understood the problems appears because version of my Python is 2.4.3 and SQLite is integrated in Python since 2.5. However, I also found out that it IS possible to build sqlite for Python 2.4 (using some tricks, probably). Does anybody know how to build sqlite for Python 2.4? As another option I could try to install higher version of Python. However I do not have root privileges. Does anybody know what will be the easiest way to solve the problem (build SQLite fro Python 2.4, or install newer version of Python)? I have to mention that I would not like to overwrite the old version version of Python. Thank you in advance.

    Read the article

  • Python try...except comma vs 'as' in except

    - by peter
    What is the difference between ',' and 'as' in except statements, eg: try: pass except Exception, exception: pass and: try: pass except Exception as exception: pass Is the second syntax legal in 2.6? It works in CPython 2.6 on Windows but the 2.5 interpreter in cygwin complains that it is invalid. If they are both valid in 2.6 which should I use?

    Read the article

  • importing files in python

    - by Yosy
    I have that file structure- Blog\DataObjects\User.py Blog\index.py I want to import the function(say_hello) at User.py from index.py. I am trying this code - from Blog.DataObjects.User import say_hello say_hello() And I have that error - Traceback (most recent call last): File "index.py", line 1, in <module> from Blog.DataObjects import User ImportError: No module named Blog.DataObjects

    Read the article

  • python __getattr__ help

    - by Stefanos Tux Zacharakis
    Reading a Book, i came across this code... # module person.py class Person: def __init__(self, name, job=None, pay=0): self.name = name self.job = job self.pay = pay def lastName(self): return self.name.split()[-1] def giveRaise(self, percent): self.pay = int(self.pay *(1 + percent)) def __str__(self): return "[Person: %s, %s]" % (self.name,self.pay) class Manager(): def __init__(self, name, pay): self.person = Person(name, "mgr", pay) def giveRaise(self, percent, bonus=.10): self.person.giveRaise(percent + bonus) def __getattr__(self, attr): return getattr(self.person, attr) def __str__(self): return str(self.person) It does what I want it to do, but i do not understand the __getattr__ function in the Manager class. I know that it Delegates all other attributes from Person class. but I do not understand the way it works. for example why from Person class? as I do not explicitly tell it to. person(module is different than Person(class) Any help is highly appreciated :)

    Read the article

  • Python with PIL and Libjpeg on Leopard

    - by thescreamingdrills
    I'm having trouble getting pictures supported with PIL - it throws me this: "IOError: decoder jpeg not available" I installed PIL from binary, not realizing I needed libjpeg. I installed libjpeg and freetype2 through fink. I tried to reinstall PIL using instructions from http://timhatch.com/ (bottom of the page) "* Download PIL 1.1.6 source package and have the Developer Tools already installed * Patch setup.py with this patch so it can find the Freetype you already have. (patch -p0 < leopard_freetype2.diff) * sudo apt-get install libjpeg if you have fink (otherwise, build by hand and adjust paths)" But I'm still getting the same error. I'm on Leopard PPC.

    Read the article

  • insert multiple elements in string in python

    - by Anurag Sharma
    I have to build a string like this { name: "john", url: "www.dkd.com", email: "[email protected]" } where john, www.dkd.com and [email protected] are to be supplied by variables I tried to do the following s1 = "{'name:' {0},'url:' {1},'emailid:' {2}}" s1.format("john","www.dkd.com","[email protected]") I am getting the following error Traceback (most recent call last): File "<stdin>", line 1, in <module> KeyError: "'name" Dont able to understand what I am doing wrong

    Read the article

  • importing files at python

    - by Yosy
    I have that file strudctue- Blog\DataObjects\User.py Blog\index.py I want to import the function(say_hello) at User.py from index.py. I am trying this code - from Blog.DataObjects.User import say_hello say_hello() And I have that error - Traceback (most recent call last): File "index.py", line 1, in from Blog.DataObjects import User ImportError: No module named Blog.DataObjects

    Read the article

  • Python Regular Expressions: Capture lookahead value (capturing text without consuming it)

    - by Lattyware
    I wish to use regular expressions to split words into groups of (vowels, not_vowels, more_vowels), using a marker to ensure every word begins and ends with a vowel. import re MARKER = "~" VOWELS = {"a", "e", "i", "o", "u", MARKER} word = "dog" if word[0] not in VOWELS: word = MARKER+word if word[-1] not in VOWELS: word += MARKER re.findall("([%]+)([^%]+)([%]+)".replace("%", "".join(VOWELS)), word) In this example we get: [('~', 'd', 'o')] The issue is that I wish the matches to overlap - the last set of vowels should become the first set of the next match. This appears possible with lookaheads, if we replace the regex as follows: re.findall("([%]+)([^%]+)(?=[%]+)".replace("%", "".join(VOWELS)), word) We get: [('~', 'd'), ('o', 'g')] Which means we are matching what I want. However, it now doesn't return the last set of vowels. The output I want is: [('~', 'd', 'o'), ('o', 'g', '~')] I feel this should be possible (if the regex can check for the second set of vowels, I see no reason it can't return them), but I can't find any way of doing it beyond the brute force method, looping through the results after I have them and appending the first character of the next match to the last match, and the last character of the string to the last match. Is there a better way in which I can do this? The two things that would work would be capturing the lookahead value, or not consuming the text on a match, while capturing the value - I can't find any way of doing either.

    Read the article

  • Testing subpackage modules in Python 3

    - by Mitchell Model
    I have been experimenting with various uses of hierarchies like this and the differences between absolute and relative imports, and can't figure out how to do routine things with the package, subpackages, and modules without simply putting everything on sys.path. I have a two-level package hierarchy: MyApp __init__.py Application __init__.py Module1 Module2 ... Domain __init__.py Module1 Module2 ... UI __init__.py Module1 Module2 ... I want to be able to do the following: Run test code in a Module's "if main" when the module imports from other modules in the same directory. Have one or more test code modules in each subpackage that runs unit tests on the modules in the subpackage. Have a set of unit tests that reside in someplace reasonable, but outside the subpackages, either in a sibling package, at the top-level package, or outside the top-level package (though all these might end up doing is running the tests in each subpackage) "Enter" the structure from any of the three subpackage levels, e.g. run code that just uses Domain modules, run code that just uses Application modules, but Application uses code from both Application and Domain modules, and run code from GUI uses code from both GUI and Application; for instance, Application test code would import Application modules but not Domain modules. After developing the bulk of the code without subpackages, continue developing and testing after organizing the modules into this hierarchy. I know how to use relative imports so that external code that puts MyApp on its sys.path can import MyApp, import any subpackages it wants, and import things from their modules, while the modules in each subpackage can import other modules from the same subpackage or from sibling packages. However, the development needs listed above seem incompatible with subpackage structuring -- in other words, I can't have it both ways: a well-structured multi-level package hierarchy used from the outside and also used from within, in particular for testing but also because modules from one design level (in particular the UI) should not import modules from a design level below the next one down. Sorry for the long essay, but I think it fairly represents the struggles a lot of people have been having adopting to the new relative import mechanisms.

    Read the article

  • python can't start a new thread

    - by Giorgos Komnino
    I am building a multi threading application. I have setup a threadPool. [ A Queue of size N and N Workers that get data from the queue] When all tasks are done I use tasks.join() where tasks is the queue . The application seems to run smoothly until suddently at some point (after 20 minutes in example) it terminates with the error thread.error: can't start new thread Any ideas? Edit: The threads are daemon Threads and the code is like: while True: t0 = time.time() keyword_statuses = DBSession.query(KeywordStatus).filter(KeywordStatus.status==0).options(joinedload(KeywordStatus.keyword)).with_lockmode("update").limit(100) if keyword_statuses.count() == 0: DBSession.commit() break for kw_status in keyword_statuses: kw_status.status = 1 DBSession.commit() t0 = time.time() w = SWorker(threads_no=32, network_server='http://192.168.1.242:8180/', keywords=keyword_statuses, cities=cities, saver=MySqlRawSave(DBSession), loglevel='debug') w.work() print 'finished' When the daemon threads are killed? When the application finishes or when the work() finishes? Look at the thread pool and the worker (it's from a recipe ) from Queue import Queue from threading import Thread, Event, current_thread import time event = Event() class Worker(Thread): """Thread executing tasks from a given tasks queue""" def __init__(self, tasks): Thread.__init__(self) self.tasks = tasks self.daemon = True self.start() def run(self): '''Start processing tasks from the queue''' while True: event.wait() #time.sleep(0.1) try: func, args, callback = self.tasks.get() except Exception, e: print str(e) return else: if callback is None: func(args) else: callback(func(args)) self.tasks.task_done() class ThreadPool: """Pool of threads consuming tasks from a queue""" def __init__(self, num_threads): self.tasks = Queue(num_threads) for _ in range(num_threads): Worker(self.tasks) def add_task(self, func, args=None, callback=None): ''''Add a task to the queue''' self.tasks.put((func, args, callback)) def wait_completion(self): '''Wait for completion of all the tasks in the queue''' self.tasks.join() def broadcast_block_event(self): '''blocks running threads''' event.clear() def broadcast_unblock_event(self): '''unblocks running threads''' event.set() def get_event(self): '''returns the event object''' return event

    Read the article

  • Using unicodedata.normalize in Python 2.7

    - by dpitch40
    Once again, I am very confused with a unicode question. I can't figure out how to successfully use unicodedata.normalize to convert non-ASCII characters as expected. For instance, I want to convert the string u"Cœur" To u"Coeur" I am pretty sure that unicodedata.normalize is the way to do this, but I can't get it to work. It just leaves the string unchanged. >>> s = u"Cœur" >>> unicodedata.normalize('NFKD', s) == s True What am I doing wrong?

    Read the article

  • python add two list and createing a new list

    - by Adam G.
    lst1 = ['company1,AAA,7381.0 ', 'company1,BBB,-8333.0 ', 'company1,CCC, 3079.999 ', 'company1,DDD,5699.0 ', 'company1,EEE,1640.0 ', 'company1,FFF,-600.0 ', 'company1,GGG,3822.0 ', 'company1,HHH,-600.0 ', 'company1,JJJ,-4631.0 ', 'company1,KKK,-400.0 '] lst2 =['company1,AAA,-4805.0 ', 'company1,ZZZ,-2576.0 ', 'company1,BBB,1674.0 ', 'company1,CCC,3600.0 ', 'company1,DDD,1743.998 '] output I need == ['company1,AAA,2576.0','company1,ZZZ,-2576.0 ','company1,KKK,-400.0 ' etc etc] I need to add it similar product number in each list and move it to a new list. I also need any symbol not being added together to be added to that new list. I am having problems with moving through each list. This is what I have: h = [] z = [] a = [] for g in hhl: spl1 = g.split(",") h.append(spl1[1]) for j in c_hhl: spl2 = j.split(",") **if spl2[1] in h: converted_num =(float(spl2[2]) +float(spl1[2])) pos=('{0},{1},{2}'.format(spl2[0],spl2[1],converted_num)) z.append(pos)** else: pos=('{0},{1},{2}'.format(spl2[0],spl2[1],spl2[2])) z.append(pos) for f in z: spl3 = f.split(",") a.append(spl3[1]) for n in hhl[:]: spl4 = n.split(",") if spl4[1] in a: got = (spl4[0],spl4[1],spl4[2]) hhl.remove(n) smash = hhl+z #for i in smash: for i in smash: print(i) I am having problem iterating through the list to make sure I get all of the simliar product to a new list,(bold) and any product not in list 1 but in lst2 to the new list and vice versa. I am sure there is a much easier way.

    Read the article

  • Loop colours from variables for graphics.py [Python 3.2]

    - by user1056548
    I am creating a graphics program that draws 100 x 100 squares next to each other depending on the user-specified grid size. The user also inputs 4 colours for the squares to be coloured (e.g. if they enter red,green,blue,yellow the squares will be coloured in that order, repeating the colours). Is it possible to loop the colours from the variables the user has given? Here is what I have so far: def main(): print ("Please enter four comma seperated colours e.g.: 'red,green,blue,yellow'\n\ Allowed colours are: red, green, blue, yellow and cyan") col1, col2, col3, col4 = input("Enter your four colours: ").split(',') win = GraphWin ("Squares", 500, 500) colours = [col1, col2, col3, col4] drawSquare (win, col1, col2, col3, col4, colours) win.getMouse() win.close() def drawSquare(win, col1, col2, col3, col4, colours): for i in range (4): for j in range (len(colours)): colour = colours[j] x = 50 + (i * 50) circle = Circle (Point (x,50), 20) circle.setFill(colour) circle.draw(win) I think I should be using a list in some way, but can't work out exactly how to do it. Can anybody help?

    Read the article

  • Python 3.0 - Dynamic Class Instance Naming

    - by Jon
    I want to use a while loop to initialize class objects with a simple incremented naming convention. The goal is to be able to scale the number of class objects at will and have the program generate the names automatically. (ex. h1...h100...h1000...) Each h1,h2,h3... being its own instance. Here is my first attempt... have been unable to find a good example. class Korker(object): def __init__(self,ident,roo): self.ident = ident self.roo = roo b = 1 hwinit = 'h' hwstart = 0 while b <= 10: showit = 'h' + str(b) print(showit) #showit seems to generate just fine as demonstrated by print str(showit) == Korker("test",2) #this is the line that fails b += 1 The errors I get range from a string error to a cannot use function type error.... Any help would be greatly appreciated.

    Read the article

  • Python Speeding Up Retrieving data from extremely large string

    - by Burninghelix123
    I have a list I converted to a very very long string as I am trying to edit it, as you can gather it's called tempString. It works as of now it just takes way to long to operate, probably because it is several different regex subs. They are as follow: tempString = ','.join(str(n) for n in coords) tempString = re.sub(',{2,6}', '_', tempString) tempString = re.sub("[^0-9\-\.\_]", ",", tempString) tempString = re.sub(',+', ',', tempString) clean1 = re.findall(('[-+]?[0-9]*\.?[0-9]+,[-+]?[0-9]*\.?[0-9]+,' '[-+]?[0-9]*\.?[0-9]+'), tempString) tempString = '_'.join(str(n) for n in clean1) tempString = re.sub(',', ' ', tempString) Basically it's a long string containing commas and about 1-5 million sets of 4 floats/ints (mixture of both possible),: -5.65500020981,6.88999986649,-0.454999923706,1,,,-5.65500020981,6.95499992371,-0.454999923706,1,,, The 4th number in each set I don't need/want, i'm essentially just trying to split the string into a list with 3 floats in each separated by a space. The above code works flawlessly but as you can imagine is quite time consuming on large strings. I have done a lot of research on here for a solution but they all seem geared towards words, i.e. swapping out one word for another. EDIT: Ok so this is the solution i'm currently using: def getValues(s): output = [] while s: # get the three values you want, discard the 3 commas, and the # remainder of the string v1, v2, v3, _, _, _, s = s.split(',', 6) output.append("%s %s %s" % (v1.strip(), v2.strip(), v3.strip())) return output coords = getValues(tempString) Anyone have any advice to speed this up even farther? After running some tests It still takes much longer than i'm hoping for. I've been glancing at numPy, but I honestly have absolutely no idea how to the above with it, I understand that after the above has been done and the values are cleaned up i could use them more efficiently with numPy, but not sure how NumPy could apply to the above. The above to clean through 50k sets takes around 20 minutes, I cant imagine how long it would be on my full string of 1 million sets. I'ts just surprising that the program that originally exported the data took only around 30 secs for the 1 million sets

    Read the article

  • Adding a decorator that converts strings to lowercase in Python

    - by user2905382
    So I am new to learning decorators and I have gone through countless tutorials and while I understand and can mostly follow all of the examples, I think the best way to learn, would be to implement a decorator myself. So I am going to use this example below. I realize a decorator is not at all necessary to do this, but for the sake of learning, I would like to add a decorator that filters the strings like dog name and breed and turns them into lowercase. Any ideas or pointers in the right direction would be appreciated. class Dogs: totalDogs = 0 dogList=[] def __init__(self, breed, color, age): self.breed=breed self.color=color self.age=age Dogs.dogList.append(self.breed) Dogs.totalDogs += 1 def displayDogs(self): print "breed: ", self.breed print "color: ",self.color print "age: ",self.age print "list of breeds:", Dogs.dogList print "total dogs: ", Dogs.totalDogs def somedecorator(*args): #now what terrier=Dogs("TeRrIer", "white", 5) terrier.displayDogs() retriever=Dogs("goldenRETRIEVER", "brown", 10) retriever.displayDogs()

    Read the article

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

< Previous Page | 14 15 16 17 18 19 20 21 22 23 24 25  | Next Page >