Search Results

Search found 13693 results on 548 pages for 'python metaprogramming'.

Page 143/548 | < Previous Page | 139 140 141 142 143 144 145 146 147 148 149 150  | Next Page >

  • Most efficient way to search the last x lines of a file in python

    - by Harley
    I have a file and I don't know how big it's going to be (it could be quite large, but the size will vary greatly). I want to search the last 10 lines or so to see if any of them match a string. I need to do this as quickly and efficiently as possible and was wondering if there's anything better than: s = "foo" last_bit = fileObj.readlines()[-10:] for line in last_bit: if line == s: print "FOUND"

    Read the article

  • Python halts while iteratively processing my 1GB csv file

    - by Dan
    I have two files: metadata.csv: contains an ID, followed by vendor name, a filename, etc hashes.csv: contains an ID, followed by a hash The ID is essentially a foreign key of sorts, relating file metadata to its hash. I wrote this script to quickly extract out all hashes associated with a particular vendor. It craps out before it finishes processing hashes.csv stored_ids = [] # this file is about 1 MB entries = csv.reader(open(options.entries, "rb")) for row in entries: # row[2] is the vendor if row[2] == options.vendor: # row[0] is the ID stored_ids.append(row[0]) # this file is 1 GB hashes = open(options.hashes, "rb") # I iteratively read the file here, # just in case the csv module doesn't do this. for line in hashes: # not sure if stored_ids contains strings or ints here... # this probably isn't the problem though if line.split(",")[0] in stored_ids: # if its one of the IDs we're looking for, print the file and hash to STDOUT print "%s,%s" % (line.split(",")[2], line.split(",")[4]) hashes.close() This script gets about 2000 entries through hashes.csv before it halts. What am I doing wrong? I thought I was processing it line by line. ps. the csv files are the popular HashKeeper format and the files I am parsing are the NSRL hash sets. http://www.nsrl.nist.gov/Downloads.htm#converter UPDATE: working solution below. Thanks everyone who commented! entries = csv.reader(open(options.entries, "rb")) stored_ids = dict((row[0],1) for row in entries if row[2] == options.vendor) hashes = csv.reader(open(options.hashes, "rb")) matches = dict((row[2], row[4]) for row in hashes if row[0] in stored_ids) for k, v in matches.iteritems(): print "%s,%s" % (k, v)

    Read the article

  • Importing files in Python from __init__.py

    - by Federico Builes
    Suppose I have the following structure: app/ __init__.py foo/ a.py b.py c.py __init__.py a.py, b.py and c.py share some common imports (logging, os, re, etc). Is it possible to import these three or four common modules from the __init__.py file so I don't have to import them in every one of the files? Edit: My goal is to avoid having to import 5-6 modules in each file and it's not related to performance reasons.

    Read the article

  • Python nested function scopes

    - by Thomas O
    I have code like this: def write_postcodes(self): """Write postcodes database. Write data to file pointer. Data is ordered. Initially index pages are written, grouping postcodes by the first three characters, allowing for faster searching.""" status("POSTCODE", "Preparing to sort...", 0, 1) # This function returns the key of x whilst updating the displayed # status of the sort. ctr = 0 def keyfunc(x): ctr += 1 status("POSTCODE", "Sorting postcodes", ctr, len(self.postcodes)) return x sort_res = self.postcodes[:] sort_res.sort(key=keyfunc) But ctr responds with a NameError: Traceback (most recent call last): File "PostcodeWriter.py", line 53, in <module> w.write_postcodes() File "PostcodeWriter.py", line 47, in write_postcodes sort_res.sort(key=keyfunc) File "PostcodeWriter.py", line 43, in keyfunc ctr += 1 UnboundLocalError: local variable 'ctr' referenced before assignment How can I fix this? I thought nester scopes would have allowed me to do this. I've tried with 'global', but it still doesn't work.

    Read the article

  • python decorator to modify variable in current scope

    - by AlexH
    Goal: Make a decorator which can modify the scope that it is used in. If it worked: class Blah(): # or perhaps class Blah(ParentClassWhichMakesThisPossible) def one(self): pass @decorated def two(self): pass Blah.decorated ["two"] Why? I essentially want to write classes which can maintain specific dictionaries of methods, so that I can retrieve lists of available methods of different types on a per class basis. errr..... I want to do this: class RuleClass(ParentClass): @rule def blah(self): pass @rule def kapow(self): pass def shazam(self): class OtherRuleClass(ParentClass): @rule def foo(self): pass def bar(self): pass RuleClass.rules.keys() ["blah", "kapow"] OtherRuleClass.rules.keys() ["foo"]

    Read the article

  • Extract strings in python

    - by shadyabhi
    Basically, I want to extract the strings "AAA", "BBB", "CCC", "DDD" from a text file.. ...... (other text goes here)..... <TD align="left" class=texttd><font class='textfont'>AAA</font></TD> ..... (useless text here)..... <TD align="left" class=texttd><font class='textfont'>BBB</font></TD> ....(more text)..... <TD align="left" class=texttd><font class='textfont'>CCC</font></TD> <TD align="left" class=texttd><font class='textfont'>DDD</font></TD> ......(more text)..... I want something like if I do:- data = foo("file.txt") i get:- data = ['AAA','BBB','CCC','DDD'] What is the best possible way? My file is not big..

    Read the article

  • strange behavior in python

    - by fsm
    The tags might not be accurate since I am not sure where the problem is. I have a module where I am trying to read some data from a socket, and write the results into a file (append) It looks something like this, (only relevant parts included) if __name__ == "__main__": <some init code> for line in file: t = Thread(target=foo, args=(line,)) t.start() while nThreads > 0: time.sleep(1) Here are the other modules, def foo(text): global countLock, nThreads countLock.acquire() nThreads += 1 countLock.release() """connect to socket, send data, read response""" writeResults(text, result) countLock.acquire() nThreads -= 1 countLock.release() def writeResults(text, result): """acquire file lock""" """append to file""" """release file lock""" Now here's the problem. Initially, I had a typo in the function 'foo', where I was passing the variable 'line' to writeResults instead of 'text'. 'line' is not defined in the function foo, it's defined in the main block, so I should have seen an error, but instead, it worked fine, except that the data was appended to the file multiple times, instead of being written just once, which is the required behavior, which I got when I fixed the typo. My question is, 1) Why didn't I get an error? 2) Why was the writeResults function being called multiple times?

    Read the article

  • Write xml file using lxml library in Python

    - by systempuntoout
    I'm using lxml to create an XML file from scratch; having a code like this: from lxml import etree root = etree.Element("root") root.set("interesting", "somewhat") child1 = etree.SubElement(root, "test") How do i write root Element object to an xml file using write() method of ElementTree class?

    Read the article

  • Python - excel - xlwt: colouring every second row

    - by konjo
    Hi, i just finish some MYSQL to excel script with xlwt and I need to colour every second row for easy reading. I have tried this: row = easyxf('pattern: pattern solid, fore_colour blue') for i in range(0,10,2): ws0.row(i).set_style(row) Alone this colouring is fine, but when when I write my data rows are again white. Can some please show me some example 'cuz I m lost in coding :/ Best Regards.

    Read the article

  • Proper way to assert type of variable in Python

    - by Morlock
    In using a function, I wish to ensure that the type of the variables are as expected. How to do it right? Here is an example fake function trying to do just this before going on with its role: def my_print(text, begin, end): """Print text in UPPER between 'begin' and 'end' in lower """ for i in (text, begin, end): assert type(i) == type("") out = begin.lower() + text.upper() + end.lower() print out Is this approach valid? Should I use something else than type(i) == type("") ? Should I use try/except instead? Thanks pythoneers

    Read the article

  • assign operator to variable in python?

    - by abhilashm86
    Usual method of applying mathematics to variables is a * b Is it able to calculate and manipulate two operands like this? a = input('enter a value') b = input('enter a value') op = raw_input('enter a operand') then how do i connect op and two variables a and b?? i know i can compare op to +, -, %, $ and then assign and compute.... but can i do something like a op b , how to tell compiler that op is an operator?? any tweaks possible?

    Read the article

  • Reading numeric Excel data as text using xlrd in Python

    - by Brian
    Hi guys, I am trying to read in an Excel file using xlrd, and I am wondering if there is a way to ignore the cell formatting used in Excel file, and just import all data as text? Here is the code I am using for far: import xlrd xls_file = 'xltest.xls' xls_workbook = xlrd.open_workbook(xls_file) xls_sheet = xls_workbook.sheet_by_index(0) raw_data = [['']*xls_sheet.ncols for _ in range(xls_sheet.nrows)] raw_str = '' feild_delim = ',' text_delim = '"' for rnum in range(xls_sheet.nrows): for cnum in range(xls_sheet.ncols): raw_data[rnum][cnum] = str(xls_sheet.cell(rnum,cnum).value) for rnum in range(len(raw_data)): for cnum in range(len(raw_data[rnum])): if (cnum == len(raw_data[rnum]) - 1): feild_delim = '\n' else: feild_delim = ',' raw_str += text_delim + raw_data[rnum][cnum] + text_delim + feild_delim final_csv = open('FINAL.csv', 'w') final_csv.write(raw_str) final_csv.close() This code is functional, but there are certain fields, such as a zip code, that are imported as numbers, so they have the decimal zero suffix. For example, is there is a zip code of '79854' in the Excel file, it will be imported as '79854.0'. I have tried finding a solution in this xlrd spec, but was unsuccessful.

    Read the article

  • How should I use random.jumpahead in Python

    - by Peter Smit
    I have a application that does a certain experiment 1000 times (multi-threaded, so that multiple experiments are done at the same time). Every experiment needs appr. 50.000 random.random() calls. What is the best approach to get this really random. I could copy a random object to every experiment and do than a jumpahead of 50.000 * expid. The documentation suggests that jumpahead(1) already scrambles the state, but is that really true? Or is there another way to do this in 'the best way'? (No, the random numbers are not used for security, but for a metropolis hasting algorithm. The only requirement is that the experiments are independent, not whether the random sequence is somehow predictable or so)

    Read the article

  • How to merge duplicates in 2D python arrays

    - by Wei Lou
    Hi, I have a set of data similar to this: No Start Time End Time CallType Info 1 13:14:37.236 13:14:53.700 Ping1 RTT(Avr):160ms 2 13:14:58.955 13:15:29.984 Ping2 RTT(Avr):40ms 3 13:19:12.754 13:19:14.757 Ping3_1 RTT(Avr):620ms 3 13:19:12.754 Ping3_2 RTT(Avr):210ms 4 13:14:58.955 13:15:29.984 Ping4 RTT(Avr):360ms 5 13:19:12.754 13:19:14.757 Ping1 RTT(Avr):40ms 6 13:19:59.862 13:20:01.522 Ping2 RTT(Avr):163ms ... when i parse through it, i need merge the results of Ping3_1 and Ping3_2. Then take average of those two row export as one row. So the end of result would be like this: No Start Time End Time CallType Info 1 13:14:37.236 13:14:53.700 Ping1 RTT(Avr):160ms 2 13:14:58.955 13:15:29.984 Ping2 RTT(Avr):40ms 3 13:19:12.754 13:19:14.757 Ping3 RTT(Avr):415ms 4 13:14:58.955 13:15:29.984 Ping4 RTT(Avr):360ms 5 13:19:12.754 13:19:14.757 Ping1 RTT(Avr):40ms 6 13:19:59.862 13:20:01.522 Ping2 RTT(Avr):163ms currently i am concatenating column 0 and 1 to make a unique key, find duplication there then doing rest of special treatment for those parallel Pings. It is not elegant at all. Just wonder what is the better way to do it. Thanks!

    Read the article

  • how to use @ in python..

    - by zjm1126
    this is my code: def a(): print 'sss' @a() def b(): print 'aaa' b() and the Traceback is: sss Traceback (most recent call last): File "D:\zjm_code\a.py", line 8, in <module> @a() TypeError: 'NoneType' object is not callable so how to use the '@' thanks

    Read the article

  • writing header in csv python with DictWriter

    - by user248237
    assume I have a csv.DictReader object and I want to write it out as a csv file. How can I do this? I thought of the following: dr = csv.DictReader(open(f), delimiter='\t') # process my dr object # ... # write out object output = csv.DictWriter(open(f2, 'w'), delimiter='\t') for item in dr: output.writerow(item) Is that the best way? More importantly, how can I make it so a header is written out too, in this case the object "dr"s .fieldnames property? thanks.

    Read the article

  • Reading Python Documentation for 3rd party modules

    - by Shadyabhi
    I recently downloaded IMDbpy moduele.. When I do, import imdb help(imdb) i dont get the full documentation.. I have to do im = imdb.IMDb() help(im) to see the available methods. I dont like this console interface. Is there any better way of reading the doc. I mean all the doc related to module imdb in one page..

    Read the article

< Previous Page | 139 140 141 142 143 144 145 146 147 148 149 150  | Next Page >