Creating interruptible process in python

Posted by Glycerine on Stack Overflow See other posts from Stack Overflow or by Glycerine
Published on 2011-01-05T00:13:55Z Indexed on 2011/01/05 19:53 UTC
Read the original article Hit count: 158

Filed under:
|
|
|

I'm creating a python script of which parses a large (but simple) CSV.

It'll take some time to process. I would like the ability to interrupt the parsing of the CSV so I can continue at a later stage.

Currently I have this - of which lives in a larger class: (unfinished)

Edit:

I have some changed code. But the system will parse over 3 million rows.

def parseData(self)
    reader = csv.reader(open(self.file))
    for id, title, disc in reader:
        print "%-5s %-50s %s" % (id, title, disc)
        l = LegacyData()
        l.old_id = int(id)
        l.name = title
        l.disc_number = disc
        l.parsed = False
        l.save()

This is the old code.

def parseData(self):
        #first line start
        fields = self.data.next()
        for row in self.data:
            items = zip(fields, row)
            item = {}
            for (name, value) in items:
                item[name] = value.strip()
            self.save(item)

Thanks guys.

© Stack Overflow or respective owner

Related posts about python

Related posts about Windows