python threading and performace?
Posted
by kumar
on Stack Overflow
See other posts from Stack Overflow
or by kumar
Published on 2010-06-10T07:39:04Z
Indexed on
2010/06/10
7:42 UTC
Read the original article
Hit count: 279
I had to do heavy I/o bound operation, i.e Parsing large files and converting from one format to other format. Initially I used to do it serially, i.e parsing one after another..! Performance was very poor ( it used take 90+ seconds). So I decided to use threading to improve the performance. I created one thread for each file. ( 4 threads)
for file in file_list:
t=threading.Thread(target = self.convertfile,args = file)
t.start()
ts.append(t)
for t in ts:
t.join()
But for my astonishment, there is no performance improvement whatsoever. Now also it takes around 90+ seconds to complete the task. As this is I/o bound operation , I had expected to improve the performance. What am I doing wrong?
© Stack Overflow or respective owner