Is there a way to circumvent Python list.append() becoming progressively slower in a loop as the lis
Posted
by Deniz
on Stack Overflow
See other posts from Stack Overflow
or by Deniz
Published on 2010-03-18T22:30:08Z
Indexed on
2010/03/18
22:31 UTC
Read the original article
Hit count: 429
I have a big file I'm reading from, and convert every few lines to an instance of an Object.
Since I'm looping through the file, I stash the instance to a list using list.append(instance), and then continue looping.
This is a file that's around ~100MB so it isn't too large, but as the list grows larger, the looping slows down progressively. (I print the time for each lap in the loop).
This is not intrinsic to the loop ~ when I print every new instance as I loop through the file, the program progresses at constant speed ~ it is only when I append them to a list it gets slow.
My friend suggested disabling garbage collection before the while loop and enabling it afterward & making a garbage collection call.
Did anyone else observe a similar problem with list.append getting slower? Is there any other way to circumvent this?
© Stack Overflow or respective owner
