Accessing items from a dictionary using pickle efficiently in Python
Posted
by
user248237
on Stack Overflow
See other posts from Stack Overflow
or by user248237
Published on 2010-12-22T19:44:36Z
Indexed on
2010/12/22
19:54 UTC
Read the original article
Hit count: 188
I have a large dictionary mapping keys (which are strings) to objects. I pickled this large dictionary and at certain times I want to pull out only a handful of entries from it. The dictionary has usually thousands of entries total. When I load the dictionary using pickle, as follows:
from cPickle import *
# my dictionary from pickle, containing thousands of entries
mydict = open(load('mypickle.pickle'))
# accessing only handful of entries here
for entry in relevant_entries:
# find relevant entry
value = mydict[entry]
I notice that it can take up to 3-4 seconds to load the entire pickle, which I don't need, since I access only a tiny subset of the dictionary entries later on (shown above.)
How can I make it so pickle only loads those entries that I have from the dictionary, to make this faster?
Thanks.
© Stack Overflow or respective owner