Loading a DB table into nested dictionaries in Python
Posted
by Hossein
on Stack Overflow
See other posts from Stack Overflow
or by Hossein
Published on 2010-05-17T22:14:42Z
Indexed on
2010/05/17
22:20 UTC
Read the original article
Hit count: 255
Hi, I have a table in MySql DB which I want to load it to a dictionary in python. the table columns is as follows:
id,url,tag,tagCount
tagCount is the number of times that a tag has been repeated for a certain url. So in that case I need a nested dictionary, in other words a dictionary of dictionary, to load this table. Because each url have several tags for which there are different tagCounts.the code that I used is this:( the whole table is about 22,000 records )
cursor.execute( ''' SELECT url,tag,tagCount
FROM wtp ''')
urlTagCount = cursor.fetchall()
d = defaultdict(defaultdict)
for url,tag,tagCount in urlTagCount:
d[url][tag]=tagCount
print d
first of all I want to know if this is correct.. and if it is why it takes so much time? Is there any faster solutions? I am loading this table into memory to have fast access to get rid of the hassle of slow database operations, but with this slow speed it has become a bottleneck itself, it is even much slower than DB access. and anyone help? thanks
© Stack Overflow or respective owner