I wrote a scraper using python a while back, and it worked fine in the command line. I have made a GUI for the application now, but I am having trouble with one issue. When I attempt to update text inside the gui (e.g. 'fetching URL 12/50'), I am unable seeing as the function within the scraper is grabbing 100+ links. Also when going from one scraping function, to a function that should update the gui, to another function, the gui update function seems to be skipped over while the next scrape function is run. An example would be:

scrapeLinksA() #takes 20 seconds
updateInfo("LinksA done")
scrapeLinksB() #takes another 20 seconds

in the above example, updateInfo is never executed, unless I end the program with a KeyboardInterrupt.

I'm thinking my solution is threading, but I'm not sure. What can I do to fix this?

I am using:

• PyQt4
• urllib2
• BeautifulSoup