What is the fastest way to send 100,000 HTTP requests in Python?
Posted
by Igor G.
on Stack Overflow
See other posts from Stack Overflow
or by Igor G.
Published on 2010-04-13T19:19:50Z
Indexed on
2010/04/13
19:23 UTC
Read the original article
Hit count: 591
Hello, I am opening a file which has 100,000 url's. I need to send an http request to each url and print the status code. I am using Python 2.6, and so far looked at the many confusing ways Python implements threading/concurrency. I have even looked at the python concurrence library, but cannot figure out how to write this program correctly. Has anyone come across a similar problem? I guess generally I need to know how to perform thousands of tasks in Python as fast as possible - I suppose that means 'concurrently'.
Thank you, Igor
© Stack Overflow or respective owner