How to set timeout with python-mechanize?

Posted by Michal Cihar on Stack Overflow See other posts from Stack Overflow or by Michal Cihar
Published on 2014-01-27T15:56:44Z Indexed on 2014/06/02 21:27 UTC
Read the original article Hit count: 349

Filed under:
|
|

I'm using python-mechanize to scrape some web sites, which sometime simply don't respond to requests and these requests stay open too long, so I need to limit timeout for these requests.

While using urlopen method, the timeout can be set using timeout parameter, but I have not found easy way for doing it with high level API such as submit or click methods. Ideally the timeout would be set just once for whole browser class and all calls would honor that.

It would be probably possible to customize this by passing custom request_class to every click and submit call, but this would just pollute the code, so I'm looking for nicer solution for setting timeout for mechanize's browser class (and no, I don't want to change default socket timeout using socket.setdefaulttimeout).

© Stack Overflow or respective owner

Related posts about python

Related posts about timeout