How to set timeout with python-mechanize?
Posted
by
Michal Cihar
on Stack Overflow
See other posts from Stack Overflow
or by Michal Cihar
Published on 2014-01-27T15:56:44Z
Indexed on
2014/06/02
21:27 UTC
Read the original article
Hit count: 350
I'm using python-mechanize to scrape some web sites, which sometime simply don't respond to requests and these requests stay open too long, so I need to limit timeout for these requests.
While using urlopen
method, the timeout can be set using timeout
parameter, but I have not found easy way for doing it with high level API such as submit
or click
methods. Ideally the timeout would be set just once for whole browser class and all calls would honor that.
It would be probably possible to customize this by passing custom request_class
to every click
and submit
call, but this would just pollute the code, so I'm looking for nicer solution for setting timeout for mechanize's browser class (and no, I don't want to change default socket timeout using socket.setdefaulttimeout
).
© Stack Overflow or respective owner