Scraping a page from a secure URL which is possibly using a session ID

Posted by VN44CA on Stack Overflow See other posts from Stack Overflow or by VN44CA
Published on 2010-04-09T03:17:22Z Indexed on 2010/04/09 3:23 UTC
Read the original article Hit count: 324

How to scrape a page like this.

https://www.procom.ca/JobList.aspx?keywords=&Cities=&reference=&JobType=0

It is secure, and requires a referrer? I can't get anything using wget or httplib2.

If you go through this page, you get a list and it works on a browser but not the command line. https://www.procom.ca/jobsearch.aspx

I am interested in command line fetching.

thx

© Stack Overflow or respective owner

Related posts about python

Related posts about screen-scraping