what is the best method or tool to scrape web sites ?

Posted by user63898 on Stack Overflow See other posts from Stack Overflow or by user63898
Published on 2010-03-08T20:14:25Z Indexed on 2010/03/08 20:21 UTC
Read the original article Hit count: 182

Filed under:

scraping

Hello all i need to scrape (with approval) web sites before I start to write my own what is the best tool/way to scrape web sites, which is both fast (multithreaded) and easy to learn?

Related posts about scraping

Screen-scraping of a secure page of any site on https:// with asp.net in C#

as seen on Stack Overflow - Search for 'Stack Overflow'
I've done site scraping of secure page of any site on http:// but when I am trying to scrap any site on https:// then i always scrape the login page not secure page. Please advice what should i do for scraping a secure page of any site on https://. >>> More
looking for alternative to Webzinc .NET , screen scraping, web automation library for .net

as seen on Stack Overflow - Search for 'Stack Overflow'
i came across this .net library http://www.webzinc.com/online/faq.aspx however, i was wondering if there was a free alternative out there ? >>> More
PHP Screen Scraping Class

as seen on Bradino - Search for 'Bradino'
After some positive feedback I have decided to continue to develop the PHP Screen Scraping class. This post will server as the permanent home for the class. Download PHP Screen Scraping Class Updates 20009-07-30 Added setHeader() function >>> More
Alert Log Scraping with Oracle&#146;s ADRCI Utility

as seen on Internet.com - Search for 'Internet.com'
Oracles new ADR with command interface shows promise for database administrators who like to script their own solution for quickly scraping the alert log and automatically looking for errors. >>> More
Web scraping etiquette

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm considering writing a simple web scraping application to extract information from a website that does not seem to specifically prohibit this. I've checked for other alternatives (eg RSS, web service) to get this information, but there are none available at this stage. Despite this I've also… >>> More

Developer IT

what is the best method or tool to scrape web sites ? - Developer IT

what is the best method or tool to scrape web sites ?

scraping

Related posts about scraping

Screen-scraping of a secure page of any site on https:// with asp.net in C#

looking for alternative to Webzinc .NET , screen scraping, web automation library for .net

PHP Screen Scraping Class

Alert Log Scraping with Oracle&#146;s ADRCI Utility

Web scraping etiquette

Categories cloud