Screen scrape a web page that uses javaScript and frames

Posted by Mello on Stack Overflow See other posts from Stack Overflow or by Mello
Published on 2010-02-07T13:05:39Z Indexed on 2010/04/19 1:03 UTC
Read the original article Hit count: 376

Filed under:
|

Hi, I want to scrape data from www.marktplaats.nl . I want to analyze the scraped description, price, date and views in Excel/Access.

I tried to scrape data with Ruby (nokogiri, scrapi) but nothing worked. (on other sites it worked well) The main problem is that for example selectorgadget and the add-on firebug (Firefox) don’t find any css I can use to scrape the page. On other sites I can extract the css with selectorgadget or firebug and use it with nokogiri or scrapi. Due to lack of experience it is difficult to identify the problem and therefore searching for a solution isn’t easy.

Can you tell me where to start solving this problem and where I maybe can find more info about a similar scraping process?

Thanks in advance!

© Stack Overflow or respective owner

Related posts about screen-scraping

Related posts about JavaScript