Using HTMLAgility Pack to Extract Links

Posted by Soham on Stack Overflow See other posts from Stack Overflow or by Soham
Published on 2010-06-05T11:15:31Z Indexed on 2010/06/05 11:22 UTC
Read the original article Hit count: 529

Filed under:
|
|
|

Hi Folks, Consider this simplest piece of code:

    using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using HtmlAgilityPack;

namespace WebScraper
{
    class Program
    {
        static void Main(string[] args)
        {
            HtmlDocument doc = new HtmlDocument();
            doc.LoadHtml("http://www.google.com");

            foreach (HtmlNode link in doc.DocumentNode.SelectNodes("//a[@href]"))
            {
            }
        }
    }
}

This effectively doesnt do anything at all, and is copied/inspired from various other StackOverflow questions like this. When compiling this, there is a runtime error which says "Object reference not set to an instance of an object." highlighting the foreach line.

I can't understand, why the environment has become irritable to this humble,innocent and useless piece of code.

I would also like to know, does HTMLAgilityPack accept HTML classes as nodes?

© Stack Overflow or respective owner

Related posts about c#

Related posts about web