C# WebClient only downloads partial html
Posted
by H4mm3rHead
on Stack Overflow
See other posts from Stack Overflow
or by H4mm3rHead
Published on 2010-04-13T09:35:57Z
Indexed on
2010/04/13
9:42 UTC
Read the original article
Hit count: 497
Hi, I am working on some scraping app, i wanted to try to get it to work but ran into a problem. I have replaced the original scraping destination in the below code with googles webpage, just for testing. It seems that my download doesnt get everything, i note that the body and the html tags are missing their close tags. How do i get it to download everything? Whats wrong with my sample code:
string filename = "test.html";
WebClient client = new WebClient();
string searchTerm = HttpUtility.UrlEncode(textBox2.Text);
client.QueryString.Add("q", searchTerm);
client.QueryString.Add("hl", "en");
string data = client.DownloadString("http://www.google.com/search");
StreamWriter writer = new StreamWriter(filename, false, Encoding.Unicode);
writer.Write(data);
writer.Flush();
writer.Close();
© Stack Overflow or respective owner