How to download file into string with progress callback?
- by Kaminari
I would like to use the WebClient (or there is another better option?) but there is a problem. I understand that opening up the stream takes some time and this can not be avoided. However, reading it takes a strangely much more amount of time compared to read it entirely immediately.
Is there a best way to do this? I mean two ways, to string and to file. Progress is my own delegate and it's working good.
FIFTH UPDATE:
Finally, I managed to do it. In the meantime I checked out some solutions what made me realize that the problem lies elsewhere.
I've tested custom WebResponse and WebRequest objects, library libCURL.NET and even Sockets.
The difference in time was gzip compression. Compressed stream lenght was simply half the normal stream lenght and thus download time was less than 3 seconds with the browser.
I put some code if someone will want to know how i solved this: (some headers are not needed)
public static string DownloadString(string URL)
{
WebClient client = new WebClient();
client.Headers["User-Agent"] = "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.1.249.1045 Safari/532.5";
client.Headers["Accept"] = "application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
client.Headers["Accept-Encoding"] = "gzip,deflate,sdch";
client.Headers["Accept-Charset"] = "ISO-8859-2,utf-8;q=0.7,*;q=0.3";
Stream inputStream = client.OpenRead(new Uri(URL));
MemoryStream memoryStream = new MemoryStream();
const int size = 32 * 4096;
byte[] buffer = new byte[size];
if (client.ResponseHeaders["Content-Encoding"] == "gzip")
{
inputStream = new GZipStream(inputStream, CompressionMode.Decompress);
}
int count = 0;
do
{
count = inputStream.Read(buffer, 0, size);
if (count > 0)
{
memoryStream.Write(buffer, 0, count);
}
}
while (count > 0);
string result = Encoding.Default.GetString(memoryStream.ToArray());
memoryStream.Close();
inputStream.Close();
return result;
}
I think that asyncro functions will be almost the same. But i will simply use another thread to fire this function. I dont need percise progress indication.
