How to query data from a password protected https website
Posted
by Addie
on Stack Overflow
See other posts from Stack Overflow
or by Addie
Published on 2010-04-14T21:30:41Z
Indexed on
2010/04/20
22:53 UTC
Read the original article
Hit count: 246
I'd like my application to query a csv file from a secure website. I have no experience with web programming so I'd appreciate detailed instructions. Currently I have the user login to the site, manually query the csv, and have my application load the file locally. I'd like to automate this by having the user enter his login information, authenticating him on the website, and querying the data. The application is written in C# .NET. The url of the site is: https://www2.emidas.com/default.asp.
I've tested the following code already and am able to access the file once the user has already authenticated himself and created a manual query.
System.Net.WebClient Client = new WebClient();
Stream strm = Client.OpenRead("https://www3.emidas.com/users/<username>/file.csv");
Here is the request sent to the site for authentication. I've angle bracketed the real userid and password.
POST /pwdVal.asp HTTP/1.1
Accept: image/jpeg, application/x-ms-application, image/gif, application/xaml+xml, image/pjpeg, application/x-ms-xbap, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, application/x-shockwave-flash, */*
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.2; Tablet PC 2.0; OfficeLiveConnector.1.4; OfficeLivePatch.1.3; .NET4.0C; .NET4.0E)
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip, deflate
Cookie: ASPSESSIONID<unsure if this data contained password info so removed>; ClientId=<username>
Host: www3.emidas.com
Content-Length: 36
Connection: Keep-Alive
Cache-Control: no-cache
Accept-Language: en-US
client_id=<username>&password=<password>
© Stack Overflow or respective owner