How to query data from a password protected https website
- by Addie
I'd like my application to query a csv file from a secure website. I have no experience with web programming so I'd appreciate detailed instructions. Currently I have the user login to the site, manually query the csv, and have my application load the file locally. I'd like to automate this by having the user enter his login information, authenticating him on the website, and querying the data. The application is written in C# .NET. The url of the site is: https://www2.emidas.com/default.asp.
I've tested the following code already and am able to access the file once the user has already authenticated himself and created a manual query.
System.Net.WebClient Client = new WebClient();
Stream strm = Client.OpenRead("https://www3.emidas.com/users/<username>/file.csv");
Here is the request sent to the site for authentication. I've angle bracketed the real userid and password.
POST /pwdVal.asp HTTP/1.1
Accept: image/jpeg, application/x-ms-application, image/gif, application/xaml+xml, image/pjpeg, application/x-ms-xbap, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, application/x-shockwave-flash, */*
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.2; Tablet PC 2.0; OfficeLiveConnector.1.4; OfficeLivePatch.1.3; .NET4.0C; .NET4.0E)
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip, deflate
Cookie: ASPSESSIONID<unsure if this data contained password info so removed>; ClientId=<username>
Host: www3.emidas.com
Content-Length: 36
Connection: Keep-Alive
Cache-Control: no-cache
Accept-Language: en-US
client_id=<username>&password=<password>
