Can YQL parse web sites requiring cookie-based authentication?
Posted
by user249488
on Stack Overflow
See other posts from Stack Overflow
or by user249488
Published on 2010-01-13T03:43:39Z
Indexed on
2010/03/20
18:01 UTC
Read the original article
Hit count: 451
First, my use case:
I'm trying to use YQL's built in XPATH capabilities to scrape content from Yahoo! Fantasy Sports. It uses some sort of cookie-based authentication scheme. Basically, the sequence is:
1) Do an HTTP GET on the Yahoo! Login page 2) Parse the hidden inputs from the response and do an HTTP PUT with your Yahoo! Login on the form URL 3) Use the cookies returned from step 2 to GET any of the Fantasy Sports! websites that you have access to
My question is, does YQL support doing this to scrape data? The only authentication based examples I've seen use OAuth, but I haven't seen any examples of using YQL to parse websites with cookie-based authentication schemes
© Stack Overflow or respective owner