Is there a good tutorial for figuring out what a website is doing so your program can do the same th
- by brian d foy
Is there a good guide or tutorial for people who need to programmatically interact with dynamic websites? There's been a rash of Perl questions about that lately, and I haven't found a good resource to point people toward. I'm asking not because I need one but because I don't want to waste my time writing it if it already exists. Although I'm most interested in Perl, the extra tools and techniques are mostly the same.
Typically, I see see these problems in people's questions:
Handling, setting, and saving cookies
Finding and interacting with forms
Handling JavaScript inside your user-agent
especially things like onLoad, onSumbit, and Ajax
Using HTTP sniffer tools
Using Web developer plugins in interactive browsers
Interacting with DOM, screen scraping, etc.
If there's no good tutorial, I'll add it to my list of things to do (unless someone else wants to do it :). Along the way, if you don't have a suggestion for an existing tutorial, please suggest the things that you think should be in a new one, including links, your favorite tools, and your own user-agent development experiences. I don't care about the particular language you use.
