What's the best way to write a maintainable web scraping app?
Posted
by Benj
on Stack Overflow
See other posts from Stack Overflow
or by Benj
Published on 2009-11-09T11:17:10Z
Indexed on
2010/05/10
13:24 UTC
Read the original article
Hit count: 247
I wrote a perl script a while ago which logged into my online banking and emailed me my balance and a mini-statement every day. I found it very useful for keeping track of my finances. The only problem is that I wrote it just using perl and curl and it was quite complicated and hard to maintain. After a few instances of my bank changing their webpage I got fed up of debugging it to keep it up to date.
So what's the best way of writing such a program in such a way that it's easy to maintain? I'd like to write a nice well engineered version in either Perl or Java which will be easy to update when the bank inevitably fiddle with their web site.
© Stack Overflow or respective owner