Load balanced proxies to avoid an API request limit
- by ClickClickClick
There is a certain API out there which limits the number of requests per day per IP. My plan is to create a bunch of EC2 instances with elastic IPs to sidestep the limitation. I'm familiar with EC2 and am just interested in the configuration of the proxies and a software load balancer.
I think I want to run a simple TCP Proxy on each instance and a software load balancer on the machine I will be requesting from. Something that allows the following to return a response from a different IP (round robin, availability, doesn't really matter..)
eg.
curl http://www.bbc.co.uk -x http://myproxyloadbalancer:port
Could anyone recommend a combination of software or even a link to an article that details a pleasing way to pull it off?
(My client won't be curl but is proxy aware.. I'll be making the requests from a Ruby script..)