Scraping with multiple IP, in java.
- by Titi Wangsa bin Damhore
Well basically I have a scraping application.
It scrapes around n items per minute.
currently i have only one IP.
The site i'm scraping allows me 3 connections per IP.
I'm thinking about getting another IP.
so i'll be able to get 6 connections.
in theory i should be able to get n items in 40 seconds, more or less.
currently i'm using java (commons-httpcore) to get the job done.
I'm not sure if this is java question or an OS question.
my machine has IP 1 and IP 2
how do i connect to, say, www.microsoft.com,
using IP 1 and using IP2?
how can i specify, which ip i want to use to do a connection?