Scraping with multiple IP, in java.
Posted
by Titi Wangsa bin Damhore
on Stack Overflow
See other posts from Stack Overflow
or by Titi Wangsa bin Damhore
Published on 2010-03-15T13:56:57Z
Indexed on
2010/03/15
13:59 UTC
Read the original article
Hit count: 151
webpagescraping
Well basically I have a scraping application. It scrapes around n items per minute. currently i have only one IP.
The site i'm scraping allows me 3 connections per IP.
I'm thinking about getting another IP.
so i'll be able to get 6 connections.
in theory i should be able to get n items in 40 seconds, more or less.
currently i'm using java (commons-httpcore) to get the job done.
I'm not sure if this is java question or an OS question.
my machine has IP 1 and IP 2 how do i connect to, say, www.microsoft.com, using IP 1 and using IP2? how can i specify, which ip i want to use to do a connection?
© Stack Overflow or respective owner
