Dynamically blocking excessive HTTP bandwidth use?
- by Jeff Atwood
We were a little surprised to see this on our Cacti graphs for June 4 web traffic:
We ran Log Parser on our IIS logs and it turns out this was a perfect storm of Yahoo and Google bots indexing us.. in that 3 hour period, we saw 287k hits from 3 different google ips, plus 104k from yahoo. Ouch?
While we don't want to block Google or Yahoo, this…