What dangers await if I block non-standard, non-major-usa search engine bots from my USA only website?
- by Ryan
I noticed tons of bandwidth being used by non-USA search engine bots, so I began blocking them in an effort to save bandwidth and cpu cycles for actual users and the search engines they come from (Google, Bing, Yahoo, Ask, etc.).
Other than potentially losing some international traffic (which isn't really important to us since all of our content is very USA-centric), what additional dangers should I be concerned about?
I'm using a modified version of Jeff Starr's User Agent Blocklist