Massive Crawling requests from Google Apps Engine useragent
- by SilentPlayer
Hi friends,
I'm badly affected with 'Google AppEngine-Google' UserAgent.. receiving 5/6 requests per second on http server. This bot is crawling my site just like GoogleBot does.
Following is the sample of url in my access logs.
72.14.192.3 - - [19/May/2010:01:27:06 +0000] "GET /some-url/etc-123.htm HTTP/1.1" 200 4707 "-" "AppEngine-Google; (+http://code.google.com/appengine; appid: harpy000)"
I have checked the ip address it is registered with Google Inc.
Can anyone tell me where i can report Abuse to Google Inc. Or any information about this issue.
Thank you!