Google Sitemap and Robots.txt Issue

Posted by Sarfaraz Soomro on Stack Overflow See other posts from Stack Overflow or by Sarfaraz Soomro
Published on 2009-09-28T08:16:17Z Indexed on 2010/03/27 21:03 UTC
Read the original article Hit count: 342

Filed under:
|
|

Hi,

We have a sitemap at our site, http://www.gamezebo.com/sitemap.xml

Some of the urls in the sitemap, are being reported in the webmaster central as being blocked by our robots.txt, see, gamezebo.com/robots.txt ! Although these urls are not Disallowed in Robots.txt. There are other such urls aswell, for example, gamezebo.com/gamelinks is present in our sitemap, but it's being reported as "URL restricted by robots.txt".

Also I have this parse result in the Webmaster Central that says, "Line 21: Crawl-delay: 10 Rule ignored by Googlebot". What does it mean?

I appreciate your help,

Thanks.

© Stack Overflow or respective owner

Related posts about google

Related posts about sitemap