Is my robots.txt working as it should?

Posted by TigerBlood on Pro Webmasters See other posts from Pro Webmasters or by TigerBlood
Published on 2011-03-04T21:22:41Z Indexed on 2011/03/04 23:33 UTC
Read the original article Hit count: 311

Filed under:

I want crawlers to have access to http://www.example.com but not http://www.example.com/

My robots.txt is as follows:

User-agent: *
Allow: /$
Disallow: /

My site is in google search results, but I am not coming up in Bing, Yahoo, etc. I have had the same robots.txt since last year, and I initially requested inclusion ~1 year ago, having also resubmitted the URL to those latter search engines several times since as well.

Is my robots.txt blocking those other crawlers? And if so, why not google as well?

Thanks in advance!

© Pro Webmasters or respective owner

Related posts about robots.txt