[Disallow: /index.php] seems to block /my-beautiful-sef-url-123
- by Jaroslav Záruba
Hello
I have robots.txt that looks like this:
User-agent: *
Disallow: /system/
Disallow: /admin/
Disallow: /index.php
The obvious goal has been to prevent all the ugly URLs from being indexed, as they all begin with "/index.php".
But for some reason all URLs like /my-beautiful-sef-url-123 are listed under Crawl errors in Google Webmaster Tools with "URL restricted by robots.txt". (When I test such URL it yields Allowed for both Googlebot and Googlebot-Mobile.)
Can anyone help please?