robots.txt, how effective is it and how long does it take?

Posted by Stefan on Pro Webmasters See other posts from Pro Webmasters or by Stefan
Published on 2012-07-09T22:18:15Z Indexed on 2012/07/10 3:23 UTC
Read the original article Hit count: 252

Filed under:

We recently updated the site to a single page site using jQuery to slide between "pages". So we now have only index.php.

When you search the company on engines such as Google, you get the site and a listing of its sub pages which now lead to outdated pages.

Our plan doesn't allow us to edit the .htaccess and the old pages are .html docs so I cannot use PHP redirects either.

So if I put in place a robots.txt telling the engines to not crawl beyond index.php, how effective will this be in preventing/removing crawled sub pages.

And rough guess, how long before the search engines would update?

© Pro Webmasters or respective owner

Related posts about robots.txt