Googlebot substitutes the links of Rails app with subdomain.

Posted by Victor on Stack Overflow See other posts from Stack Overflow or by Victor
Published on 2010-12-29T02:35:49Z Indexed on 2010/12/29 2:54 UTC
Read the original article Hit count: 238

I have this Rails app, with domain name abc.com. I am also having a separate subdomain for Piwik stats, in this subdomain stats.abc.com.

Googlebot somehow listed some of the links with my subdomain too.

http://abc.com/login
http://stats.abc.com/login
http://abc.com/signup
http://stats.abc.com/signup

The ones with stats will reference to the same page in the app, but are treated entirely different website.

I have put in robots.txt in stats after this matter, but wondering if there is any appropriate way to block this because I may have new subdomains in future. Here's my content in robots.txt

User-agent: *
Disallow: /

Thanks.

© Stack Overflow or respective owner

Related posts about ruby-on-rails

Related posts about robots.txt