Blocking Just the Parent Domain via robots.txt
- by Bryan Hadaway
Let's say you have a parent domain:
parent.com
and children subdomains under that parent domain:
child1.com
child2.com
child3.com
Is there a way to use just the following within parent.com:
User-agent: *
Disallow: /
Considering each child has their own robots.txt stating:
User-agent: *
Allow: /
Or is the parent robots.txt still going to have to make an exception for every single subdomain:
User-agent: *
Disallow: /
Allow: /child1/
Allow: /child2/
Allow: /child3/
Obviously this is important and tricky territory SEO wise so I'm looking to learn the definitive and safe, best practice method here to sharpen my skills.
Thanks, Bryan