Is there any advantage/disadvantage to using robots.txt to disallow access to legal pages such as terms, privacy policy, etc.?
- by CaptainCodeman
As I understand, having repetitive content is a detriment to search engine placement.
Given that many websites that use similar or even identical "Terms and Conditions" and "Privacy Policy" pages due to similar legal wording or due to copy & pasting from the same source, would it be a good idea to disallow access to these pages via robots.txt, in order to avoid being penalized for "non-original content"?
Or, on the contrary, could the search engines identify this as circumvention and penalize the site for trying to hide content?
Or does it not matter?