Hiding php includes from search spiders?
- by 21stcn
Quick and simple question.
I have 80+ html files which I want to be crawled. They are individual product pages. Each of these pages calls its content using php includes. These php include files are in a separate folder on the server and contain the core content for the individual product pages.
I just wanted to ask, if I use robots.txt or…