How do I control how often search engines visit my site?
- by Nick
I've been using the following line in the <head> of my sites for years:
<meta name="revisit-after" content="3 days" />
I recently discovered that it's not one of the meta tags that Google understands, which I take to mean that there's no point in including it, and that it's been doing no good at all for years.
How often do search engines crawl a website by default, and what reliable ways are there to increase or decrease that frequency?
