Valid robots.txt? [closed]
- by psot
I am waiting for Google to crawl my site and display the results in search. Is my robots.txt alright and will it let google, bing etc crawl my site? Thanks!
User-agent: *
Disallow: /cgi-bin/
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /wp-content/
Disallow: /build/
Disallow: /css/
Disallow: /trackback/
Disallow: /comments
Disallow: /assets/graphics/
Disallow: /assets/visual/
Disallow: /category/*/*
Disallow: */trackback
Disallow: */feed
Disallow: */comments
Disallow: /*?*
Disallow: /*?
User-agent: Slurp
Disallow: /
User-agent: Baiduspider
Disallow: /
User-agent: ia_archiver
Disallow: /
User-agent: duggmirror
Disallow: /
User-agent: Yandex
Disallow: /
Sitemap: http://example.com/sitemap.xml.gz