How to protect/monitor your site from crawling by malicious user
- by deathy
Situation:
Site with content protected by username/password (not all controlled since they can be trial/test users)
a normal search engine can't get at it because of username/password restrictions
a malicious user can still login and pass the session cookie to a "wget -r" or something else.
The question would be what is the best solution to…