I'm analyzing web server logs both in Apache and IIS log formats. I want to find the evidence for automatic browsing, like web robots, spiders, bots, etc. I used python robot-detection 0.2.8 for detecting robots in my log files, but I know there may be other robots (automatic programs) which have traversed through the web site but robot-detection can not identify.

So I want to ask:

1. Are there any specific clues that can be found in log files that human users do not leave but automated software would?
2. Do they follow a specific navigation pattern?
3. I saw some requests for favicon.ico - does this implicate that it is a automatic browsing?.

I found this article and this question with some valuable points.