Using SSH to find access to a problematic script in logs of multiple domains

Posted by Hanan Cohen on Server Fault See other posts from Server Fault or by Hanan Cohen
Published on 2011-11-20T09:47:30Z Indexed on 2011/11/20 9:58 UTC
Read the original article Hit count: 388

Filed under:

ssh

|

webserver

|

logging

|

scripting

|

dreamhost

I run several (~20) sites on a Dreamhost VPS. Lately I max my memory allocation for the VPS and I want to find the problem.

I would like to have an SSH script that will scan all the log files of all the domains and show me what object (image, php script etc) gets lots of calls.

It will count the calls in each /logs/*/http/access.log, do an descending sort and show me the top 10 across domains.

But I don't know how to do that.

Can it be done?

Can anyone suggest a script that will do that?

Thanks.

(Cross posted to Stack Overflow)

© Server Fault or respective owner

Related posts about ssh

Problem with hadoop start-dfs.sh

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I installed and configured hadoop on my Ubuntu 14.04 server, virtualized inside of hyper-v, however I am getting an issue when i run start-dfs.sh root@sUbuntu01:/var/log# start-dfs.sh 14/06/04 15:27:08 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java… >>> More
SSH problems (ssh_exchange_identification: read: Connection reset by peer)

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I was running 11.10 and decided to do the full upgrade and come up to 12.04 after the update SSH (not SSHD) is now misbehaving when attempting to connect to other OpenSSH instances. I say OpenSSH as I am running a DropBear sshd on my router and I am able to connect to it. When attempting to connect… >>> More
SSH main process ended

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I have a running ubuntu server 10.04.1. When I tried to login to the server via ssh, I could not. Instead, I got connection refused error. I tried to ping the machine and I got reply! So, the clear reason is that SSH daemon is stopped. After reboot, I was able to login to my server via ssh. After… >>> More
SSH server not working (respawns until stopped)

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I have a running Ubuntu Server 10.04.1. When I tried to login to the server via ssh, I could not. Instead, I got connection refused error. I tried to ping the machine and I got reply! So, the clear reason is that SSH daemon is stopped. After reboot, I was able to login to my server via ssh. After… >>> More
Cannot SSH after resetting firewall on VPS

as seen on Server Fault - Search for 'Server Fault'
I'm having trouble trying to SSH to my Debian 5 VPS with blacknight. It was working fine until I did the following: Logged into 'Parallels Infrastructure Manager' - Container - Firewall - Set to 'Normal Firewall settings'. It told me there was an error with the IPTables and offered the option again… >>> More

Related posts about webserver

Headers to set for AJAX calls in a custom webserver

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm writing a custom web server. When I enter the URL of my server in the browser, I get the sample text I write out to the socket in my browser correctly. This is the HTTP response that I write: HTTP/1.1 200 OK\r\n Server: My Server\r\n Date: Blah\r\n \r\n This is some sample text. This appears… >>> More
Erlang: 2 database on 2 webserver ??

as seen on Stack Overflow - Search for 'Stack Overflow'
I have created a blogging system with php+postgresql. Now I want to add a web chat ( in REAL TIME for Million of users simultaneously ) where every message is saved in database. I am thinking to use Erlang+Mnesia on a different webserver for this issue. Message's table will be like this: message_id… >>> More
Why is this mod_rewrite RewriteRule directive not working in the .htaccess file?

as seen on Server Fault - Search for 'Server Fault'
I've got a site that was hosted on a linux el cheapo hosting service that I'm migrating to my Mac OS X 10.5 Leopard Server server running Apache 2.2.8 & PHP 5.2.5 w/rewrite_module enabled and AllowOverride All, but I'm running into an issue with the following lines in the .htaccess file: RewriteEngine… >>> More
Why is this mod_rewrite RewriteRule directive not working in the .htaccess file?

as seen on Server Fault - Search for 'Server Fault'
I've got a site that was hosted on a linux el cheapo hosting service that I'm migrating to my Mac OS X 10.5 Leopard Server server running Apache 2.2.8 & PHP 5.2.5 w/rewrite_module enabled and AllowOverride All, but I'm running into an issue with the following lines in the .htaccess file: RewriteEngine… >>> More
How ISP or dns server find the nameserevr [on hold]

as seen on Pro Webmasters - Search for 'Pro Webmasters'
I saw some articles about how DNS propagation happens.I know that ISP or DNS server(such as google public dns) cache the ip address of website which it uses to convert domain name to ip address. But my doubt is from where these ISP or dns serevr know which nameserver to go for particular domain name… >>> More