We have a RHEL server. Today it suddenly became slow. Symptoms - It was responding slow to ping queries from other server. When I try to login using ssh, it was taking about 10 seconds to login. I was able to resolve the problem by doing some guess work. I killed one process which I thought was culprit. Which resolved the problem. Though I would like to know what's proper approach to detect the culprit in such kind of "slow server" situations. Le me know proper way to resolving such slowness issues and decting the process causing the slowness.
These were the conditions when the server was slow -
# vmstat 3 3
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
1 1 176 6730868 285052 4899676 0 0 3 4 0 0 1 1 97 1 0
0 0 176 6751576 285064 4899704 0 0 0 115 15307 37171 1 1 96 3 0
0 0 176 6751948 285068 4899700 0 0 0 23 14813 39559 1 1 98 1 0
# top
top - 16:38:18 up 150 days, 19:36, 64 users, load average: 1.68, 1.46, 1.44
Tasks: 1287 total, 2 running, 1284 sleeping, 1 stopped, 0 zombie
Cpu(s): 1.3%us, 1.7%sy, 0.1%ni, 95.9%id, 0.7%wa, 0.0%hi, 0.2%si, 0.0%st
Mem: 16620824k total, 9867124k used, 6753700k free, 287424k buffers
Swap: 8193140k total, 176k used, 8192964k free, 4898996k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
26258 khk 34 19 130m 47m 7088 S 11.2 0.3 385:32.42 edm
Though I would like to know what's proper approach to detect the culprit in such kind of "slow server" situations. Le me know proper way to resolving such slowness issues and decting the process causing the slowness.