We have a RHEL server. Today it suddenly became slow. Symptoms - It was responding slow to ping queries from other server. When I try to login using ssh, it was taking about 10 seconds to login. I was able to resolve the problem by doing some guess work. I killed one process which I thought was culprit. Which resolved the problem. Though I would like to know what's proper approach to detect the culprit in such kind of "slow server" situations. Le me know proper way to resolving such slowness issues and decting the process causing the slowness.

These were the conditions when the server was slow -

# vmstat 3 3
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 1  1    176 6730868 285052 4899676    0    0     3     4    0    0  1  1 97  1  0
 0  0    176 6751576 285064 4899704    0    0     0   115 15307 37171  1  1 96  3  0
 0  0    176 6751948 285068 4899700    0    0     0    23 14813 39559  1  1 98  1  0

# top

top - 16:38:18 up 150 days, 19:36, 64 users,  load average: 1.68, 1.46, 1.44
Tasks: 1287 total,   2 running, 1284 sleeping,   1 stopped,   0 zombie
Cpu(s):  1.3%us,  1.7%sy,  0.1%ni, 95.9%id,  0.7%wa,  0.0%hi,  0.2%si,  0.0%st
Mem:  16620824k total,  9867124k used,  6753700k free,   287424k buffers
Swap:  8193140k total,      176k used,  8192964k free,  4898996k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
26258 khk       34  19  130m  47m 7088 S 11.2  0.3 385:32.42 edm

Though I would like to know what's proper approach to detect the culprit in such kind of "slow server" situations. Le me know proper way to resolving such slowness issues and decting the process causing the slowness.