I am running VMWare Server 2.0.2 (Build 203138) on a dual core Intel i5 with Ubuntu Server 10.04 LTS system (kernel 2.6.32-22-server #33-Ubuntu SMP). Disk Subsystem is a software RAID5 array.

The system has been set up for a little over a week. For the past 5 days I have been running at leat 3 VMs (Linux and a variety of Windows OSes) with no issues whatsoever. But while I was installing Linux onto a new VM, suddenly all VMs became unresponsive, including the one I was installing to. I could not log in to the VMWare Management Interface, and the system was somewhat unresponsive via SSH. When I looked at top, I saw:

top - 16:14:51 up 6 days,  1:49,  8 users,  load average: 24.29, 24.33 17.54
Tasks: 203 total,   7 running, 195 sleeping,   0 stopped,   1 zombie
Cpu(s):  0.2%us, 25.6%sy,  0.0%ni, 74.3%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   8056656k total,  5927580k used,  2129076k free,    20320k buffers
Swap:  7811064k total,   240216k used,  7570848k free,  5045884k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                                                                                                               
21549 root      39  19     0    0    0 Z  100  0.0  15:02.44 [vmware-vmx] <defunct>
 2115 root      20   0     0    0    0 S    1  0.0 170:32.08 [vmware-rtc]
 2231 root      21   1 1494m 126m 100m S    1  1.6 892:58.05 /usr/lib/vmware/bin/vmware-vmx -# product=2;                                                                                                                                                                                       
 2280 jnet      20   0 19320 1164  800 R    0  0.0  30:04.55 top 
12236 root      20   0  833m  41m  34m S    0  0.5  88:34.24 /usr/lib/vmware/bin/vmware-vmx -# product=2;
    1 root      20   0 23704 1476  920 S    0  0.0   0:00.80 /sbin/init                                                                                                                                                                                             
    2 root      20   0     0    0    0 S    0  0.0   0:00.01 [kthreadd]                                                                                                                                                                                             
    3 root      RT   0     0    0    0 S    0  0.0   0:00.00 [migration/0]                                                                                                                                                                                          
    4 root      20   0     0    0    0 S    0  0.0   0:00.84 [ksoftirqd/0]                                                                                                                                                                                          
    5 root      RT   0     0    0    0 S    0  0.0   0:00.00 [watchdog/0]                                                                                                                                                                                           
    6 root      RT   0     0    0    0 S    0  0.0   0:00.00 [migration/1]                                                                                                                                                                                          

The VMWare process for the virtual machine I was installing into became a zombie. Yet, it was still consuming 100% of the CPU time on one of the cores, and I couldn't reach it or any other virtual machines. (I was logged in to one virtual machine over SSH, another via X11, and a third via VNC. All three connections died). When I ran ps -ef and similar commands, I found that the defunct vmware-vmx process had it's parent PID set to init (1). I also used lsof -p 21549 and found that the defunct process had no open files. Yet it was using 100% of CPU time...

I was unable to kill any vmware-vmx processes, including the defunct one, even with kill -9. As a last resort to resolve the situation I tried to reboot the box, however shutdown, halt, reboot, and init 6 all failed to reboot/shutdown, even when given appropriate --force settings. ControlAltDel produced a message about rebooting on the console, but the system would not reboot. I had to hard power-cycle the box to resolve the situation. (See my other question, Should I worry about the integrity of my linux software RAID5 after a crash or kernel panic?)

What would cause a scenario like this? What else could I have done to resolve it besides a hard reboot? What can I do to prevent such a situation in the future?