I'm running Linux Mint 10, although I've had this same issue with other variants of Linux. I've been told/found while researching that if the X server hangs or otherwise errors, one can drop to a root prompt, usually at another tty, and execute init 3 (to drop to single user mode) and then init 5 to return to the default, graphical session.
Needless to say, I've tried this before in multiple configurations on multiple machines to no avail. The only feedback I receive form executing those two commands is a listing of VMWare services (from a kernel module) that are stopped and then restarted.
Note: If I run startx (either before or after init 3), then I am told that the xserver is still running and that I should remove /tmp/.X0-lock. Having tried that, it removes that error message, but claims that the xserver cannot be attached as another instance is running.
How do I kill the xserver completely? Can I killall some process name?