Following the instructions in this thread, I added the Dvorak layout to the Regional and Language Options control panel, set it as the default keyboard layout and removed the US (QWERTY) layout.

However, even though I removed the QWERTY layout, it still appears in my language bar, and my system defaults to it in every new window. This persists after a log-out/log-in and even a system restart.

How do I remove the QWERTY keyboard layout from my system permanently?

Alternatively (if outright removing QWERTY is just impossible), can I get Windows to default to Dvorak instead of QWERTY for new windows?