I have zsh configured to use vi keybindings. I've noticed some unexpected behavior with "escape". In vim (I haven't tested vanilla vi) if I hit escape twice, I can hit 'i' once to return to insert mode. In zsh if I happen to hit escape twice, hitting 'i' won't return me to insert mode, I have to hit it twice. Another example of this comes up in navigation. If I hit escape once, I can use '^' and '$' as expected. But if I've accidentally hit escape twice (or more) they fail to work until I return to insert mode and escape again.

Do I somehow have zsh configured incorrectly, or is this just a known difference in zsh's vi emulation?