Get Illegal Instruction error when booting Linux in VirtualBox, works fine when booted directly

Posted by rkjnsn on Super User See other posts from Super User or by rkjnsn
Published on 2012-10-21T01:25:25Z Indexed on 2012/10/22 23:05 UTC
Read the original article Hit count: 311

Filed under:
|

I have a computer on which I am dual booting Windows 7 and Gentoo Linux (both 64-bit). I want to be able to load up my Linux installation in a VM while I am booted into Windows. I have installed VirtualBox and followed the instructions for creating a raw disk VMDK. When I start the VM, Linux starts booting, but then fails with the following error when unlocking my root partition:

truecrypt[441] trap invalid opcode ip:373615538e0 sp:3dd0e0dfb60 error:0 in libpixman-1.so.0[373614d6000+8d000]

Everything works fine when I boot into Linux directly.

What could cause an illegal instruction to be hit in libpixman only when booting in VirtualBox?

Update:

As a troubleshooting step, I recompiled pixman without "-march", and no longer get an illegal instruction error in that library. (The boot fails in the same spot with the same error in a different library, however.)

How can I determine the specific opcode that isn't working in VirtualBox so I can disable it in my CFLAGS without having to disable all CPU-specific optimizations?

I am still confused as to why there would be any user-mode instruction that would fail to work in a VM. Is this a known limitation? My CPU is an Intel Core i7 3720QM, and I have hardware virtualization support enabled.

© Super User or respective owner

Related posts about linux

Related posts about virtualbox