"Error getting vgic maintenance irq from DT" for KVM at boot-up

Hello,
That is the error message I get when I start up a TX2 with the NVIDIA 4.4 kernel built from the ‘tegra18_defconfig’ .config file with KVM virtualization enabled. Boot-up continues on successfully, but once everything is up the /dev/kvm file never appears.

Trying the generic 4.10 kernel gets me past that step, but then boot-up crashes when I get to the real-time clock step.

Suggestions?

I’ve been guided to stay with the 4.4 kernel since it is known to work with the TX2, but it is very tempting to go with to the 4.10 kernel that I downloaded from kernel.org
Thanks

Dmesg output from 4.4:

[ 1.895249] kvm [1]: error getting vgic maintenance irq from DT
[ 1.895367] kvm [1]: error initializing Hyp mode: -6

Dmesg output from 4.10:

[ 1.118518] kvm [1]: 8-bit VMID
[ 1.121784] kvm [1]: IDMAP page: 808ec000
[ 1.125949] kvm [1]: HYP VA range: 4000000000:7fffffffff
[ 1.132160] kvm [1]: Hyp mode initialized successfully
[ 1.137561] genirq: Flags mismatch irq 1. 00004408 (kvm guest timer) vs. 00004401 (arch_timer)
[ 1.146516] kvm [1]: kvm_arch_timer: can’t request interrupt 1 (-16)
(lines omitted)
[ 1.395227] ima: No TPM chip found, activating TPM-bypass! (rc=-19)
[ 1.401805] evm: HMAC attrs: 0x1
[ 1.405232] hctosys: unable to open rtc device (rtc1)
(crash)

Are you trying to use the board as a virtualization host?

Yes. For Linux guest OSes.

I have once managed to run a guest OS within KVM at Jetson JTK1.
But it was rather tremendously slow.
" but the speed decrease is like a factor of 10000 or more" as quoted from https://forums.virtualbox.org/viewtopic.php?f=9&t=65426
I do not think Jetson is a best hit for virtualization.
However, nvidia has interesting solutions for virtualization. E.g. : http://www.nvidia.com/object/tesla-m6.html
References:
https://en.wikipedia.org/wiki/Hypervisor
http://www.virtualopensystems.com/en/solutions/guides/kvm-on-armv8/

Thanks for the responses.

The 10000x speed decrease is for x86 emulation. I am looking at virtualization of ARM-compiled software, and only at virtualization of the CPU. GPU virtualization is not under consideration.

Thank you for your update.
It seems that arm includes virtualization support since ARM7v-A. But initially ARM architecture didn’t support virtualization. (Paolino 2015)

References:
http://www-lia.deis.unibo.it/Courses/som1516/materiale/VOSYS_BolognaKVMARM_2_12_2015.pdf
http://www.virtualopensystems.com/en/solutions/guides/kvm-on-arm/

Do you need full virtualization, or can you live with cgroups/containers?

Containers are great, but not for our needs. Full virtualization.

Any news on this? I would also like to have KVM for ARMv8a guests on the TX2…

Hi,

Could you please let me know if the complete code required for KVM virtualization is ported? If not, do you plan to port the code? I am using Jetson TX1 linux kernel version 4.4 to enable hypervisor mode and it fails with error messages as mentioned by markummitchell

I was able to boot the arm cores in HYP mode and run QEMU/KVM guests with a small device tree modification. change the main interrupt controller section to read:

intc: interrupt-controller@3881000 {
                compatible = "arm,cortex-a15-gic";
                #interrupt-cells = <3>;
                interrupt-controller;
                reg = <0x0 0x03881000 0x0 0x00001000>,
                      <0x0 0x03882000 0x0 0x00002000>,
                      <0x0 0x03884000 0x0 0x00002000>,
                      <0x0 0x03886000 0x0 0x00002000>;
                interrupts = <GIC_PPI 9 (GIC_CPU_MASK_SIMPLE(4) | IRQ_TYPE_LEVEL_HIGH)>;
                status = "disabled";
        };

just guessed the 2 extra register addresses needed by the kernel.

screenshot of installing windows 10, we’ll see how it goes… https://www.dropbox.com/s/px6totbveutsuw0/2017-12-14-092531_2160x3780_scrot.png?dl=0

(of course you also have to change the kernel config to enable the KVM options and recompile it)

An x86_64 guest will not be able to leverage KVM; you’d need to try win10 for arm64 (is it available yet?) or some other ARM guest.

Still, if your DT patch makes /dev/kvm appear then it is an important step forward.

ah you’re right it’s not any faster than plain old qemu on x86_64

Thanks to gattis’s reply.

By following that, I am able to boot TX2 into Hyp mode and see the /dev/kvm.https://i.imgur.com/L1zCpqA.png

https://i.imgur.com/b8FM3gR.png

However, modprobe and modinfo can’t find any kvm.ko modules files, how can i fix that?

I used https://github.com/good5dog5/buildJetsonTX2Kernel to do native kernel compile.

Hi.
If you have CONFIG_KVM set to “y” rather than “m”, all the KVM functionality is built into the kernel image itself, so there is no separate “kvm.ko”.
I.e. there is nothing to fix, you have KVM already (as evidenced by /dev/kvm being present).

Thank you marcus_c!

I now encounter new problem: kvm_arm_vcpu_init Failed, invalid argument

The following command will cause the problem while without -enable-kvm will successful boot up the kernel.

./aarch64-softmmu/qemu-system-aarch64 -enable-kvm  -machine virt -cpu cortex-a57  -nographic -smp 1 -m 2048 -kernel aarch64-linux-3.15rc2-buildroot.img

Any suggestions?

Hi Jordan. Thanks for testing.

The KVM_ARM_VCPU_INIT ioctl will return invalid argument if the specified virtual CPU type (Cortex-A57 in your case) does not match the physical CPU.

Since two of the cores in the TX2 are Denver 2 rather than Cortex-A57, maybe the qemu process needs to be pinned to one of the actual A57 cores, such as core 0?

Could you please try

taskset 0x1 ./aarch64-softmmu/qemu-system-aarch64 ...

Hi marcus_c, It works like a charm!, thanks a lot! (I use nvpmodel to disable 2 Denver cores)

https://i.imgur.com/cYFL92n.png