Getting KVM to work on NVIDIA Jetson Nano

I tried to run a KVM virtualization thing (firecracker) on my Jetson NANO and it fails because /dev/kvm does not exist. I think it’s because jetson nano’s kernel was compiled without KVM support.

The closest I found was kvm-aosp-jetson-nano/README_Jetson_Nano_BSP.md at master · antmicro/kvm-aosp-jetson-nano · GitHub which instructs to recompile the kernel, but this is a very different kernel.

Is there a way to get the official kernel fork for Jetson Nano so I can just enable KVM and recompile?

Or is there an easier way to ket KVM working? Maybe I should just enable it somewhere?

You can get the kernel follow below document.

https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/kernel_custom.html#

I made all of this, then I tried substituting the /boot/Image with the new compiled Image and it booted with some kernel logs then didn’t start the desktop

I then tried also copying the dtb and modules, but this also did not work.

How do I update the jetson kernel with the newly compiled one? My jetson nano does not have emmc so I can’t flash and also I think flashing is too much. What if I want to test lots of kernels? Putting the micro SD in and out of the Nano for flashing would be tedious

If the Image build from source tree without any modification still can’t boot to desktop that could be the version incorrect.
You can check the version by cat /etc/nv_tegra_release then download the correct source for it.

lz@jetson:~$ cat /etc/nv_tegra_release
# R32 (release), REVISION: 5.1, GCID: 26202423, BOARD: t210ref, EABI: aarch64, DATE: Fri Feb 19 16:45:52 UTC 2021

how do I know which is the correct source for this? I though there was only one. On the tutorials there’s no mention on different sources

I’m building R32.5.1, as far as I understood, it should work on any Nvidia Jetson Nano

Do I really need to replace the dtb and modules, or should it work by just replacing the /boot/Image?

It’s working by just replace the /boot/Image.
Make sure export LOCALVERSION=-tegra when build the kernel Image

I tried again and unfortunately I get this:

I can’t even ssh to it (before the /boot/Image replacement it worked)

Could you check this Image that build from my host.

I found the problem.

Compiling the kernel without any modifications, work. That is, it boots correctly.

However, if I do

echo "CONFIG_KVM=y
CONFIG_VHOST_NET=m" >> arch/arm64/configs/tegra_defconfig

to activate KVM support on the kernel, then this kernel does not boot

But I need KVM support

Have reference to this doc to try.

thanks. This is for the TX2, I’m trying to adapt for my Jetson Nano but I don’t know which .dtb I need to decompile, there are a lot of them:

lz@jetson:~$ ls /boot
dtb
extlinux
grub
Image
initrd
initrd.img
initrd.img-4.9.201-tegra
OriginalImage2
OriginalImage3
tegra210-jetson-tx1-p2597-2180-a01-devkit-adafruit-sph0645lm4h.dtbo
tegra210-jetson-tx1-p2597-2180-a01-devkit.dtb
tegra210-jetson-tx1-p2597-2180-a01-devkit-fe-pi-audio.dtbo
tegra210-jetson-tx1-p2597-2180-a01-devkit-hdr40.dtbo
tegra210-jetson-tx1-p2597-2180-a01-devkit-respeaker-4-mic-array.dtbo
tegra210-jetson-tx1-p2597-2180-a02-devkit-24x7.dtb
tegra210-p3448-0000-p3449-0000-a00.dtb
tegra210-p3448-0000-p3449-0000-a01-adafruit-sph0645lm4h.dtbo
tegra210-p3448-0000-p3449-0000-a01.dtb
tegra210-p3448-0000-p3449-0000-a01-fe-pi-audio.dtbo
tegra210-p3448-0000-p3449-0000-a01-hdr40.dtbo
tegra210-p3448-0000-p3449-0000-a01-mcp251x.dtbo
tegra210-p3448-0000-p3449-0000-a01-respeaker-4-mic-array.dtbo
tegra210-p3448-0000-p3449-0000-a02-adafruit-sph0645lm4h.dtbo
tegra210-p3448-0000-p3449-0000-a02.dtb
tegra210-p3448-0000-p3449-0000-a02-fe-pi-audio.dtbo
tegra210-p3448-0000-p3449-0000-a02-hdr40.dtbo
tegra210-p3448-0000-p3449-0000-a02-mcp251x.dtbo
tegra210-p3448-0000-p3449-0000-a02-respeaker-4-mic-array.dtbo
tegra210-p3448-0000-p3449-0000-b00.dtb
tegra210-p3448-0002-p3449-0000-a02.dtb
tegra210-p3448-0002-p3449-0000-b00.dtb
tegra210-p3448-0003-p3542-0000-adafruit-sph0645lm4h.dtbo
tegra210-p3448-0003-p3542-0000.dtb
tegra210-p3448-0003-p3542-0000-fe-pi-audio.dtbo
tegra210-p3448-0003-p3542-0000-hdr40.dtbo
tegra210-p3448-0003-p3542-0000-respeaker-4-mic-array.dtbo

mine is the devkit jetson nano, which would be p3448, but there are lots of file with its name

since I don’t know which dtb to decompile, I edited them all from the kernel source (that is, I edited the main one and generated all them) and placed them into /boot. That is, I replace the old ones. However, doing hexedit /proc/device-tree/interrupt-controller reg shows that it did not alter the regs I placed. Decompiling each of the .dtb from /boot shows that they are in fact there.

Where does jetson nano take the dtb files from if not /boot?

You can check which dtb been loaded by Nano from below command.

dmesg | grep -i kernel

this gives

lz@jetson:~/Linux_for_Tegra/source/public$ dmesg | grep -i kernel
[    0.000000] Kernel command line: tegraid=21.1.2.0.0 ddr_die=4096M@2048M section=512M memtype=0 vpr_resize usb_port_owner_info=0 lane_owner_info=0 emc_max_dvfs=0 touch_id=0@63 video=tegrafb no_console_suspend=1 console=ttyS0,115200n8 debug_uartport=lsport,4 earlyprintk=uart8250-32bit,0x70006000 maxcpus=4 usbcore.old_scheme_first=1 lp0_vec=0x1000@0xff780000 core_edp_mv=1075 core_edp_ma=4000 gpt tegra_fbmem=0x800000@0x92ca9000 is_hdmi_initialised=1  earlycon=uart8250,mmio32,0x70006000  root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 console=ttyS0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0  
[    0.000000] Memory: 3559708K/4159488K available (15486K kernel code, 2962K rwdata, 7052K rodata, 8640K init, 609K bss, 124644K reserved, 475136K cma-reserved)
[    0.000000] Virtual kernel memory layout:
[    0.022544] kmemleak: Kernel memory leak detector disabled
[    0.236710] DTS File Name: /home/lz/Linux_for_Tegra/source/public/kernel/kernel-4.9/arch/arm64/boot/dts/../../../../../../hardware/nvidia/platform/t210/porg/kernel-dts/tegra210-p3448-0000-p3449-0000-a00.dts
[    0.349717] alternatives: patching kernel code
[    0.449149] DTS File Name: /home/lz/Linux_for_Tegra/source/public/kernel/kernel-4.9/arch/arm64/boot/dts/../../../../../../hardware/nvidia/platform/t210/porg/kernel-dts/tegra210-p3448-0000-p3449-0000-a00.dts
[    0.594271] eventlib_kernel: keventlib is initialized, test id: 0
[    5.416625] KERNEL: PMC reset status reg: 0x3
[    5.994490] Freeing unused kernel memory: 8640K

how can it be loading a dts file from

DTS File Name: /home/lz/Linux_for_Tegra/source/public/kernel/kernel-4.9/arch/arm64/boot/dts/../../../../../../hardware/nvidia/platform/t210/porg/kernel-dts/tegra210-p3448-0000-p3449-0000-a00.dts

?

this is local.

That tell the file is tegra210-p3448-0000-p3449-0000-a00.dtb

this is weird, it’s loading a local file, not the one at root. I think this has to do with the fact that I compiled the kernel locally, and it’s linking to the old place of the device tree files. Do you know how to change that? I had to do this on /boot/extlinux/extlinux.conf

LABEL primary
      MENU LABEL kvm device tree with gic
      LINUX /boot/Image
      INITRD /boot/initrd
      FDT /boot/tegra210-p3448-0000-p3449-0000-a00.dtb
      APPEND ${cbootargs} quiet root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 loglevel=7 console=ttyS0,115200n8 console=tty0 fbco$

but it should just be a matter of replacing the device tree binary files on /boot

Should be fine to use FDT to load desired dtb file.
Also you can update by flash command.

  1. Replace the dtb file located …/Linux_for_Tegra/kernel/dtb
  2. sudo ./flash -r -k DTB jetson-nano-xxx mmcblk0p1

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.