Jetson Nano board setup

Adding a module which was compiled against the exiting kernel’s configuration is just a file copy, followed by either rebooting or using the right command to load the module.

Adding a new kernel implies also needing to add all new modules, although there are some things you can cheat with and sometimes get away with using the same modules.

Adding a new kernel can itself be done either with a file copy or flashing. You should know that there are alternate boot designs where you can boot from a kernel in a partition (especially useful if you are burning security fuses and wish to guarantee your kernel did not change), but for the most part you are better off just using a file copy in combination with an edit of “/boot/extlinux/extlinux.conf”.

If the kernel is found from the extlinux.conf, then that is booted. Otherwise it looks for a partition based kernel. It is far easier and lower risk to add a new kernel with a new name in “/boot”, along with extlinux.conf edits, than it is to flash. However, there is a good chance you don’t even need to modify the kernel, but would instead only add modules.

The real value of this thread is if you have set up serial console and understand picking kernels at boot via new extlinux.conf entries. Combine this with understanding how a module can be built against the existing kernel and you have most of what you need at very low risk for failure.

If you look at “/boot/extlinux/extlinux.conf” you will see a block similar to this (which is actually from an NX and not a Nano):

LABEL primary
      MENU LABEL primary kernel
      LINUX /boot/Image
      INITRD /boot/initrd
      APPEND ${cbootargs} root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 console=ttyTCU0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0

Note that the “MENU LABEL” is what you’d see if you interrupt boot at the right moment using serial console. The first block would name that menu label, and number it. The actual kernel would be the path from “LINUX”. If you were to duplicate this block and slightly alter the block, then you could have both kernels available, and simply pick at boot with serial console which to load. Example:

LABEL primary
      MENU LABEL primary kernel
      LINUX /boot/Image
      INITRD /boot/initrd
      APPEND ${cbootargs} root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 console=ttyTCU0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0

LABEL testing
      MENU LABEL test kernel
      LINUX /boot/Image-test
      INITRD /boot/initrd
      APPEND ${cbootargs} root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 console=ttyTCU0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0

In the above the original “primary” kernel would be selection “1” for picking kernels, and the “testing” kernel would be available as well. Since the “DEFAULT primary” (further towards the top of the extlinux.conf file) is the same, then if you don’t interrupt the boot and pick something else, it would continue on booting with the original kernel.

If it turns out your test kernel works, then you could change the default:
DEFAULT testing
(notice that MENU LABEL is what the menu presents, but the actual LABEL is what the computer sees)

Moving on to how the kernel itself finds modules is also useful. After booting any kernel can report its version via the command “uname -r”. The left hand side of that version is from the base kernel source version. The rest is from an option which is set at the time of compile, “CONFIG_LOCALVERSION”. If your kernel source is version 4.9.140, and if you have this CONFIG_LOCALVERSION at the time of compile, your “uname -r” will be “4.9.140-tegra”:
CONFIG_LOCALVERSION="-tegra"

It happens that the “-tegra” is what NVIDIA uses by default.

The search location for modules will always be here:
/lib/modules/$(uname -r)/
(actual content will be in the “kernel/” subdirectory of that)

If you have two kernels which are an exact duplicate other than the “uname -r”, then the one with the altered “uname -r” will fail to find its modules.

If you have two kernels with the same “uname -r”, but configurations are not compatible, then your new kernel would attempt to use the old modules, but probably fail in places due to the lack of matching configuration.

If you have two kernels with the same “uname -r”, and they start with the same configuration, but you add additional features without removing the old configuration’s features, then very likely your new kernel will function without any error using the old modules. You might need to copy a module file into the right place within “/lib/modules/$(uname -r)/kernel/” for the new features you added, but that is trivial.

Very conveniently, when you boot a given kernel on the Jetson, a pseudo file (not really on the hard drive/eMMC/SD, but a reflection in RAM generated by the kernel and pretending to be a file) will be a “nearly” exact match to your running kernel:
/proc/config.gz
(you could copy config.gz somewhere else, and then “gunzip config.gz” to see its content)

By “nearly” exact I mean everything is an exact match with the exception of “CONFIG_LOCALVERSION” typically not being set. Most configuration items in this file are dangerous to hand edit (config editors know about dependencies), but CONFIG_LOCALVERSION has no dependencies, so you could hand edit this feature to match the running system:
CONFIG_LOCALVERSION="-tegra"

Then if you were to compile the full kernel it would be a perfect duplicate of the original and use the original modules. If you were to further edit this (preferably with a config editor) to add new modules, then you wouldn’t even need to replace the kernel…you could simply copy the files into the right place under “/lib/modules/$(uname -r)/kernel/”. If you did change something which was not a module, and merely added a feature with the rest of the features remaining unchanged, then the new kernel could be added, e.g., as “LINUX /boot/Image-testing”, and it would use the same modules (plus any new module needing to be copied in).

Any time you use extlinux.conf such that the old boot is still present it is quite easy to recover from a failed kernel. Any time you merely copy in a module file, then it is also quite easy to fix issues with the module file. Any time you remove the old kernel or flash to a partition it is likely a failure means reflashing the entire system and losing most everything. There are tricks you might pull like cloning, fixing the clone, and the flashing with the clone, but this takes a lot of disk space and time.

Incidentally, if you were to build a module from kernel source, and the module (after build) appears as a “.ko” file in subdirectory “drivers/usb/”, e.g., if it is “drivers/usb/sample.ko”, then in the module tree (based at “/lib/modules/$(uname -r)/kernel”) you would copy sample.ko to that same subdirectory tree:

/lib/modules/$(uname -r)/kernel/drivers/usb/sample.ko`

Commands can then be used to tell the kernel to update its knowledge of that file and load it, or you could just reboot.

Before telling you about building kernels, do you really need a full kernel install, or just a module? Not all features are available as a module, but most are. What feature/symbol do you need (a “symbol” is just the Kconfig system’s way of naming a feature)?