Issues Building Custom Kernel 36.4 new Jetson Orin Nano Dev Kit

Robotics & Edge Computing Jetson & Embedded Systems Jetson Orin Nano
23

Hi @DaneLLL

I have already tried that, that works well, but I need those additional modules I mentioned.

Hi @linuxdev

I don’t add any code, or do any modification to the kernel code, I just add some modules to the kernel config.

I generated my defconfig with the make menuconfig command, but when I go for another try, I replace the default defconfig with the previously generated one, could this be the issue?

About the compiler, I have been using the suggested GCC from the Jetson Linux realease, and it seems to work well.

I’ll try again, with everything from zero and using only the make commands, I’ll let you know the outcome, thank you.

24

Hi,
We would suggest edit the default defconfig to add/remove kernel configs. If the issue persists, suggest add/remove the configs one by one, to clarify which config triggers the issue.

1 Like
25

I will do this, let’s see if I can find out which modules are in conflict, thank you for your support @DaneLLL .

26

Hi @DaneLLL @EduardoSalazar96 @linuxdev

I have an update, I added just a couple of modules, not all of them, and very importantly, interacting with the config files ONLY with the menuconfig.

I have successfully generated the Image file (41MB) and also compiled the modules make modules correctly.
However, as make -C kernel always rewrites defconfig to .config, I have compiled directly in the kernel-jammy-src directory, not really an issue.

Just to make sure the following steps are correct, if I want to generate an image ready to flash to the SD card, would I have to execute these commands?
Just setting -b to “jetson-orin-nano-devkit”, and -r 300?

sudo ./apply_binaries.sh -r rootfs
sudo ./tools/jetson-disk-image-creator.sh -o sdcard.img -b jetson-nano -r 300

I got these commands from this blog post, it is from 2021, I guess it should still work, since the script is still included in the last release of Linux_for_Tegra: Compiling Custom Kernel Modules on the Jetson Nano | Kevin’s Blog

27

Hi,
Generating SD card image is a further query. Please create a new topic thread for clearness.

1 Like
28

Hi,

After the flashing of this Image, obtained as described in my previous post, I did not get many errors in the systemd init, in fact, I could add the modules I included in the config file of the kernel with the command modprobe.

However, it seems to not be complete the installation, because the OS can’t communicate with the Nvidia drivers in order to execute for example a nvidia-smi command, or with the ethernet or Wi-Fi drivers for connecting to the internet.

I have also noticed that in /boot I have an Image file, and an initrd.img-5.15.148, but in the /lib/modules directory, I have two directories 5.15.148 and 5.15.148-tegra, if I execute a uname -r, I get the modules from 5.15.148. Does this installation look okay?

So I don’t know if I am still missing something in the compilation of the kernel or if it is now a problem with the flashing of the image…

Kind regards.

29

Generating an original defconfig with menuconfig will invalidate the kernel config. You have to use the actual defconfig, or some known working configuration. Then, and only then, modify this with “make menuconfig”. Was a good working configuration set up prior to “make menuconfig”? If you started with the actual defconfig from the mainline kernel, then this is correct; if you started with menuconfig, then the configuration is invalidated.

It isn’t so much about “previous one” as it is using a known working configuration for particular hardware. If you’ve modified the actual defconfig file without menuconfig, then you probably need to download the kernel source again. From your description I cannot tell which of these two specific points exist:

  • You used the actual defconfig and modified this only with menuconfig (which ok).
  • You created something with menuconfig and called it your defconfig (which will fail).

Short answer: Always use a known working configuration, and then edit. defconfig, if you have the original, or one which was edited and tested as known working, can be used after further menuconfig edits. If and only if your system boots, then you can also use the running system’s configuration found at “/proc/config.gz” (you would still need to set CONFIG_LOCALVERSION depending on circumstances).

Incidentally, if it compiles, then a particular compiler is likely ok. It is less often, but sometimes a different compiler will use assembler which only fails at runtime (but that is somewhat rare; there would be a specific error at runtime which you’d find on Google as incorrect compiler release).

1 Like
30

Incidentally, up until L4T R36.x (see “head -n 1 /etc/nv_tegra_release”), there was no nvidia-smi on Jetsons. The version which is present in R36.x is only partially functions.

To explain why, know that Jetsons have a GPU which is integrated directly to the memory controller (an iGPU). nvidia-smi is a tool of the desktop PC, and always expects a GPU to be found through the PCI bus (a discrete GPU, or dGPU, sits on the PCI bus). None of the PCI search functions exist for the GPU on Jetsons, and the version of nvidia-smi which now exists in L4T R36.x is missing a lot of functionality and is incomplete. Naturally, anything related to the PCI bus is either missing or just a stub.

1 Like
31

Regarding directories found in “/lib/modules/”, the correct directory is found via a combination of the version of the booted kernel, and the value of the CONFIG_LOCALVERSION used during compile of that kernel. If your kernel source is version 5.15.148, and there was on CONFIG_LOCALVERSION, then the subdirectory is “5.15.148/”. If your CONFIG_LOCALVERSION was set to the string “-tegra”, then this is appended, and the subdirectory searched for modules becomes “5.15.148-tegra/”.

Each kernel build has a configuration, and within that configuration there are “integrated” symbols. Those are the symbols chosen with “=y”, and these are always loaded and not in the form of a module. Those become part of the actual “Image” file in “/boot”.

There are also modules, added with the “=m” configuration. These are separate files and are loaded as needed at runtime. Those all appear in a subdirectory of:
/lib/modules/$(uname -r)/kernel/

If a module was compiled against the same integrated (=y) features, then you can load that into the existing kernel and you do not need a new Image, you only need to copy the module to the correct place and “sudo depmod -a”. Then it will work.

One would normally keep the same “CONFIG_LOCALVERSION” (by default on Jetsons you’d use “-tegra”) if you are only working on modules. Then everything that preexisted would be reused and the module would be a simple file copy. Once you’ve changed the integrated features when building a module, then there is a strong possibility that the module will not load into your running kernel. This is when you must build a whole new kernel, and as a side effect, you must also change the CONFIG_LOCALVERSION and install not only the “Image”, but also all modules (not just the module you want; every module).

Run “uname -r”, and you will know your correct subdirectory in “/lib/modules/”.

This is in fact related to the initrd. Think of the initrd as an “adapter” between the bootloader and the kernel when the bootloader is trying to end its own existence and overwrite itself with the Linux kernel. A couple of things are needed to do this…

First, the bootloader must be able to read the kernel and put it in memory. Usually this is from “/boot/Image” as a file on the ext4 filesystem. This can also be from a binary partition which has no formatting.

If from the partition, then bootloader must have the driver for the disk which the kernel is retrieved from. If that is eMMC, then the driver for that is needed. If from an NVMe, then that driver is needed. If over USB, then the correct USB driver is needed. So on and so on.

The ext4 filesystem is in fact understood by the bootloader. This is why the bootloader can load “/boot/Image”. But what about the kernel itself? This does not mean the kernel can read ext4! If the driver for ext4 is built in to the kernel with an “=y” integrated feature, then the kernel can read its own modules if the modules are located at “/lib/modules/$(uname -r)/”, and simultaneously that location is a simple ext4 filesystem.

What happens if there are further drivers required? For example, what if “/lib/modules/$(uname -r)/” is a RAID partition? What if the partition as a whole is from a logical volume manager instead of just a simple ext4 filesystem? In that case, success occurs only if the feature for that other option (e.g., a driver for LVM or RAID) is integrated into the kernel (an “=y” feature). If you’ve built those other features as a module, then boot will now fail.

The initrd is how you work around this. The initrd is a filesystem type which the bootloader and the kernel always understand. If your “/boot” is ext4, or you are reading the Image (integrated kernel features) from a binary partition, then you can always load the main kernel. If the initial root filesystem is an initrd pretending to be a partition, and if that initrd has any extra modules on it needed to load the real rootfs (such as a RAID module, an LVM module, or some strange filesystem type neither kernel nor bootloader understand), then the kernel will have what is needed to continue simply by loading its modules.

No initrd under those circumstances? You would then have a problem, the proverbial “Catch 22” where the module is now needed to read the filesystem that has the module telling it how to read the filesystem. Also, if you have modules in your initrd, and if those modules won’t load because you’ve changed the Image and the modules now only load for a different kernel config, then you’ve lost that driver and the ability that driver provided to mount the root filesystem.

The lesson of that latter paragraph is that if you don’t change the “=y” features, but you do add a module, then all of the original modules will work (this includes having the same CONFIG_LOCALVERSION; presumably for Jetsons the default is “-tegra”). The modules in the initrd will still work with that kernel and you won’t need to touch that module. If you’ve changed CONFIG_LOCALVERSION, and if there is a module needed for boot in your initrd adapter, then boot will break if you don’t edit the actual initrd and place all required modules (after recompile against that configuration with the new CONFIG_LOCALVERSION).

Examples, assuming the initrd has mandatory boot files:

  • Change “uname -r” because you didn’t match CONFIG_LOCALVERSION: You must rebuild the initrd with all kernel modules installed to the rootfs, and with all boot requirement modules installed into the new initrd.
  • Change only a module which is compiled against the original config’s CONFIG_LOCALVERSION, and no integrated feature is changed: Nothing needs to be done with the initrd, and the rootfs module install is just a file copy.
  • If the initrd was never required, at least not for reason of modules, then boot should continue. However, if modules have changed due to changing CONFIG_LOCALVERSION, all of your module-based drivers will fail to load. It is a question then of whether the system can boot. Mabye it will boot and you can correct the issue.
  • If you do not set CONFIG_LOCALVERSION, and if that feature was previously set to some string like “-tegra”, then installing that kernel (or a module built against that kernel) will likely fail (even if the module would succeed loading the kernel would be looking in the wrong place; often the module would also fail to load if found when some kernel features are mismatched between module and Image).
1 Like
32

Hi, sorry for replying so late @linuxdev ,

I am very grateful for the time you took with your posts, I have learned some more important things about the Linux kernel, thank you a lot.

I have applied the “-tegra” in CONFIG_LOCALVERSION configuration, so I had all the modules installed in the same /lib/modules directory, and added only two configurations as dynamic modules (“=m”), for being more specific, only these two: “CONFIG_NETFILTER_XT_TARGET_TCPMSS”, and “CONFIG_NETFILTER_XT_MATCH_TCPMSS”, and without changing any other configuration.

Also, for the management of the config file, I have done it as you say, only with the menuconfig, although, I need to compile the kernel from the kernel-jammy-src directory like this: make -j$(nproc), because if I do it from sources with make -C kernel, it always rewrites my .config with the defconfig without the previous enabled configurations… I do not know if this can make the difference, but the Makefile of the kernel directory always takes as default the arch/arm64/configs/defconfig file no matter what.

The first Image is the one generated by me, and the second is the default one given by Nvidia:

ubuntu@xxx:~/nvidia/nvidia_sdk/JetPack_6.1_Linux_JETSON_ORIN_NANO_TARGETS/Linux_for_Tegra/sources/kernel/kernel-jammy-src$ ls -lah /home/ubuntu/nvidia/nvidia_sdk/JetPack_6.1_Linux_JETSON_ORIN_NANO_TARGETS/Linux_for_Tegra/sources/kernel/kernel-jammy-src/arch/arm64/boot/Image
-rw-rw-r-- 1 ubuntu ubuntu 41M Oct 25 10:55 /home/ubuntu/nvidia/nvidia_sdk/JetPack_6.1_Linux_JETSON_ORIN_NANO_TARGETS/Linux_for_Tegra/sources/kernel/kernel-jammy-src/arch/arm64/boot/Image

ubuntu@xxx:~/nvidia/nvidia_sdk/JetPack_6.1_Linux_JETSON_ORIN_NANO_TARGETS/Linux_for_Tegra/sources/kernel/kernel-jammy-src$ l -lah /home/ubuntu/nvidia/nvidia_sdk/JetPack_6.1_Linux_JETSON_ORIN_NANO_TARGETS/Linux_for_Tegra/kernel/Image
-rw-rw-r-- 1 ubuntu ubuntu 42M Sep 13 04:29 /home/ubuntu/nvidia/nvidia_sdk/JetPack_6.1_Linux_JETSON_ORIN_NANO_TARGETS/Linux_for_Tegra/kernel/Image

Very similar in size, however I think there must be an issue here, because the one I generated is 41MB, and the default one is 42MB, it does not make sense… Also, after the installation of the modules, as stated in the developer for the 36.4 release of Jetson Linux, I update the initramfs, like this sudo ./tools/l4t_update_initrd.sh, I didn’t forget to do it, as you say, it is very important.

In the following compilation steps I don’t see any problem, but when I generate the final image ready to flash to the Jetson, I realized that my output image after executing the “apply_binaries.sh” and “jetson-disk-image-creator.sh” scripts, was of less than 9GB, and the default one provided by Nvidia has a size of 23GB (JetPack-6.1 release), also very strange…

33

@azulu There’s a lot in this thread, but I have been going through a similar process recently with a Jetson Orin Nano Developer Kit 8GB. I also wanted to enable a networking related kernel module that was not available in the standard kernel build. I used make -C kernel KERNEL_DEF_CONFIG='defconfig menuconfig' so that the defconfig target ran first, then menuconfig. No modifications to the defconfig files. I got the kernel to build, and then I had some trouble with the OOT modules. I also had errors referencing “unknown symbols”. I grepped for some of the failing symbols in the OOT modules source code, and after reading a file or two, noticed that there were some CONFIG_* options it required. I checked my kernel/kernel-jammy-src/.config file and realized those CONFIG_* options were not enabled. So I rebuilt with just KERNEL_DEF_CONFIG=menuconfig and enabled those extra options in the kernel config, and then tried re-compiling the OOT modules, and they compiled!

That is where my success ends. I ran sudo make install on the kernel as well as modules_install and same for the OOT modules, and finally reconfigured /boot/extlinux/extlinux.conf with a second boot option to allow selecting the new test kernel but when the device reboots it just shows a black screen. That probably doesn’t help you, but hopefully the hint about missing kernel config options does, and perhaps you will figure out how to get your custom kernel to boot and let me know. :)

1 Like
34

Assuming you started with the “defconfig”, or some other known working configuration, and only added modules, then that approach of adding the modules you want via menuconfig is a good way to go. You still need to know that all of the modules in “/lib/modules/$(uname -r)/kernel” can load correctly. I assume they probably can if all of the modules in that directory (prior to adding yours) originally paired with that kernel which ran with the original modules. Mostly if the “/boot/Image” file itself has never changed, then you know this is correct. If you’ve ever changed the actual “/boot/Image” file, then those modules might differ. Most likely what you have is correct (I’m just trying to be thorough).

What currently shows up from the command “uname -r”? If answer names a kernel version with “-tegra” suffix, then all checks out for module placement. This command comes as an answer from the kernel itself, and so regardless of whether this is the old Image or a new Image, this decides where modules are searched for. When the modules are found, and when you see no load failures in “dmesg”, the implication is that everything “fits together” as it should. The process of flashing can of course do things you won’t expect, so it is important to check that after any flash. You might even run a simple checksum on the kernel which is present at “/boot/Image” and the kernel you think you added, e.g., “md5sum /boot/Image” is a fast checksum. Any Image you modified to flash with should have its checksum verified against the /boot file.

Note that the above checksum is for a comparison of the newly built kernel Image and what you’ve flashed. I generally prefer a direct file copy instead of flash, but that is a different topic. It is very likely that any kernel you’ve just built, even if it has the same exact options as the previous kernel, will have a different size and checksum. One reason is that files distributed by NVIDIA are likely stripped of debug metadata, but your build is not stripped the same way. Even when they are stripped, if it turns out there is a slightly different compiler version, then that too might produce different assembler, and thus a different size/checksum. However, if you think you’ve added your Image of a given size to flash content, and then flashed, you can expect that the flashed Jetson’s Image should not have a different size or checksum.

Two different Image files, even if one is stripped and the other is not, and even if built on different compilers using different assembler, depend on the configuration to decide if a module can load into it. Just beware as you work:

  • Configurations on a given kernel source code release determine if modules are compatible.
  • Matching checksums and file sizes looks at whether or not files are exact matches (useful for testing installations having what you expect).

Regarding the initrd, the only time this mandates an update is if the modules inside that initrd cannot load into the “/boot/Image”. Examples causing a change mandating update:

  • The “/boot/Image” kernel source version changed. Changing the actual kernel source version changes “uname -r” and also stops most modules from loading.
  • The “uname -r” changed (which can be same kernel source version, but new CONFIG_LOCALVERSION). Changing “uname -r” causes failure to find modules, and possibly failure to load even if found.
  • A new module has been created, and that module is required to boot (examples being a module for the type of filesystem the kernel is accessing, or perhaps a RAID module if rootfs is a RAID array). If no boot dependency has changed, then you don’t need a new initrd.

On the topic of flashing, if you’ve changed only modules, then you don’t need to flash. The existing kernel will answer where it is looking for modules via “uname -r”. If you’ve changed the “/boot/Image”, then you will normally find information for adding this via flash, but it still is not mandatory to flash. You can always ask for information on how to manually add files.

I’ll add that if you go to “/boot/extlinux/extlinux.conf”, then you can see your boot entries. Normally you have only one. You can duplicate that entry and give it new labels, and then it is possible to boot by either entry. Now let’s say you’ve added a new kernel, but you instead name your kernel file something like “/boot/Image-new”; then you can edit the file name in the second boot entry…you will still have your original in case something goes wrong, but you will have the ability at boot to pick the new or old entry.

If you make any boot entry a default, and it fails, then you simply figure out how to pick the alternate entry. Both entries can name their own initrd, their own device tree, their own Image file name, so on. I try to always keep one completely original entry. If what I modify works as I want, then I turn that entry into the first entry and make it default. For this to work with alternate boot schemes you probably need an alternate initrd as well, but often everything can be kept the same other than the Image file itself. You can always create multiple boot entries and test different initrds or device trees or Image files, so on.

Do beware that “apply_binaries.sh” is meant to run only once. This is what adds the NVIDIA content into an otherwise purely Ubuntu rootfs. Once that content is present you don’t need to add it again.

Also, it is useful to illustrate some of what goes on in flash using a manual command line flash (this won’t work for initrd flash, but it is a useful illustration). An example “ordinary” flash (this is for an AGX Orin, so it is definitely not applicable directly to your nano) which does not use an initrd is:
sudo ./flash.sh jetson-agx-orin-devkit mmcblk0p1

If you were to go to the “Linux_for_Tegra/” directory where you would flash from (and in which you’ve added the AGX Orin software), then you would see a number of files with the suffix “.conf”. The prefix would include “jetson-agx-orin-devkit”, and the full file name would be “jetson-agx-orin-devkit.conf”. Each .conf file, with the suffix removed, is a flash target. Your version of Orin Nano will have such a file.

Part of what is specified in the .conf file is any boot requirement. This includes the actual kernel Image. It is during flash that the final boot-dependent files are added into “Linux_for_Tegra/rootfs/”. Unless you’ve placed your kernel Image file in the correct place you can expect the default for that target to overwrite your “rootfs/boot/Image”. This also might include replacing the extlinux.conf file, the initrd, so on. Most of everything else in “rootfs/” is taken verbatim and any modification you make on that other content will live through any and every flash.

1 Like
35

When you build a new kernel you have to put all modules in place. What typically happens is that someone will miss the NVIDIA GPU kernel module. Boot with the original kernel, check “lsmod”, and then boot on the failing kernel; check “lsmod” again for a loss of an NVIDIA module. You can use serial console or ssh to see what is going on.

Note that if lsmod is missing something, then there are two choices:

  • The module is there, but won’t load because it is incompatible.
  • The module is missing (this is the most common problem).
36

Thank you for your helpful replies! One other thing that I recalled from my efforts was that to build the out-of-tree modules, I had to modify the Makefile that came with the OOT modules to allow me to set the 5.15.148-tegra version. Without setting the version in the Makefile, it was just defaulting to 5.10.192-tegra (output of uname -r), and the modules did not match in /lib/modules. I modified the Makefile like this

-KERNEL_HEADERS ?= /lib/modules/$(shell uname -r)/build
+KERNEL_VERSION ?= $(shell uname -r)
+KERNEL_HEADERS ?= /lib/modules/${KERNEL_VERSION}/build

and then ran it with make modules KERNEL_VERSION=5.15.148-tegra

37

Also, I just used sudo make install to install the kernel and modules, as well as the out of tree modules. I have not messed with the flashing methods, since it seemed that the kernel/modules were the only things changing, but perhaps I am wrong about that.

38

Note that if your running kernel is 5.15.148, then you should not try to build a module against the 5.15.192 and then load it into the wrong release. You might be able to fool it with edits, but it is "a bad idea"™. Or the other way around. Modules should be built against both the same kernel release and the same configuration. If it happens that nothing related to the module you are working with has changed between 5.15.148 and 5.15.192, then you can probably get away with it.

When modules change there isn’t much to it, you just copy the new modules to the right place under “/lib/modules/$(uname -r)/kernel/”. When you install a new actual kernel Image, then you should just by default install everything again.

Keep in mind these possible errors:

  • Module exists, but won’t load.
  • Module does not exist, and so it cannot load.
  • Image has changed, and thus 100% of all modules not compiled against that Image configuration (and source code release version) are now suspect. You would want to build and install 100% of all modules again for this case.
39

I didn’t build against 5.15.192. Version 5.10.192 was version running on my build device. I was just saying that the kernel OOT modules Makefile always uses uname -r. The version of the source that I was building was not 5.10.192 but 5.15.148. Because of this, I modified the Makefile to allow me to change the version from uname -r (5.10.192-tegra) to 5.15.148-tegra since I knew that the versions must align.

40

Hi @linuxdev ,

Thank you very much for your reply.

Just for making sure that it is clear what I am really doing. I am doing a cross compile from an Ubuntu 22.04 machine, a virtual machine. That’s why after I compile the Image I generate a SD card image, I flash this image to the SD card and then use it in the Jetson Orin Nano.

What it seems strange for me is that when I generate the image in the make -j$(nproc), is that there is a difference between the my generated Image, and the one that Nvidia provides as default in the /Linux_for_Tegra/kernel/Image directory, a difference of only 1MB, but I’m sure there must be some problem from here.

Also when the SD card image is generated, it is a lot smaller than the same release SD card image that Nvidia provides from their website, here the difference in size is huge (less than 9GB mine, vs 22GB the official one).

I don’t have any idea what else could go wrong from the kernel compilation, I checked that the uname -r output was correct (same as the value set in CONFIG_LOCALVERSION), and the config file generated only adding the two previous modules as dynamic modules without any additional change, and using always the menuconfig, regarding the script apply_binaries.sh I just executed it one time, as I previously read that executing it more than once it can lead to problems, and after the generation of the Sd card image with the script sudo ./tools/jetson-disk-image-creator.sh -o sdcard.img -b jetson-orin-nano-devkit -r 100 -d SD, I flash the image with the BalenaEtcher tool.

I hope you can come up with a solution, I’m out of ideas, thank you.

41

Yes, there is a confusing “uname -r” on a build machine; if you use the “INSTALL_MOD_PATH=/some/where”, then it will “install” to that directory. Within that will be a “lib/modules/"uname -r of the source code instead of the build machine"/kernel/”. I always do this anyway so I can examine for errors, but if you cross compile, or if you are developing for anything custom, then you’d do this during the “modules_install” step. It is kind of amazing at all of the conveniences the kernel people have put into building such a complicated thing as the Linux kernel.

42

You don’t need to actually “flash” the content to the SD card if you are only installing a kernel. What “flash” means though to different people is what confuses things. Sometimes actually writing a new SD card image is easier, but sometimes just copying files to the right place of the SD card on the VM machine is better. To some extent it is a matter of taste, and sometimes it is a matter of how “safe” it is if the operation fails (which gets even more confusing because then you need to consider if the user has set up things on the SD card to save which flash overwrites).

Size from flashing an SD card might differ just in debug content. There are a lot of files (including kernel and kernel modules) which might be stripped in some cases, but not in others. In theory they are the same “logic” (but if there is a flaw which corrupts an argument stack they will differ). I don’t know what your goals are so far as if you have content on the SD card you wish to preserve. Also, it is possible the list of packages installed from flash differ from the default image.

In the case of debugging, I’m going to suggest you create a default SD card image. Don’t modify it. Then experiment with manually adding the kernel changes instead of using a flash tool. It really helps if you have saved a copy of any SD card image which you have modified and works; then you can try kernel modifications and simply copy those into the SD card instead of rewriting the entire SD card image. That of course takes a lot of disk space on the host PC, but if you have that disk space, then it will make life easier. Just ask if you want to know how to make the copies (a copy would be the size of your SD card).

Having the “-tegra” CONFIG_LOCALVERSION means that if you compile modules against the running kernel Image’s configuration, then the module should be compatible with that Image and capable of loading. Is the kernel which actually runs on the Jetson the starting configuration before adding the modules? If so, then this is perfect and should work. I’m assuming that the “uname -r” of that running kernel also has a “-tegra” suffix.

When you do add a new feature as a module (or even integrated), then there are times when a dependency will be triggered. That dependency (if it exists) will be another module to copy beyond the ones you are expecting. If you’ve booted up the Jetson, and the module(s) show as loaded via “lsmod”, then all is good. If this has not loaded, then perhaps nothing required it (yet). You can still manually “modprobe <module>” to load this even if nothing automatically triggered module load. After that you will see the module via “lsmod”.

If modprobe fails (or anything inserting a module), then it might mention missing symbols. This would indicate that there was a dependency for that module, and that the dependency must load prior to the module you are trying to load. The module would have been built because menuconfig is dependency aware, but you might not have noticed. The insmod or modprobe success/failure is the “acid test”.

One complication is that not all features/symbols can be build as a module. Not all can be built as integrated. Most can. However, if you’ve enabled a feature as a module, and if there were dependencies, and simultaneously if one of those dependencies cannot be built as a module, then you end up having to install a new kernel Image as well as 100% of the modules. This rare.

Something seems wrong if “make -C” always rewrites your .config. Are you providing an explicit build command, e.g., a make target of “modules” or “Image”? If you have simply run “make -C ...” and did not specify what you are building, then it will not do as you want it to. This would be a problem, but it wouldn’t be the fault of the “make -C”. Whether you use “-C” and a location, or if you are directly in the kernel source directory, it is incorrect to simply run “make” (the “-j” doesn’t matter, you can ignore this for what we are speaking of).

We should ignore the l4t_update_initrd.sh unless some module in that initrd is needed for boot. Any module in this initrd which loads after the Linux kernel is running won’t care about the initrd. Changing an entire kernel Image though is indeed a reason to update the initrd since any module which is in that initrd would no longer load. This is one reason why copy of the kernel Image and modules (manual copy) can be easier: Sometimes a new initrd is not required.

Also, some people use the NVIDIA build scripts, I just manually build. It is possible something in an NVIDIA kernel build script confuses the issue with how I am explaining it. What is your exact build command line? Despite the smaller installation size, does this boot correctly? Install size cannot be used in most cases to determine if everything is correct.