Some partial information…
There is a “whole”, or “integrated” kernel…the kernel image. Which coincidentally, when on the filesystem and not used directly from a binary partition, is file “/boot/Image
”. This is also a kernel compile target: “make Image
”.
Modules are also kernel code, but they can be loaded and unloaded from a running system. They’re a simple file copy to add if you know where to copy them to. Replacing the whole Image
file is riskier and has other possible requirements to replace. Not all parts of the kernel can be built as a module, but most of the device drivers can be in the form of a module. Content of the running system’s modules can be seen via “lsmod
”, but note that the vast majority of kernel content is not in the form of a module, and thus won’t be visible via lsusb
.
On a Jetson the running kernel tells you want its configuration is via the pseudo-file “/boot/config.gz
” (it is not a real file, it is part of the kernel running in RAM pretending to be a file). If you were to compile a kernel, then aside from the “CONFIG_LOCALVERSION
”, a copy and decompress of “config.gz
” would be an exact match for the configuration used to build that kernel. This lets you start modifications via an exact match.
Incidentally, the command “uname -r
” results in a combination of the base kernel version, along with the “CONFIG_LOCALVERSION
” which was set at the time of compile. The CONFIG_LOCALVERSION
suffix is almost universally “-tegra
” on Jetsons. The reason why a match of this too is important is that the kernel Image
finds its modules at some subdirectory of:
/lib/modules/$(uname -r)/kernel
“Out of tree” is referring to compile, and does not change that it is a module being built. Note that if you were building the entire Image
file, then by definition you are “in-tree” because Image
is the tree. Modules can be compiled without building the Image
file, but then you’d need some extra setup. You could also compile some modules in some separate directory away from the kernel source, but then you’d need more setup to point to the kernel source. This is what is meant by out-of-tree: Your module code being built is in its own private subdirectory somewhere and pointing at the kernel code, versus being in some subdirectory of the kernel code during build. It is still a module regardless of whether it is located in kernel source during compile, versus being elsewhere. Also, a module is still kernel code, it just binds differently.
Quite often third party drivers are shown for “out-of-tree” build. These drivers almost universally expect the kernel headers (which are really the part needed for pointing to during out-of-tree compile) to be in place and to be an exact match of the running system. Those same headers are not necessarily available on a Jetson, at least not configured correctly. What the instructions fail to mention is that if you have full kernel source, and if you’ve configured that source to match your running system, then this is a valid location to point your out-of-tree content to. This works, even on Jetsons.
Note that NVIDIA provides more content when downloading the kernel than does a typical kernel source download. When building kernel source some content can be referred to via a relative path using “../
”, which means the parent directory of the current directory. The “TOP” of source is main parent subdirectory of the kernel source, and most compiles would never refer to something outside of kernel source, but the NVIDIA content can refer (via relative paths) to content not in the kernel source itself. Thus some content requires more than just headers to build, even if you are out-of-tree and pointing at full source.
In the case where you do unpack the full NVIDIA version of kernel source, you will have a structure such as that I put below (you could put this content somewhere else, this is just where I put it):
# tree -d -L 2 /usr/src/sources
/usr/src/sources
├── hardware
│ └── nvidia
└── kernel
├── kernel-4.9
├── nvgpu
└── nvidia
In that case the “TOP” is “/usr/src/sources/kernel/kernel-4.9
”. In cases where you are told to point at kernel headers, then for this example, you could substitute “/usr/src/sources/kernel/kernel-4.9
”. You would want to configure that kernel source to match your running system before using it, but otherwise this would work for your compile of out-of-tree content just like kernel headers would work if they were there and configured. On a Jetson, don’t use the kernel headers, use the full source configured to match your current running system. You’ll save yourself some headaches and grief.
So far as commands go for compiling there are two categories: (A) Native compile, directly on a system of the same architecture (e.g., built directly on the Jetson, or perhaps a 64-bit RPi), or (B) cross-compile from a system of one architecture which differs from the destination for the kernel content.
Whenever you see “ARCH=arm64
”, then it means you are looking at instructions for cross-compile. You would thus also need the correct cross-compile tools installed since the “native” tools on your system would be for something like x86_64/amd64 and could not result in usable code on arm64/aarch64.
If you compile natively, there are far fewer things to set up. You would never use “ARCH=arm64
” if compiling natively on the Jetson…this would trigger some “cross-compile” behavior which would actually make it a pain to compile natively. If you compile natively you can just leave out the “ARCH=arm64
”.
The official documents explain cross compile quite well and it isn’t all that difficult, but you’d need to read that once before cross compiling. Those docs do not explain out-of-tree builds, but you would need to set up 100% like what those docs explain, and then add the slight differences for out-of-tree builds. You would still want to unpack the full NVIDIA kernel source somewhere and configure that correctly, and then point at that for your build instead of at some kernel headers (you could use kernel headers if configured correctly, but like I said before, you’re going to be less frustrated if you just install the full source and treat it as headers).
I always advise people to build the full kernel Image
first, even if only building modules. I say this because it completes some of the configuration which you’d have to perform separately before performing a module build. You’re more likely have a correct configuration if you successfully build a kernel Image
once, and then move on to building modules or out-of-tree content. Think of it as validation.
If I were to add kernel source on my desktop PC for cross compile, then I’d be tempted to unpack it from:
/usr/src/arm64
…which would result in having this:
# tree -d -L 2 /usr/src/sources
/usr/src/arm64/sources
├── hardware
│ └── nvidia
└── kernel
├── kernel-4.9
├── nvgpu
└── nvidia
…and this would result in pointing here when it wants headers:
/usr/src/arm64/sources/kernel/kernel-4.9
There are a lot of other details which would simplify your life, but which require setup only once. You’re free to ask for more details at any time, but this would be a much longer reply if I tried to put everything into this first reply.
Almost forgot: Building kernel code is not building a library. The code is part of the kernel (in bare metal “kernel space”). A library is in “user space”, and is not directly part of the kernel.