Need help rebuilding kernel with an additional item

[nick223]

Hello,
I am trying to add support for GFS2 on my Jetson Orin AGX with Jetson linux 35.4.1. I have experience compiling kernels so not a noob, but at the same time I’ve tried several of the guides on the Nvidia site as well as followed some threads, but no luck. Can someone help me with some basic steps?. I can cross compile OR modify on my system, whatever is easiest. All I’m trying to do is add support for GFS2 to the system.

thanks,

[nick223]

So I figured out how to successfully recompile, but now i need to change the .config and compile again. Not sure how to do that.
And once i’ve recompiled a second time, how do I flash the Orin with the new kernel?

[linuxdev]

You might find this useful:
https://forums.developer.nvidia.com/t/topic/262647/12

Are you using the compile option with “O=/some/where” to put temporary output there? This makes it easier to explain, but basically, whatever your output location is, any configuration will generate the “.config” file. The “make mrproper” removes configuration, so the two are inverses of each other (well, sort of). One normally starts with pristine source, then set a base config of “make tegra_defconfig” (perhaps with “O=...”). After that you can use a dependency-aware editor. There are actually several editors, and the most common is “make menuconfig” (perhaps with “O=...”). I prefer “make nconfig” because although it appears to be a nearly exact duplicate of menuconfig it has a symbol search function.

Any configuration item is represented by a “symbol”, which has a name “CONFIG_something...”. This is what the “.config” file contains (it’s human-readable, take a look at it after a tegra_defconfig). For example, this is a symbol which nconfig can find and set as either module or integrated (more on that below):
CONFIG_GFS2_FS

Content which is only in a module is very easy to work with. You just copy it to the right place (see that URL on kernel config above), and run “sudo depmod -a” (perhaps reboot), and you are done. It’s fairly risk free. Modules are part of the kernel, they just load or unload while running. The rest is “integrated”. In the editor (I mean a dependency-aware editor, not a regular editor), the m key sets the feature as a module, the y key sets the feature as integrated, and the n key disables the feature (either commenting out the symbol or setting it “=n” disables the feature).

Do not forget to set the “CONFIG_LOCALVERSION”, which is a string, and on Jetsons it defaults to “-tegra”. If you build a module, then keep that as “-tegra”. If you build more than a module, then set a new name, e.g., “-test”. The output of “uname -r” is the base kernel version, plus CONFIG_LOCALVERSION. This is part of the path the kernel uses to search for modules:
/lib/module/$(uname -r)/kernel

Installing any change to the integrated kernel, the Image file, is far riskier and more involved. Not all features can be a module (most can), and not all features can be integrated. If you can build it as a module, then it appears somewhere in a subdirectory of the build. The kernel source might have produced your module here (a contrived example, I have no idea if this is really the module name):
$TOP/drivers/net/gfs2.ko

In that case you would copy to:
/lib/modules/$(uname -r)/kernel/drivers/net/gfs2.ko
…then “sudo depmod -a”.

[nick223]

Thank you! This and the link are very helpful. I think i’m slowly getting the flow of it all. I do have a few questions:
so far the steps i’ve followed:
1.setup cross compile environment, untar public sources, untar kernel source
2. Setup compile environment paths, including a KERNEL_OUT path.
2. execute nvbuild.sh with -o KERNEL_OUT option where its placing the compiled result.

Seems like I need to edit the .config with nconfig in between steps 2 & 3 as to get the configured build as I would like it right? I think i’m skipping straight to compiling with the nvbuild script and not getting the configuration that I want.

Reading your other thread, it seems you are entering the location of the pristine source, then making your nconfig choices, then compiling just the modules required to the KERNEL_OUT path? In my case, this may work as I only need the GFS2 module that I can copy over to my target system.

Just thinking out loud here…

[linuxdev]

A kernel is “bare metal”. That means there are no dynamic link libraries. It’s code purely referencing itself. This, in turn, means you only need a cross compiler. There is no need for a full environment, e.g., no need for a cross linker, and no need for a sysroot (which is what contains the foreign architecture libraries). The instructions in the official NVIDIA documentation usually just has you install a cross linker via the apt mechanism on the host PC. If the cross compiler is not available in a correct version, then the same download page for the documentation of a given L4T release also contain the download of the particular cross compiler (although it might be difficult to find).

Setting up the “KERNEL_OUT” and then using “O=$KERNEL_OUT” is a very good practice. Just make sure you start with an empty directory for $KERNEL_OUT. If there is a “.config” you produced in that location, then you can copy that to a safe spot if you wish to preserve it. Then, any time you delete that directory and recreate it, you know you are starting cleanly. If you then copy the “.config” file you’ve restored all of your configuration (you would still need to “propagate” that config if not starting with the “Image” build target; e.g., “make O=$KERNEL_OUT modules_prepare” would propagate that config for building a module without first building Image).

Running “make O=$KERNEL_OUT tegra_defconfig” creates “$KERNEL_OUT/.config”. It still needs CONFIG_LOCALVERSION to be set up. If you then run something like “make O=$KERNEL_OUT nconfig”, then any edits will be placed in “$KERNEL_OUT/.config”. When satisfied, or just want to track progress, save “$KERNEL_OUT/.config” somewhere else, perhaps renamed with a note in the name for what it is. Create with tegra_defconfig, and edit with nconfig (or menuconfig, but nconfig can find symbols). You can adjust CONFIG_LOCALVERSION via nconfig, just search for localversion (hint: search already knows about the CONFIG_ prefix, and also is case-insensitive).

Incidentally, for menuconfig or nconfig to work, you need the ncurses dev package on the host PC;
sudo apt-get install libncurses5-dev

I don’t use the nvbuild.sh script, but it does the same thing manually running the commands would do. I prefer fine control of where everything goes, and it isn’t too difficult. One of the reasons why nvbuild.sh exists is that people may not want to go through the details and learning curve for manually building on command line, but once you’ve done this, it’s actually quite easy.

It is very good to start with pristine source. That means running make mrproper without the “O=...somewhere...”. If that source is owned by root after the mrproper, and not modifiable for anyone else, and if you always compile as a regular user with O=...somewhere, you won’t hit any of the “old config sticking around in part and playing a joke on you 😈” issue.

In your case this might be a checklist (assuming you don’t use the nvbuild script):

• Unpack pristine source as root such that the regular user cannot modify it.
• If cross compiling be sure to set up any environment variables specific to that, e.g., you’d modify ARCH to be arm64, but if native compiling, do not add ARCH. The cross tools also need to be set up via an environment variable.
• Run “sudo make mrproper” to be certain it is clean.
• From then on use the “O=...somewhere...”, e.g., via the KERNEL_OUT environment variable. Use a regular user for that and make KERNEL_OUT a place your regular user has permissions to write to.
• Start with "make O=$KERNEL_OUT tegra_defconfig. This gives you a .config at “$KERNEL_OUT/.config”, but it is still missing CONFIG_LOCALVERSION setup.
• If you are only building a module, then edit CONFIG_LOCALVERSION to “-tegra”, the default. That’s a configuration which normally matches the running system. If the system has been modified, then you also have to account for that, e.g., using the “/proc/config.gz” instead of tegra_defconfig, and maybe altering the CONFIG_LOCALVERSION.
• If you are building with a modified “integrated” feature, then use a modified CONFIG_LOCALVERSION, e.g., if you were adding the GFS2 integrated and not as a module, you might set CONFIG_LOCALVERSION to “-gfs2” as a reminder.
• Build using that $KERNEL_OUT. Any module with a matching CONFIG_LOCALVERSION, and with integrated features matching, can then be a simple copy.
• Special note on building only modules: I advise building Image once as an acid test to see if it works. However, you could skip this, but you’d then need target modules_prepare. Building Image does this for you.
• When you go to copy a module, if the kernel source is at $TOP, note the subdirectory to that which the module ends up at, e.g., “$TOP/drivers/net/something.ko”, and copy it to the Jetson at:
  /lib/modules/$(uname -r)/kernel/drivers/net/something.ko
  (to mirror the subdirectory)

One more tip: If your host PC has 8 cores, then you might want to build with the “-j 8” option to use up to 8 cores during build. If you don’t have enough RAM and it starts to swap out, then you might be better of with a lower job server count (the -j is the job server), e.g., “-j 4” uses less RAM than “-j 8”.