My /lib/modules/$(uname -r)/build is mismatch

Hello,

I am experiencing an issue when attempting to load a kernel module using insmod.

While the driver compiles successfully into a .ko file, insmod consistently returns an “invalid format” error.

I have researched relevant topics and attempted solutions such as source_sync.sh and removing the RT patch. However, the problem is still there.

I noticed the compiler output includes the following warning:

warning: the compiler differs from the one used to build the kernel nano

Upon investigation, I discovered a discrepancy in the build symbolic link within the /lib/modules/$(uname -r)/build directory.

My custom kernel build points to:

/media/softpc/ssd/nvidia/Jetson_AGX_Orin_V6/Linux_for_Tegra/source/kernel/kernel-jammy-src

The build symbolic link from a Jetson Linux installation via SDK Manager points to:

/usr/src/linux-headers-5.15.148-tegra-ubuntu22.04_aarch64/3rdparty/canonical/linux-jammy/kernel-source

I suspect this difference is the root cause of the issue. Could you please advise on how to align my /lib/modules/$(uname -r)/build symbolic link with the structure used by the SDK Manager installation?"

My Jetpack version is 6.1

Hi,
Please enable the configs and rebuild kernel image by following:

Kernel Customization — NVIDIA Jetson Linux Developer Guide

Here is a post for reference:
No data from Joystick Logitech-f710 - #10 by DaneLLL

Hi Dane,

I checked the website you provided. We followed all the instructions to build the kernel and DTB. I don’t see any differences from the website’s instructions. Could you please explain why the symbolic link of my OS is mismatched?

I noticed there is a code in ‘Linux_for_Tegra/apply_binaries.sh’ below.

# Link to the kernel headers from /lib/modules/<version>/build
if [ ! -d "${LDK_ROOTFS_DIR}/usr/src/${KERNEL_HEADERS_A64_DIR}/${KERNEL_SUBDIR}" ]; then
        if [ -d "${LDK_ROOTFS_DIR}/usr/src/${KERNEL_HEADERS_A64_DIR}/3rdparty/canonical/linux-jammy/kernel-source" ]; then
                KERNEL_SUBDIR="3rdparty/canonical/linux-jammy/kernel-source"
        elif [ -d "${LDK_ROOTFS_DIR}/usr/src/${KERNEL_HEADERS_A64_DIR}/kernel-oot" ]; then
                KERNEL_SUBDIR="kernel-oot"
        elif [ -d "${LDK_ROOTFS_DIR}/usr/src/${KERNEL_HEADERS_A64_DIR}/stable" ]; then
                KERNEL_SUBDIR="stable"
        fi
fi
KERNEL_MODULES_DIR="${LDK_ROOTFS_DIR}/lib/modules/${KERNEL_VERSION}"
if [ -d "${KERNEL_MODULES_DIR}" ]; then
        echo "Adding symlink ${KERNEL_MODULES_DIR}/build --> /usr/src/${KERNEL_HEADERS_A64_DIR}/${KERNEL_SUBDIR}"
        [ -h "${KERNEL_MODULES_DIR}/build" ] && unlink "${KERNEL_MODULES_DIR}/build" && rm -f "${KERNEL_MODULES_DIR}/build"
        [ ! -h "${KERNEL_MODULES_DIR}/build" ] && ln -s "/usr/src/${KERNEL_HEADERS_A64_DIR}/${KERNEL_SUBDIR}" "${KERNEL_MODULES_DIR}/build"
fi
popd > /dev/null

if [ -f "${LDK_KERN_DIR}/kernel_oot_headers.tbz2" ];then
        echo "Extracting module headers to ${LDK_ROOTFS_DIR}/usr/src/nvidia"
        pushd "${LDK_ROOTFS_DIR}/usr/src/nvidia" > /dev/null 2>&1
        tar ${TAR_ROOTFS_DIR_OPTS} "${LDK_KERN_DIR}/kernel_oot_headers.tbz2"
        popd > /dev/null 2>&1
fi

Is it possible that this code creates an incorrect symbolic link during kernel build?

Sorry. I have another question. Do we need to do the commands below after building my own kernel?

sudo ./apply_binaries.sh
sudo ./tools/l4t_flash_prerequisites.sh
sudo ./tools/l4t_create_default_user.sh

Hi,
If you enable the kernel modules in the two config files and build kernel image, the ko files should be included as part of kernel. Not sure if you have tried this. It seems like you build the ko files individually.

Have you set the CROSS_COMPILER path using EXPORT command before building the ko file on your host PC?

sudo ./apply_binaries.sh
sudo ./tools/l4t_flash_prerequisites.sh
sudo ./tools/l4t_create_default_user.sh

These files have their own purpose and they are not all related to kernels.

sudo ./apply_binaries.sh

This file will install nvidia drivers and has chance changing your kernel, so please do not run it again.

l4t_flash_prerequisites.sh

This script is just to install some prerequisites software on your host PC so that flash process won’t hit problem. Nothing to do with kernel.

l4t_create_default_user.sh

This tool is just to create a default user account/pwd in your rootfs so that you won’t get asked to do configuration in your first boot. Nothing to do with kernel.

Hello Nagesh_Accord,

Yes, I have exported those environment variables for building my kernel. And the kernel seems to be working fine.

I’m just confused as to why the driver compiles and works on the SDK Manager Jetson Linux, but it doesn’t work in the my kernel(it also can be compiled).

Just my thoughts:

Since Jetson device is your native ARM platform where you are compiling and the driver module is loading fine.

But your host pc is different architecture x86, so that might be causing issue, even though it may be compiling. May be the cross compiler settings or some other dependency which is making the KO file not in sync to run on the native Jetson

Modules have to be built against a given kernel or they won’t load. Changing configuration of non-module aspects can alter module load, especially the CONFIG_LOCALVERSION changing. You might find these of interest:

Thank you, Sir.

I finally found the solution. The key difference between the SDKManager’s Linux (which I’ll call OS1) and my built OS (which I’ll call OS2) is the cross-compiler.

I noticed that OS1 uses /usr/bin/aarch64-linux-gnu-gcc, while OS2 uses aarch64-buildroot-linux-gnu-gcc, which comes from the aarch64–glibc–stable-2022.08-1.tar.bz2 file.

Following the quick start guide, I exported the cross-compiler:

export CROSS_COMPILE=/bin/aarch64-buildroot-linux-gnu-

So I was thinking what if I use aarch64-buildroot-linux-gnu-gcc to compile my driver.

The experimental results were as I expected. The driver can be loaded in OS2 and works fine.

Now, I’m confused because OS1 doesn’t seem to be compiled according to the quick start guide. I was able to successfully compile the driver and use insmod on OS1, so I expected to do the same on OS2." But it’s not.

1 Like

so finally, were you able to resolve the issue and it is loading the KO module fine?

Sure. Thank you so much.

1 Like

Mark your post as solution so that it will be helpful to others

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.