Building external Kernel Modules

Hi,

Yesterday I flashed Tx1 with R28.1. I would like to modify some drivers.

In NVIDIA Tegra Linux Driver Package, Chapter Kernel Customization -> Building External Kernel Modules it’s written to
cd /usr/src/linux-headers-`uname -r` sudo make modules_prepare

uname -r returns 4.4.38+, but in /usr/src there is only linux-headers-4.4.38-tegra.
Why there are no headers for the system that is currently installed? How can I compile natively external kernel modules?

Look at your Makefile…CONFIG_LOCALVERSION can be set through “make menuconfig”, but there is also another mechanism in place to do the same thing (only I think it has to be edited by hand). In your Makefile is there a non-empty “EXTRAVERSION”? If so, then this may append to the 4.4.38, or overwrite LOCALVERSION. I looked in the kernel source but didn’t see EXTRAVERSION set here.

same question
could you tell some detailed information!
LOCALVERSION = -tegra
but in my TX1,
$uname -r
show “3.10.96-tegra+”
why not is 3.10.96-tegra
i don’t understand “+”
else
$sudo apt-get install linux-headers-“uname -r”
show “don’t find linux-headers-3.10.96-tegra”

“uname -r” suffix from CONFIG_LOCALVERSION is determined from the base kernel Image, not modules…though it will change if the modules load as expected. Is your base kernel not compiled with this CONFIG_LOCALVERSION? Does the top level Makefile have any EXTRAVERSION?

Hi mmalczak,

uname -r returns “4.4.38-tegra” on my TX1. Do you modify anything before build?

I don’t remember modyfing anything.

4.4.38+ is what I need, because it contains the driver tc358743. Now what I want to do is to modify the driver, but in /usr/src/ there are only headers for 4.4.38-tegra. I don’t understand why only these sources are available, since in /lib/modules/ there are both folders: 4.4.38+ and 4.4.38-tegra.

Hi mmalczak,

I don’t see 4.4.38+ under my /lib/modules/ after flashed by Jetpack3.1.

Could you confirm it?

Hi,

Under my /lib/modules/ I can see both: 4.4.38+ and 4.4.38-tegra

for jetpack-L4T24.2.1
when i rebuild the kernel , use the ./flash.sh jetson-tx1 mmcblk0p1 to reflash the TX1 ;
under my /lib/modules , i also can see both 3.10.96+ and 3.10.96-tegra

if only use jetpack_3.0 reflash the TX1 ,
under my /lib/modules , i can see only 3.10.96-tegra

when i build "riffa"http://riffa.ucsd.edu/ on the TX1 and the kernel is 3.10.96-tegra+ and the kernel headers is linux-headers-3.10.96-tegra
“riffa” is a driver of FPGA and correlation function
when I
$sudo make
error:
CC [M] /home/ubuntu/riffa_2.2.2/source/driver/linux/riffa_driver.o
scripts/basic/fixdep: 1: scripts/basic/fixdep: Syntax error: “(” unexpected
…and so on

should what i do

For @??? (there are actually two conversations in the thread)…

Are you cross compiling or native compile on the Jetson? I assume native on the Jetson, just checking.

I downloaded riffa 2.1 and am examining its Makefile. Notice in “source/driver/linux/Makefile” that it uses “uname -r”. If you cross compile this is not correct (the host uname would mistakenly be used instead of Jetson uname), but should be ok for native build (be sure “uname -r” is “3.10.96-tegra” prior to starting build…FYI, you might also consider first upgrading to R28.1 which has a 4.x kernel).

This driver also looks at “/etc/redhat-release”. Then this block is used in the Makefile:

setup:
        if [ -f "$(RHR)" ]; then yum install kernel-devel-`uname -r`;\
        else apt-get install linux-headers-`uname -r`; fi

Since there is no redhat-release, it installs linux-headers package over the internet and wants “linux-headers-3.10.96-tegra”, but this doesn’t exist in default repositories. You could place the headers there yourself, e.g., just via a symbolic link to the kernel source’s headers as distributed by NVIDIA. Had the public repo for linux-headers succeeded, then possibly it differs from the NVIDIA supplied kernel anyway…not sure on that. Comment that block out, then verify if there are already files in “/usr/include/linux/” (I already have this, but it could have been from another package you didn’t install…check). Completely commenting out that block in the Makefile mentioned above will be correct it if you have headers already.

The actual headers are at “/usr/src/linux-headers-$(uname -r)/” and are part of the L4T install (this directory is not owned by a package, it is a direct file addition). Verify you have this directory, then find out if this symbolic link points there:

ls -ld /lib/modules/$(uname -r)/build

If not, then you can add a symbolic link (don’t do this if link is already valid):

sudo ln -sf /usr/src/linux-headers-$(uname -r) /lib/modules/$(uname -r)/build

Run a make clean, then try to build again and see how far it gets.

For @mmalczak

I gave a reply to @??? which is somewhat related to this. The directory “/usr/src/linux-headers-(uname -r)" provides content pointed at by symbolic link "/lib/modules/(uname -r)/build”. This is an NVIDIA directory tree not owned by a package. The module directory for your case should be the “/lib/modules/4.4.38-tegra/” directory, and the “/lib/modules/4.4.38+/” directory would only be valid for a “/boot/Image” file compiled as “4.4.38+” instead of “4.4.38-tegra”. Anything not matching your running “uname -r” would be a mistake (unless you have more than one boot entry and even then only the current boot “uname -r” would be relevant).

The short answer is that “/usr/src/linux-headers-4.4.38-tegra/” is the default linux-headers package under R28.1 (“4.4.38-tegra”), and that the “+” variation is a third party modification. “/usr/src/linux-headers-4.4.38-tegra/” is correct with the existing kernel, “/usr/src/linux-headers-4.4.38+/” is an unknown source. Neither of these are available via the apt package system since neither are in a public repository for default search under Ubuntu 16.04.

The question becomes whether you replaced the default “/boot/Image” with a 4.4.38+ version? Or perhaps there is a second boot entry in extlinux.conf and it points at a different Image than the default Image? Somewhere in your build must be third-party software which is using the “+”, or else you have previously replaced the “4.4.38-tegra” kernel with a “4.4.38+” kernel…what modifications or additions have you added to either the default boot entry or the “/boot” files?

The logic is that “uname -r” is determined entirely by the base kernel Image file. The module directory for each kernel belongs at “/lib/modules/(uname -r)/". The "build" symbolic link at "/lib/modules/(uname -r)/build” will always point to “/usr/src/linux-headers-$(uname -r)/”. dpkg will not own any of these unless they came from a public repository, and neither are available by default…the “-tegra” version is installed as part of the sample rootfs, the “+” version is an unknown source.

thankyou linuxdev
i proceeding native compile on the Jetson! in my jetson TX1 , /lib/modules/3.10.96-tegra+/build , the link is existed!
because when I execute “make”,

.....
make[1]: Entering directory '/usr/src/linux-headers-3.10.96-tegra'
CC [M] /home/ubuntu/riffa_2.2.2/source/driver/linux/riffa_driver.o
scripts/basic/fixdep: 1: scripts/basic/fixdep: Syntax error: "(" unexpected
...and so on

but in my jetson TX1 ,
$cd /usr/src/linux-headers-3.10.96-tegra
$sudo make modules_prepare
in the terminal , it show “loop execution” make;
in other terminal,
$cd …/riffa_2.2.2/source/driver/linux/
$sudo make
it show :

.....
make[1]: Entering directory '/usr/src/linux-headers-3.10.96-tegra'
CC [M] /home/ubuntu/riffa_2.2.2/source/driver/linux/riffa_driver.o
CC [M] /home/ubuntu/riffa_2.2.2/source/driver/linux/circ_queue.o
LD [M] /home/ubuntu/riffa_2.2.2/source/driver/linux/riffa.o
scripts/mod/modpost: 1: scripts/mod/modpost: Syntax error: "(" unexpected
...and so on

For @???..

If “uname -r” is “3.10.96-tegra”, then you must use the directories with the “3.10.96-tegra” name, and no “+”. If you use “3.10.96+”, then you must not use “3.10.96-tegra”. If “uname -r” is “3.10.96-tegra+”, then this is not compatible with either “3.10.96-tegra” or “3.10.96+”. Where did the base kernel image come from? It isn’t stock, else it would say “3.10.96-tegra”.

Can you reinstall the kernel from NVIDIA first? I ask because from there you can alter options and keep all of the CONFIG_LOCALVERSION and EXTRAVERSION the same. Trying to mix two “uname -r” suffixes will probably end up only causing grief. The “make modules_prepare” is probably having a hard time dealing with this.

thankyou linuxdev for your attention!
in my TX1, i need to reflash this kernel! everytime i reflash the kernel, if i change “EXTRAVERSION=-tegra”, “uname -r” is always “3.10.96-tegra+”. or "EXTRAVERSION= ", “uname -r” is always “3.10.96+” .
but I make the “riffa”, i run into the same question!

I find that my kernel_headers is always “linux-headers-3.10.96-tegra”.

whether “3.10.96-tegra+” or “3.10.96+” is compatible with “linux-headers-3.10.96-tegra” ?

do I need to reinstall the kernel of “3.10.96+”?

when I rebiuld the kernel ! I cannot gain the kernel of “3.10.96-tegra”, no “+” .

in addition to this , the “make modules_prepare” is endless loop! i think this condition is wrong! but I don’t know its reason!

CONFIG_LOCALVERSION and EXTRAVERSION is repetitive!
when i change “EXTRAVERSION=-tegra” in …/kernel_sources/Makefile
under the circumstance , in the “make menuconfig” , add CONFIG_LOCALVERSION “3.10.96-tegra”.

at last, in my TX1 , “/lib/modules/” show “3.10.96-tegra3.10.96-tegra+”.
it is the combination of CONFIG_LOCALVERSION and “EXTRAVERSION”
don’t have kernel headers ? it can be matched with kernel version ‘3.10.96-tegra+’ or ‘3.10.96+’?

Let’s look at the start of this since there is some important detail being missed. Are you compiling natively on the JTX1? Have you updated the kernel Image prior to this? Second, where was your kernel source downloaded from (including any kernel updates prior to this)? FYI, normally EXTRAVERSION is not used…I mention this because it is possible it interferes by adding something like “+” which shouldn’t be there (and third-party distributed source may change this). The native kernel version and “uname -r” on R24.2.1 should respond as “3.10.96-tegra”, the “+” can only be from outside source or edits combined with replacing “/boot/Image”, so to know why this is happening we need to know more about those edits regardless of whether you did the edit or if the edit came from outside source code.

operation sequence!
first, Extracting Tegra Linux Driver Package

sudo tar -vxjf Tegra210_Linux_R24.2.1_aarch64.tbz2

second, Setting Up Your File System

$ cd <your_L4T_root>/Linux_for_Tegra/rootfs
$ sudo tar -jxpf ../../Tegra-Linux-Sample-Root-Filesystem_R24.2.1_aarch64.tbz2

third, Getting the Kernel Sources

$ sudo ./source_sync.sh -k tegra-l4t-r24.2.1

I change nothing in the kernel source.

fourth, Building the NVIDIA Kernel

export CROSS_COMPILE=/opt/linaro/gcc-linaro-5.3-2016.02-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-
export CROSS32CC=/opt/linaro/gcc-linaro-5.3-2016.02-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-gcc
export ARCH=arm64
export TEGRA_KERNEL_OUT=<outdir>
cd <myworkspace>/<kernel_source>
mkdir $TEGRA_KERNEL_OUT
make O=$TEGRA_KERNEL_OUT tegra21_defconfig
make O=$TEGRA_KERNEL_OUT zImage
make O=$TEGRA_KERNEL_OUT dtbs
make O=$TEGRA_KERNEL_OUT modules
make O=$TEGRA_KERNEL_OUT modules_install INSTALL_MOD_PATH=<your_destination>
cd <modules_install_path>
tar --owner root --group root -cjf kernel_supplements.tbz2 lib/modules

fifth, replace the Image zImage dtb and kernel_supplements.tbz2 in the “/Linux_for_Tegra/kernel”
finally, reflash my TX1.

sudo ./apply_binaries.sh
sudo ./flash.sh jetson-tx1 mmcblk0p1

now in my TX1 , “uname -r” show “3.10.96+”

Is it not necessary to replace the kernel_supplement.tbz2 in the “/Linux_for_Tegra/kernel”

thank you linuxdev for your patience!
I have found this reason!
when “make menuconfig”
“>> General setup -->>Local version - append to kernel release” add “-tegra”
but “Automatically append version information to the version string” is crucial importance , it must be “excluded”!

but for new kernel sources, when “make menuconfig”, it is “excluded” default. this condition that i compile new kernel sources is inconformity with that i compile an older kernel sources.

Good catch :) Simple things are not always simple to find.