Is it possible to build kernel module in Xavier? (Solved)

guo.tang · February 1, 2020, 6:59am

When trying to build a kernel module in Xavier, got the following error

/bin/sh: 1: scripts/basic/fixdep: Exec format error

Check the tool fixdep

nvidia@jetson-0423318029348:/lib/modules/4.9.108-tegra/build$ file ./scripts/basic/fixdep
./scripts/basic/fixdep: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=86275d346cb092cbc6e626f89d46bc50594d2c92, not stripped

It appears fixdep executable installed by jetpack in Xavier is a X86-64 executable, not ARM version.

Question: Why install X86 executable in Xavier? Should we only do kernel module build in cross compilation fashion on X86 host side?

davidjuny74v9 · February 1, 2020, 6:59am

I think cross compilation is recommended because it’s faster and you will quickly run out of disk space if compiling natively. That being said, it is possible:
https://devtalk.nvidia.com/default/topic/1042005/jetson-agx-xavier/building-driver-for-asus-usb-ac-53-nano-wi-fi-adapter/post/5287207/#5287207

guo.tang · February 1, 2020, 6:59am

@davidjuny74v9

Tried cross-compilation, it works. Thanks!

WayneWWW · February 1, 2020, 6:59am

Hi,

Glad that you’ve resolved your issue.

The correct way to build kernel/kernel module is listed in L4T document.

Please note that we have updated new toolchain now. You need to use GCC 6.4.1 instead of 4.8.5.

parafin · February 1, 2020, 7:00am

Hi,

I just did a fresh install of L4T 31.0.2 and tried building an external kernel module. It fails because kernel headers installed to /usr/src/linux-headers-4.9.108-tegra are broken - several files are missing and there are binaries built for x86_64 architecture present. I’ve managed to work around this issue by doing the following:

cd /usr/src/
sudo wget -O public_sources_31.0.2.tbz2 'https://developer.nvidia.com/embedded/dlc/l4t-sources-31-0-2'
sudo tar --no-same-owner -xf public_sources_31.0.2.tbz2 public_sources/kernel_src.tbz2
sudo tar --no-same-owner -xf public_sources/kernel_src.tbz2
zcat /proc/config.gz | sudo tee kernel/kernel-4.9/.config >/dev/null
sudo sed -i '/^EXTRAVERSION/ c EXTRAVERSION = -tegra' kernel/kernel-4.9/Makefile
sudo cp linux-headers-4.9.108-tegra/{modules.builtin,modules.order,Module.symvers,System.map} kernel/kernel-4.9/
sudo ln -snf /usr/src/kernel/kernel-4.9 /lib/modules/4.9.108-tegra/build

Then I prepared the sources the usual way:

sudo make -C /lib/modules/"$(uname -r)"/build modules_prepare

After that a kernel module can be built as normal.

It would be nice if linux-headers were fixed in next release. For example everything works fine with TX1 and TX2 in L4T R28.*.

ross-newman · February 1, 2020, 7:08am

Hey parafin,

You saved me a ton of time figuring this out so big thanks for sharing. Managed to build video capture driver module on Xavier (for Sensoray mPCIe card) and it works great.

WayneWWW · February 1, 2020, 7:08am

parafin,

Sounds like an old issue even for tx2/tx1. I believe the steps from L4T document to build external modules should work fine. Could you confirm?

ross-newman · February 1, 2020, 7:08am

I will probably need to re-flash but you are right the L4T documentation implies this should work without tinkering on the target. Just need to do module_prepare. I will give this a go when I next re-flash and confirm.

I was having issues but have been hacking away so really need to re-flash before I can confirm all is good.

linuxdev · May 6, 2020, 5:40pm

FYI, you should be able to build the kernel and modules natively on the Xavier. However, you’d want to install the compatibility “gcc-6” package (this is the version 6.4 compiler mentioned in docs for kernel build):

sudo apt-get install gcc-6

Then, prior to compile:

export CC=/usr/bin/gcc-6

parafin · February 1, 2020, 7:08am

TX2/TX1 had similar issues with some versions of L4T, but they were fixed in R28.x - make modules_prepare works there as expected. On L4T R31.0.2 make modules_prepare fails. I don’t want to cross-compile external module from host PC, I need to do it on Tegra itself. So which steps are we talking about exactly?
Basically the issue here is that kernel headers that get installed in L4T R31.0.2 are useless. Something got broken in headers package generation when you moved from R28 to R31.

linuxdev · February 1, 2020, 7:08am

I just put the full source on my Xavier. Then, after the notes from #9 above, it worked for me (but I am still using R31.0.1).

Andrey1984 · February 29, 2020, 1:22am

root@jetson-123:/usr/src/linux-headers-4.9.108-tegra/kernel-4.9# make modules_prepare

seems to work, finally,
Now I can continue with my investigation of the issue of use of onboard CSI camera for video-chat purpose.
However, now I am stuck at the point where I need to rewrite the sequence:

gst-launch-1.0 nvcamerasrc ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! nvtee ! nvvidconv ! 'video/x-raw, format=(string)I420, framerate=(fraction)30/1' ! tee ! v4l2sink device=/dev/video2

WARNING: erroneous pipeline: no element "nvcamerasrc"

WayneWWW · February 1, 2020, 7:09am

Andrey1984,

Please try nvarguscamerasrc. nvcamerasrc is deprecated.

Andrey1984 · February 29, 2020, 1:57am

like that?

gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1' ! nvtee ! nvvidconv ! 'video/x-raw, format=(string)I420, framerate=(fraction)30/1' ! tee ! v4l2sink device=/dev/video2

gst-launch-1.0 -ev nvarguscamerasrc ! 'video/x-raw(memory:NVMM),width=640, height=480, framerate=30/1, format=NV12' ! nvvidconv flip-method=0 ! videoconvert ! v4l2sink device=/dev/video2
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)640, height=(int)480, format=(string)NV12, framerate=(fraction)30/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)640, height=(int)480, format=(string)NV12, framerate=(fraction)30/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)640, height=(int)480, format=(string)NV12, framerate=(fraction)30/1
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :

GST_ARGUS: 2592 x 1944 FR = 29.999999 fps; Analog Gain range min 1.000000, max 16.000000; Exposure Range min 34000, max 550385000;

GST_ARGUS: 2592 x 1458 FR = 29.999999 fps; Analog Gain range min 1.000000, max 16.000000; Exposure Range min 34000, max 550385000;

GST_ARGUS: 1280 x 720 FR = 120.000005 fps; Analog Gain range min 1.000000, max 16.000000; Exposure Range min 22000, max 358733000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 2 
   Output Stream W = 1280 H = 720 
   seconds to Run    = 0 
   Frame Rate = 120.000005 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
ERROR: from element /GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc0:
streaming stopped, reason not-negotiated (-4)
EOS on shutdown enabled -- waiting for EOS after Error
Waiting for EOS...

Please find the linked threads 1 and 2. Thanks