Build PCIe driver on a Jetson TX1 32 bits


I need to build miniPCIe driver sources on a Jetson TX1 tegra 32 bits.
uname - a gives: Linux tegra-ubuntu 3.10.96-tegra #1 SMP PREEMPT Tue May 17 16:31:40 PDT 2016 aarch64 aarch64 aarch64 GNU/Linux

The card is:
Driver sources are given by the manfacturer with the following makefile (KDIR updated)
obj-m := vldrivep.o
KDIR := /usr/src/linux-headers-3.10.96-tegra/
PWD := (shell pwd) default: (MAKE) -C (KDIR) SUBDIRS=(PWD)
rm -f *.o *.ko modules.order Module.symvers vldrivep.mod.c

“make” command gives the following error:
root@tegra-ubuntu:/opt/ML-MPEe-U2/VersaAPI_Linux_32bit_v1.1.0/src/vldrivep# make
make -C /usr/src/linux-headers-3.10.96-tegra SUBDIRS=/opt/ML-MPEe-U2/VersaAPI_Linux_32bit_v1.1.0/src/vldrivep
make[1]: Entering directory /usr/src/linux-headers-3.10.96-tegra' LD /opt/ML-MPEe-U2/VersaAPI_Linux_32bit_v1.1.0/src/vldrivep/built-in.o CC [M] /opt/ML-MPEe-U2/VersaAPI_Linux_32bit_v1.1.0/src/vldrivep/vldrivep.o <b>gcc: error: unrecognized command line option ‘-mgeneral-regs-only’</b> make[2]: *** [/opt/ML-MPEe-U2/VersaAPI_Linux_32bit_v1.1.0/src/vldrivep/vldrivep.o] Error 1 make[1]: *** [_module_/opt/ML-MPEe-U2/VersaAPI_Linux_32bit_v1.1.0/src/vldrivep] Error 2 make[1]: Leaving directory /usr/src/linux-headers-3.10.96-tegra’
make: *** [default] Error 2

I tried to link gcc with both compilers with the same result:
/usr/bin/gcc-4.8: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.32, BuildID[sha1]=a42b62e7497d7f55673b6e49d8839b975579bf4c, stripped
/usr/bin/gcc-5: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.32, BuildID[sha1]=d869661663b3385b908c53934c4512bbe9a9a28f, stripped

Thanks for your help

See this on cross compiling the kernel:

Basically you need to be able to cross compile the kernel before trying to build modules. The above URL should give the information required for that.

Thanks for your reply.
I’ll try to cross compile my module on a virtual box Ubuntu 14.04.1 x64 but I don’t understand why I need to cross compile the kernel.

However I hope there is a way to compile my module directly on Tegra without needing to perform cross compilation.


A short explanation on why cross compile is here:

So, I tried cross-compilation on a Linux Ubuntu x64, my target (Jetson Tx1) is still arm 32-bit. I managed to compile my module for arm 64-bit but not for arm 32-bit.

I get these errors:
root@ssa-vbox:/opt/jetson/kernel# make -j8 ARCH=arm CROSS_COMPILE=/usr/local/gcc-linaro-5.3.1-2016.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu- CROSS32CC=/usr/local/gcc-linaro-5.3.1-2016.05-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-gcc O=$TEGRA_KERNEL_OUT prepare
GEN /opt/jetson/build/Makefile
CHK include/generated/uapi/linux/version.h
Using /opt/jetson/kernel as source for kernel
CHK include/generated/utsrelease.h
make[2]: ‘include/generated/mach-types.h’ is up to date.
CC kernel/bounds.s
aarch64-linux-gnu-gcc: error: unrecognized argument in option ‘-mabi=apcs-gnu’
aarch64-linux-gnu-gcc: note: valid arguments to ‘-mabi=’ are: ilp32 lp64
aarch64-linux-gnu-gcc: error: unrecognized command line option ‘-mapcs’
aarch64-linux-gnu-gcc: error: unrecognized command line option ‘-mno-sched-prolog’
aarch64-linux-gnu-gcc: error: unrecognized command line option ‘-msoft-float’
/opt/jetson/kernel/./Kbuild:35: recipe for target ‘kernel/bounds.s’ failed
make[2]: *** [kernel/bounds.s] Error 1
/opt/jetson/kernel/Makefile:848: recipe for target ‘prepare0’ failed
make[1]: *** [prepare0] Error 2
Makefile:130: recipe for target ‘sub-make’ failed
make: *** [sub-make] Error 2


The “ARCH” part is for the kernel, which is 64-bit even if user space is 32-bit. You must use “ARCH=arm64” to compile a kernel or kernel modules. The “CROSS32CC” takes care of subsets of 32-bit code within the build.

In terms of compiling 32-bit user space software, you would not involve the kernel source for most cases (libraries interfacing with the kernel would be a typical exception). If you want 32-bit support in a driver under a 64-bit kernel, I believe you would need to rewrite the driver or go through extra steps when mixing it in 64-bit (I don’t know what those changes would be, but it won’t be the exact setup from simply copying the 32-bit driver to the 64-bit kernel…other changes would be mandatory).

Thanks for your answer but I don’t understand what you mean.

We have a Jetson TX1 with a 32-bit kernel. We need to compile a module for this. So as my initial post says, native Jetson 32-bit compiler gives an error when trying to compile my module source (*.c) on the TX1: 'gcc: error: unrecognized command line option ‘-mgeneral-regs-only’. This *.c is written for 32-bit.
So following your advice, I tried cross compilation on another machine with Ubuntu 64-bit without success when ARCH=arm and with success when ARCH=arm64.

Do you mean that the compiled module for arm64 will work on the 32-bit kernel Jetson TX1 ?

No Jetson TX1 kernel is 32-bit, regardless of L4T release. All kernels are 64-bit.

What differs is the root file system (user space, versus kernel space). The R23.x series is 32-bit user space, R24.1 has both 32-bit and 64-bit user space available, R24.2 is only available with 64-bit user space. The CPUs are natively 64-bit, but can operate in a 32-bit compatibility mode. So you have to differentiate between whether code is run as an ordinary application (user space), or if it is running in the kernel (such as a driver). Under R24.2 they are the same. Under most of R24.1 they are the same (to not be 64-bit you’d have to specifically use the 32-bit rootfs). Under any R23.x they differ (a case of both rootfs and kernel always differing). Some code within the kernel either runs as 32-bit for legacy reasons (such as the code from a user space 32-bit application), but will always be in compatibility mode…if user space has something which runs as 32-bit code, it switches to compatibility mode…two compilers are needed because of this.

So yes, arm64 works on a Jetson TX1 driver. So does 32-bit if 32-bit is detected and compatibility mode is configured and detected…it would be rare for 32-bit code to be used as a driver, it would be common that a user space application uses 32-bit mode.

The JTK1 is purely 32-bit. Support for JTX1 is shifting to purely 64-bit (which has performance benefits).