Failed to compile kernel module (gcc cannot recognize aarch64 option: -mgeneral-regs-only)

I’m currently working on a setup that requires communication between Jetson TX1 and FPGA through PCIE. I use the PCIE driver provided by http://xillybus.com/, which is loaded as a kernel module. I have another setup with Jetson TK1 using this driver and it works very well. However, I’m having trouble compiling the same kernel module on TX1.

The error I got during compilation is

gcc: error: unrecognized command line option ‘-mgeneral-regs-only’

This is an aarch64 specific option that the ARM A57 on TX1 should support, so I guess it’s a toolchain issue. I came across the other thread on this forum discussing about the native support for aarch64, and I follow the instructions to install arm64 version of gcc like

sudo dpkg --add-architecture arm64
sudo apt-get update
sudo apt-get install libc6:arm64 binutils:arm64 cpp-4.8:arm64 gcc-4.8:arm64

Then I got the following error when compiling

scripts/basic/fixdep: 1: scripts/basic/fixdep: Syntax error: "(" unexpected

Now I have no clue where the problem comes from. Could it be the problem of build-essential? I’ve also tried to force Kbuild to build against ARCH=arm instead of arm64, but with no luck either.

Any help is really appreciated!

I was able to build TX1 kernel from source using Ubuntu X64 14.04 and 15.10. Have you tried the following

export ARCH=arm64

sudo apt-get install gcc-aarch64-linux-gnu
sudo apt-get install gcc-arm-linux-gnueabi

export CROSS_COMPILE=/usr/bin/aarch64-linux-gnu-

export CROSS32CC=arm-linux-gnueabi-gcc

export TEGRA_KERNEL_OUT=../OutDir1

mkdir $TEGRA_KERNEL_OUT

make mrproper
make O=$TEGRA_KERNEL_OUT mrproper

make O=$TEGRA_KERNEL_OUT tegra21_defconfig

make O=$TEGRA_KERNEL_OUT zImage

You may need to change “driver/platform/tegra/tegra21_clocks.c” line 1065 from

c->state = (!is_lp_cluster() == (c->u.cpu.mode == MODE_G)) ? ON : OFF;

to

c->state = ((!is_lp_cluster()) == (c->u.cpu.mode == MODE_G)) ? ON : OFF;

I haven’t tried any cross-compilation but I’ll give it a shot. Thanks for the suggestion.

One thing which isn’t obvious is an important note in the L4T getting started documents…two cross-compilers are required for kernel cross-compile builds.

EXCERPT:

To build the Tegra Kernel
1. Export the following environment variables:
$ export <b>CROSS_COMPILE</b>=<crossbin>
$ export <b>CROSS32CC</b>=<cross32bin>gcc
$ export TEGRA_KERNEL_OUT=<outdir>
$ export ARCH=arm64
Where:
•<crossbin> is the prefix applied to form the path to the tool chain for cross compilation targeting arm64, e.g., gcc. For a Linaro tool chain, it will look something like:
<linaro_install>/aarch64-unknown-linux-gnu/bin/aarch64-unknown-linux-gnu-
Note: This example requires GCC 4.9 or above.
•<cross32bin> is the prefix applied to form the path to the tool chain for cross compilation targeting arm32, e.g., gcc. For a CodeSourcery tool chain, it will look something like:
<csinstall>/arm-2009q1-203-arm-none-linux-gnueabi/bin/arm-none-linux-gnueabi-
And CROSS32CC would be:
<csinstall>/arm-2009q1-203-arm-none-linux-gnueabi/bin/arm-none-linux-gnueabi-gcc
Note: This example requires GCC 4.7 or above.
•<outdir> is the desired destination for the compiled kernel.

Note that CROSS_COMPILE is a prefix to an aarch64 tool chain (e.g., the path plus “gcc” would name the 64-bit aarch64 gcc). CROSS32CC is the full path to the ARMv7 32-bit gcc (only the compiler itself, not the full chain…i.e., this variable ends in “gcc”).

One detail often forgotten (and causing confusion) is that most of the compile commands for menuconfig and other config can also use the “make O=$TEGRA_KERNEL_OUT” option to deal with a config to be placed in the output directory. Examples of commands usually only show this for the compile itself and fail to mention needing this in more obscure commands as well.

Hi All

I think it is worth some clarification here. OP was trying to compile the kernel on the Jetson X1 itself. The issue with this is that gcc on the Jetson is a 32 bit (arm-linux-gnueabihf) but the kernel needs to be compiled for aarch64. To remedy this I guess there are (at least) 3 options.

  1. Compile the kernel on a host machine using a cross-compile to aarch64 and then install said kernel on the Jetson.

  2. Update (or add) an aarch64 GCC on the Jetson board. I am not sure if this trashes the install on that board but there are some other threads that discuss this.

  3. Install a arm->aarch64 cross compiler on the Jetson.

So the KEY here (and something that it took me a few hours to discover yesterday ;-)) is that the default kernel on the Jetson is 64 bit (aarch64) but all the user-space code (and gcc) are 32 bit (arm).

Cheers

Stephen

PS Any feedback on the best out of 1-3 above appreciated!

I am having the exact same problem as the OP. The only difference is I’m trying to compile drivers for a Magewell Pro Capture Mini HDMI card. Error messages were exactly the same even after trying the steps the OP mentioned from the other post.

Was there any success cross-compiling? Would mind sharing how you did it? I apologize for my ignorance–I’m in pretty far over my head on this project.

If the driver is part of the Linux kernel, then the cross-compile information above should work.

Hi,

Another way to solve the “fixdep” problem is to add the option “modules_prepare” to your make command when you compile the Xillybus driver. However, I haven’t tried yet the PCIe connection with a FPGA so I can’t certified the result between this method and re-compile the kernel with the fix.

I also install the arm64 architecture as mentioned here:
https://devtalk.nvidia.com/default/topic/897280/jetson-tx1/jetson-tx1-with-l4t-23-1-doesn-t-support-native-aarch64-binaries-/post/4732697/#4732697.

P.S 1: I planned to try Xillybus and RIFFA with a Xillinx FPGA, so if someone know a blog, website or may share his experience about FPGA and the JTX1 that could be very interesting.

P.S 2: Solution found here : http://www.incentivespro.com/forum/viewtopic.php?t=214 (or http://www.mjmwired.net/kernel/Documentation/kbuild/modules.txt for more details).

EDIT: The FPGA-JTX1 PCIe communication works well with the Xillybus samples.

Was this ever resolved? I too am trying to compile drivers for a Magewell Pro HD Capture and having no luck due to this error. You’d think since their drivers are specifically FOR the TX1, they would address this issue in the installation guide.

Would be lovely if someone would post a HOWTO on this instead of piece-meal across several threads and guides. There are assumptions here, for example, that people even know how to get Linaro into their environments. This shouldn’t be assumed.

I’ve been struggling with getting cross compilation to work with the Jetson TX1 kernel source for a while and also found a lot of the posts on the forum to be only partial answers. I’m going to step through it here.

As a more direct response to the question above, the -mgeneral-regs-only error means you’re using the wrong compiler to attempt to compile the kernel source (one which doesn’t take the -mgeneral-regs-only flag). You might be trying to just compile the kernel source on the Jetson or you might be trying to cross compile on another machine and the ARCH variable is not set correctly.

Beginning Notes:
I have only needed to compile the CP210x and FTDI kernel modules. All of this should be extensible and work for any additional modules you want to compile (once your cross compilation environment is setup, it should just be a matter of changing things in menuconfig).

If you weren’t sure, all of the cross compiling should take place on a different workstation. I am using a desktop running Ubuntu 14.04 x86_64 with kernel 3.13.0-92-generic

The kernel source version I downloaded/installed was L4T R23.2, and my Jetson TX1 kernel version was 3.10.67-g458d45c (note the general kernel version should match 3.10.67 in my case). This guide should not be specific to either of these.

apt-get Dependencies:
{menuconfig}

sudo apt-get install libncurses5-dev

Setup Helpful Directories:
Anywhere you’d like, setup a directory structure to manage the kernel sources, compilers, and the output files. I used /usr/local for the unpacked, binaries of the Linaro compilers I used, and a directory structure shown below.

jetson_kernel
|
|- build
|- modules
|- kernel
   |
   |- ${UNPACKED_KERNEL_SOURCE_LOCATION}
|- bin
   |
   |- jetson_latest_kernel
      |
      |- ${KERNEL_SOURCE_TAR_LOCATION}
   |- linaro_11
      |
      |- ${LINARO_COMPILER_TARS}

# LINARO_COMPILER_BINARIES at /usr/local/

Kernel Source: https://developer.nvidia.com/embedded/dlc/l4t-23-2-kernel-sources
aarch64 compiler: http://releases.linaro.org/components/toolchain/binaries/5.2-2015.11/aarch64-linux-gnu/gcc-linaro-5.2-2015.11-x86_64_aarch64-linux-gnu.tar.xz
arm compiler: http://releases.linaro.org/components/toolchain/binaries/5.2-2015.11/arm-linux-gnueabihf/gcc-linaro-5.2-2015.11-x86_64_arm-linux-gnueabihf.tar.xz

Move each to respective dir shown above, unpack, and move to /jetson_kernel/kernel/ and /usr/local respectively as shown above.

Setup Environment Variables:
NOTE: unlike other users at https://devtalk.nvidia.com/default/topic/929186/jetson-tx1-kernel-compilation/ the Makefile for the kernel sources would NOT use my env paths for CROSS_COMPILE, CROSS32CC, or ARCH. I decided to flag these explicitly below.

export TEGRA_KERNEL_OUT=/full/path/to/jetson_kernel/build/
export TEGRA_MODULES_OUT=/full/path/to/jetson_kernel/modules/

Then env to make sure they show up.

Fixing Makefile and c file errors:
NOTE: I would recommend trying to compile things before making these changes, just to verify that they are a problem for you. They should occur while trying zImage.

Error: r7 cannot be used in asm here

sudo vim jetson_kernel/kernel/Makefile
#Search for KBUILD_CFLAGS_KERNEL (around line 378)
#Edit to add the -fomit-frame-pointer flag to read
KBUILD_CFLAGS_KERNEL := -fomit-frame-pointer

logical not is only applied to the left hand side of comparison

sudo vim jetson_kernel/kernel/drivers/platform/tegra/tegra21_clocks.c
#Search for is_lp_cluster (around line 1064)
#Edit to add parenthesis around the !is_lp_cluster() expression such that line reads
c->state = ((!is_lp_cluster()) == (c->u.cpu.mode == MODE_G)) ? ON : OFF;

Compilation Setup makes:
Note: I am compiling with -j8, you might have to change this depending on the number of cores in your computer.
Note 2: As stated above, I am specifying all of my path locations in the make because environment variables were not being inherited by the kernel source Makefile.
Note 3: Do not copy paste all of these commands in at once. Run each individually and make sure it compiles/runs through.

sudo make -j8 ARCH=arm64 CROSS_COMPILE=/usr/local/gcc-linaro-5.2-2015.11-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu- CROSS32CC=/usr/local/gcc-linaro-5.2-2015.11-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-gcc O=$TEGRA_KERNEL_OUT prepare

sudo make -j8 ARCH=arm64 CROSS_COMPILE=/usr/local/gcc-linaro-5.2-2015.11-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu- CROSS32CC=/usr/local/gcc-linaro-5.2-2015.11-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-gcc O=$TEGRA_KERNEL_OUT mrproper

sudo make -j8 ARCH=arm64 CROSS_COMPILE=/usr/local/gcc-linaro-5.2-2015.11-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu- CROSS32CC=/usr/local/gcc-linaro-5.2-2015.11-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-gcc O=$TEGRA_KERNEL_OUT tegra21_defconfig

sudo make -j8 ARCH=arm64 CROSS_COMPILE=/usr/local/gcc-linaro-5.2-2015.11-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu- CROSS32CC=/usr/local/gcc-linaro-5.2-2015.11-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-gcc O=$TEGRA_KERNEL_OUT menuconfig

In the menuconfig, you will want to go to General Setup -> Local Version and enter the dash and what comes after of uname -r on your Jetson TX1.

uname -r
3.10.67-g458d45c
-g458d45c
#^^^ What we want

Then we will want to enable the compilation of the modules we need in menuconfig. For me these are CP210x and FTDI. So I will navigate to Device Drivers -> USB Support -> USB Serial Converter Support and then type “M” in each “< >” field of the drivers I want compiled.

Once this is done, navigate to “Save” at the bottom, then “Exit” until menuconfig exits.

Finish compiling the kernel source:

sudo make -j8 ARCH=arm64 CROSS_COMPILE=/usr/local/gcc-linaro-5.2-2015.11-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu- CROSS32CC=/usr/local/gcc-linaro-5.2-2015.11-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-gcc O=$TEGRA_KERNEL_OUT zImage

sudo make -j8 ARCH=arm64 CROSS_COMPILE=/usr/local/gcc-linaro-5.2-2015.11-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu- CROSS32CC=/usr/local/gcc-linaro-5.2-2015.11-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-gcc O=$TEGRA_KERNEL_OUT dtbs

sudo make -j8 ARCH=arm64 CROSS_COMPILE=/usr/local/gcc-linaro-5.2-2015.11-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu- CROSS32CC=/usr/local/gcc-linaro-5.2-2015.11-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-gcc O=$TEGRA_KERNEL_OUT modules

sudo make -j8 ARCH=arm64 CROSS_COMPILE=/usr/local/gcc-linaro-5.2-2015.11-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu- CROSS32CC=/usr/local/gcc-linaro-5.2-2015.11-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-gcc O=$TEGRA_KERNEL_OUT modules_install INSTALL_MOD_PATH=$TEGRA_MODULES_OUT

Now your compiled kernel modules should be located in the /jetson_kernel/modules directory you created at the beginning, down in a drivers directory, under your specific device.

Ex. - /jetson_kernel/modules/lib/modules/3.10.67-g458d45c/kernel/drivers/usb/serial/cp210x.ko

Installing the Compiled Modules:
Once you have these modules compiled you just need to copy them into their respective kernel module directory on the Jetson TX1.

For me, this was done with

sudo cp cp210x.ko ftdi_sio.ko /lib/modules/3.10.67-g458d45c/kernel/drivers/usb/serial/

Then you just need to install them via

sudo depmod -a
sudo modprobe -v /lib/modules/3.10.67-g458d45c/kernel/drivers/usb/serial/cp210x.ko
sudo modprobe -v /lib/modules/3.10.67-g458d45c/kernel/drivers/usb/serial/ftdi_sio.ko

I really hope this helps anyone struggling with getting devices up and running on their Jetson TX1, and sorry for the length.

References:
https://devtalk.nvidia.com/default/topic/929186/jetson-tx1-kernel-compilation/
http://developer.ridgerun.com/wiki/index.php?title=Compiling_Tegra_X1_source_code
http://releases.linaro.org/components/toolchain/binaries/5.2-2015.11/

Thank you RACKGNOME for the wonderful post!!
I am completely new to linux / embedded system development. And faced the same problem (well, slightly different, since I am trying to get the ch341 chip to work).
together with a few tips from other posts, I finally managed it, so I am posting it here, in case someone else is facing the same issue.

Problem statement:
The sub-serial converter was taken as ftdi_sio even if ch341 driver was compiled and installed.

Background:
I followed the steps above and managed to get the driver (in my case for ch341 usb-serial converter) into the folders “/lib/modules/3.10.67-g458d45c/kernel/drivers/usb/serial/”.

I tried modprobe, and dempod just as mentioned by RACKGNOME. Rebooted the machine a few times.
However, when I plug-in the device it was recognized as ftdi_sio divice instead of the ch341.

When I look back, I guess it was because I was forcing the ftdi_sio driver to take this divice by doing

echo vendorID productID > path/to/driver/new_id

I read this from this post
https://ubuntuforums.org/showthread.php?t=2259068
So,don’t do it if your driver does not match your device. Will only give you more trouble.

If you have already done that (just like me), the below post might be useful for you.

Steps to solve:

sudo su
cd path/to/wrong/usb/driver #/sys/bus/usb/ftdi_sio/ in my case
ls #to see the device name
echo <device name> > unbind
cd path/to/wrong/usb-serial/driver #/sys/bus/usb-serial/ftdi_sio/ in my case
ls #to see the device name
echo <device name> > unbind
#now the device should be unlinked
cd path/to/correct/usb/driver #/sys/bus/usb/ch341/ in my case
ls #to see the device name
echo <device name> > bind
cd path/to/correct/usb-serial/driver #/sys/bus/usb-serial/ch341-uart/ in my case
ls #to see the device name
echo <device name> > bind

After all these steps, you can check the device tree are correct by:

root@tegra-ubuntu:/sys/bus/usb-serial/drivers/ch341-uart# tree
.
|-- bind
|-- module -> ../../../../module/ch341
|-- new_id
|-- ttyUSB0 -> ../../../../devices/platform/tegra-xhci/usb2/2-3/2-3:1.0/ttyUSB0
|-- uevent
`-- unbind