[SOLVED]Installing driver and compiling kernel - educating needed :)

Autonomous Machines Jetson & Embedded Systems Jetson TX1
1

I am currently involved in a project where I must interact with a USB analyser device. To do this I need to 1) install the driver on the Jetson TX1 and 2) use the libraries provided to interface with the device. The manufacturer has provided a .c file and a Makefile to generate the driver .ko file as well as the .cpp and .h files to create the library .so file. My coding experience has been primarily creating GUIs in C++ (Qt), and my Linux knowledge is rapidly increasing while being on the project, however is still minimal.

I am just struggling to formulate and summarize the information I’ve found from many sources. I am aiming to better my understanding of the processes required for this to work. Below are some things I am struggling with:

  1. To install the driver on my virtual box 64-bit Ubuntu I would just run ‘make’ and the .ko file is created and it’s added to /lib/modules, a line is added to the etc/modules folder and the device is good to go. It is my understanding that since the user space is 32-bit and the kernel is 64-bit, I cannot compile the driver on the Jetson tx1. I don’t exactly understand this 32/64 bit situation, could someone explain this?

  2. Now, it was very easy to install it on Ubuntu 64-bit, but I see it gets a whole lot harder for the tx1. I have read that I need to compile the tx1 kernel (i.e. OS?) using another Linux system. Now, I haven’t really attempted it yet, but once you have a compiled tx1 kernel, where does the implementation/compiling of the driver come about? Is the result of this cross-compiling a .ko file that can go directly to the Jetson?

Any help would be appreciated. Please, please forgive my ignorance. Thanks :)

2

I’m also learning about this stuff too but I can relate some of my experience with building kernel modules on the TX1. I found that I could build and install kernel modules on the TX1 itself (no need for a cross compiler). Here is how I did it, perhaps someone more experienced then me can help refine this.

Disclaimer: I am using the 24.1 version of L4T so I don’t know if there are differences with the 23.2 and 24.1

I wrote a kernel driver template for myself (I’m sure there are a lot more out there that are much better than this) but I’ll use it for this demo.

https://github.com/CospanDesign/kernel-module

Do the following steps inside a terminal on the TX1

  1. Run ‘make modules_prepare’ in the kernel headers directory. I think this is important to prepare the build system to link your module with the currently running kernel.
cd /usr/src/linux-headers-3.10.96-tegra
sudo make modules_prepare
  1. Clone, build and install module
mkdir Projects
cd Projects
git clone https://github.com/CospanDesign/kernel-module.git
cd kernel-module
make
sudo make install

Here is what the entire process looks like

buntu@tegra-ubuntu:~$ cd /usr/src/linux-headers-3.10.96-tegra/
ubuntu@tegra-ubuntu:/usr/src/linux-headers-3.10.96-tegra$ sudo make modules_prepare
[sudo] password for ubuntu: 
  CHK     include/generated/uapi/linux/version.h
  CHK     include/generated/utsrelease.h
make[1]: `include/generated/mach-types.h' is up to date.
  CC      scripts/mod/devicetable-offsets.s
  GEN     scripts/mod/devicetable-offsets.h
  HOSTCC  scripts/mod/file2alias.o
  HOSTLD  scripts/mod/modpost
ubuntu@tegra-ubuntu:/usr/src/linux-headers-3.10.96-tegra$ cd ~/
ubuntu@tegra-ubuntu:~$ git clone https://github.com/CospanDesign/kernel-module.git
Cloning into 'kernel-module'...
remote: Counting objects: 77, done.
remote: Total 77 (delta 0), reused 0 (delta 0), pack-reused 77
Unpacking objects: 100% (77/77), done.
Checking connectivity... done.
ubuntu@tegra-ubuntu:~$ cd kernel-module/
ubuntu@tegra-ubuntu:~/kernel-module$ make
make -C /home/ubuntu/kernel-module/src
make[1]: Entering directory `/home/ubuntu/kernel-module/src'
make -C /lib/modules/3.10.96-tegra/build M=/home/ubuntu/kernel-module/src modules
make[2]: Entering directory `/usr/src/linux-headers-3.10.96-tegra'
  CC [M]  /home/ubuntu/kernel-module/src/mymodule.o
  Building modules, stage 2.
  MODPOST 1 modules
  CC      /home/ubuntu/kernel-module/src/mymodule.mod.o
  LD [M]  /home/ubuntu/kernel-module/src/mymodule.ko
make[2]: Leaving directory `/usr/src/linux-headers-3.10.96-tegra'
make[1]: Leaving directory `/home/ubuntu/kernel-module/src'
ubuntu@tegra-ubuntu:~/kernel-module$ sudo make install
sudo cp rules/61-mymodule.rules /etc/udev/rules.d/
Installing Drivermake -C /home/ubuntu/kernel-module/src install
make[1]: Entering directory `/home/ubuntu/kernel-module/src'
sudo insmod ./mymodule.ko
make[1]: Leaving directory `/home/ubuntu/kernel-module/src'
ubuntu@tegra-ubuntu:~/kernel-module$ lsmod
Module                  Size  Used by
mymodule                2232  0 
rfcomm                 66118  0 
bnep                   14864  2 
bcmdhd               7453607  0 
cfg80211              452766  1 bcmdhd
bluedroid_pm           11452  0 

ubuntu@tegra-ubuntu:~/kernel-module$ echo "Removing Module..."
Removing Module...
ubuntu@tegra-ubuntu:~/kernel-module$ sudo make remove
sudo rm /etc/udev/rules.d/61-mymodule.rules
make -C /home/ubuntu/Projects/kernel-module/src remove
make[1]: Entering directory `/home/ubuntu/Projects/kernel-module/src'
sudo rmmod mymodule
make[1]: Leaving directory `/home/ubuntu/Projects/kernel-module/src'

To modify this code to build your own file you should be able to replace the ‘mymodule.c’ file with the source file you received from the manufacturer and then go into the ‘Makefile’ within the ‘src’ directory and change it so that ‘TARGET’ and ‘obj-m’ is pointing to your filename.

As an example if I were to change the module to point to a new file called ‘my_new_module’ the process would look like this:

cd <kernel-module-base>/src
mv mymodule.c my_new_module.c
sed -i 's/mymodule/my_new_module/g' Makefile
cd ..
make clean
make
sudo make install
lsmod

WARNING: Your module might be intended for a future version of the kernel

I originally wrote my driver for the 4.6 version of the kernel and needed to modify it a bit to make it work on the 3.10 currently used by the TX1.

Experienced kernel driver writers will have preprocessor conditional statements to adjust various aspects of the driver depending on what version is available. If the driver is written with this in mind then it should compile cleanly otherwise you can use the link below to help you modify the driver.

http://lxr.free-electrons.com/

It’s very helpful because it will let you isolate a specific version of the kernel. For the TX1 you will need to select the ‘3.10’

I hope you will have an easier time writing a kernel module than I did.

Dave

3

About the first question, see [url]https://devtalk.nvidia.com/default/topic/962782/jetson-tx1/problems-communicating-via-usb-to-serial-cp210x-/post/4969368/#4969368[/url].

Like @cospan I am referring to R24.1, so if you are not using this there will possibly be differences, but some R24.1-specific info is here:
[url]https://devtalk.nvidia.com/default/topic/936880/jetson-tx1/jetson-tx1-24-1-release-need-help-with-complier-directions-can-not-complie/post/4885136/#4885136[/url]

Normally someone might just build directly in the source tree and use the install targets to not worry about individual files. In that case you don’t really need to use outside directories to separate build and temporary stuff from the kernel source (although this does have advantages even for native builds). As soon as you are cross compiling for a different machine using those environment variables to point temporary and build output to somewhere else becomes helpful.

In the kernel output directory for 64-bit you’ll find “arch/arm64/boot/Image”. In some form (or perhaps naming scheme to keep unique) this goes in the Jetson’s “/boot” directory (this is unnecessary if you are only changing modules and not the kernel itself). If 64-bit compression were supported in u-boot you’d use zImage instead.

If you do the modules install command they end up in that alternate modules output location. There is always a subdirectory for modules of the scheme “lib/modules/${uname -r}/”. Be sure to set CONFIG_LOCALVERSION if you want “uname -r” to have a suffix matching an existing kernel…this is what you would want if just building modules. If you are building a new kernel with changes to integrated features you’d change CONFIG_LOCALVERSION, and from the unmodified older module directory you’d still recursively copy the subdirectory “extra” into your new “lib/modules/${uname -r}/” directory (the “extra” is provided during flash via the “apply_binaries.sh” step…you can just copy those files from the original module directory when you build a completely new kernel).

It is unlikely you need to change anything for DTB or firmware files. These are probably correct across both module and integrated feature changes.

Essentially the install steps which are automated copy new modules into the modules directory. If the kernel were changed, and not just modules, then the Image is also copied…possibly with a new name and extlinux.conf edit. I like renaming Image to “Image-${uname -r}” to make it distinct from the original kernel…you never know when you might want the old kernel for testing or rescue.

4

I don’t have the Jetson with me but this is pretty much the same method that I have used, however I keep getting the error “gcc: error: unrecognized comand line option ‘-mgeneral-regs-only’” when running ‘make install’ which leads me to here:

https://devtalk.nvidia.com/default/topic/906942/failed-to-compile-kernel-module-gcc-cannot-recognize-aarch64-option-mgeneral-regs-only-/

Which goes into compiling the kernel itself, which is where I am having problems. Have you seen this problem error code before?

5

The URL I gave earlier lists a couple of edits for bugs…one fixes the “-mgeneral-regs-only” issue:
[url]https://devtalk.nvidia.com/default/topic/936880/jetson-tx1/jetson-tx1-24-1-release-need-help-with-complier-directions-can-not-complie/post/4885136/#4885136[/url]

6

I am in a similar situation as the OP and have compiled a bunch of notes here [url]https://gist.github.com/chutsu/9bb6abe6f61924c88521adec859c7006[/url], please let me know if they make sense or help.

The instructions are mostly built upon David Soto’s wiki ([url]http://developer.ridgerun.com/wiki/index.php?title=Compiling_Tegra_X1_source_code[/url]), however the instructions are very slightly outdated (only a small change in directory layout of the Jetpack).

7

Thank you all for your help. I managed to get it done using these instructions (similar to a lot on this site, however a tad more clear):

[url]https://devtalk.nvidia.com/default/topic/906942/jetson-tx1/failed-to-compile-kernel-module-gcc-cannot-recognize-aarch64-option-mgeneral-regs-only-/post/4940753/#4940753[/url]

Just for anyone who reads this and struggled like me, the steps I used to get the driver compiled are as follows:

  1. Now go into the driver source folder and edit the Makefile, ensuring that the KDIR := $TEGRA_MODULES_OUT/build (in my case)

  2. Build the driver .ko file using the same cross-compile environment as before:

    sudo make ARCH=arm64 CROSS_COMPILE=/usr/local/gcc-linaro-5.2-2015.11-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu- CROSS32CC=/usr/local/gcc-linaro-5.2-2015.11-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-gcc

  3. Copy the .ko file from the linux host into the jetson directory /lib/modules/uname -r/kernel/drivers/etcetcetc (or where ever yours are supposed to go).

  4. Add the name of the .ko file (without ‘.ko’) to its own line in /etc/modules using sudo gedit then run “sudo depmod”.

  5. Copy provided .rules file or add your own to etc/udev/rules.d/ then run “sudo restart udev” if required.