Installing Xillybus Driver on Jetson TX2 with modified dtb

I am currently modifying the dts/dtsi files to have them correct for my custom carrier board. I am following the guide here and the usb on my custom carrier board is now working https://devtalk.nvidia.com/default/topic/1049988/jetson-tx2/third-usb-on-custom-board-not-working-with-jetpack-4-2/post/5330117/#5330117.

I am now trying to test the FPGA interface and I wish to use a Xillybus device driver to do so. I am having great difficulty having a device file for the Xillybus to appear on the Jetson.

I attempted to use menuconfig to include “Xillybus generic FPGA interface” and “Xillybus over PCIe” into my Jetson TX2, however after subsequently flashing the device, the device file did not show up. Is there a step that I am missing to have this device file come up? Should I include “Xillybus with Device Tree” as well?

Thanks,
Ben

To add more context to this problem, when I run

sudo modprobe xillybus_pcie

on the Jetson TX2, flashed with the Xillybus PCIe included (not modularized), I receive the following error:

modprobe: FATAL: Module xillybus_pcie not found in directory /lib/modules/4.9.140+

When I run find on “xillybus” from the root, I recieve the following results

/run/udev/data/+module:xillybus_pcie
/run/udev/data+module:xillybus_core
/run/udev/data+drivers:pci:xillybus_pcie
/usr/src/linux-headers-4.9.140-tegra-ubuntu18.04_aarch64/kernel-4.9/drivers/char/xillybus
/usr/src/linux-headers-4.9.140-tegra-linux_x86_64/kernel-4.9/drivers/char/xillybus
/sys/class/xillybus
/sys/bus/pci/drivers/xillybus_pcie
/sys/module/xillybus_pcie
/sys/module/xillybus_pcie/drivers/pci:xillybus_pcie
/sys/module/xillybus_core

Should I just cp one of these folders into /lib/modules/4.9.140+? Should I cross compile xillybus as a module instead?

Thanks,
Ben

Attempted to write the following command in a Jetson TX2 flashed with a modularized xillybus

modprobe xillybus_pcie

I still receive the error which I described above. On the same Jetson TX2, I attempted to download the xillybus driver and make it on the Jetson itself. Upon running make, I receive the following error:

make -C /lib/modules/4.9.140+/build SUBDIRS=/home/<user name>/Downloads/xillybus/module modules
make[1]: *** /lib/modules/4.9.140+/build: No such file or directory. Stop.
Makefile:25: recipe for target 'default' failed
make: *** [default] Error 2

Upon cd-ing to the described directory, I saw that a “build” file does exist, but it is a broken link. Before flashed to the Jetson, this linked to

/<whatever>/<directories>/<before>/Linux_for_Tegra/image

This directory is my target for TEGRA_KERNEL_OUT. On top of this, (on the Jetson TX2) the directory

/lib/modules/4.9.140-tegra/

has a “build” file within it, which is properly linked to the directory

/usr/src/linux-headers-4.9.140-tegra-ubuntu18.04_aarch64/kernel-4.9

Should I recreate the build file within

/lib/modules/4.9.140+

to point to somewhere else in the file system, and if I do, what should that be? Should I rewrite the Makefile so that it tries to make the directory with the functional build link?

Thanks,
Ben

I can’t answer anything specific to the Xillybus, but some basic information might start things off.

Kernel modules must all exist on the Jetson at the location:

/lib/modules/$(uname -r)/

You have a modified kernel, and its “uname -r” is “4.9.14+”. Thus you are correct that all modules needed must be at “/lib/modules/4.9.140+/”.

The device tree goes only to the Jetson. A long time ago vendors had to add code to the mainline kernel for every variation of hardware due to different wire routing or slightly different parameters even though the driver for the hardware was the same. The kernel source tree was growing, and Linux Torvalds didn’t like it (and I’m glad he didn’t like). So a form of abstraction was used whereby the driver was the only thing which was included in the kernel itself, and a mechanism was provided to read a device tree and allow drivers to use the values from the device tree rather than hard coding that value. The device tree can be though of as a kind of way to pass arguments to drivers about the current hardware configuration. Your device tree would need to describe things like how the USB wiring is routed. I couldn’t give you a specific answer for your case.

The device tree is actually flashed to a partition these days. Early on device trees only needed to be read from the bootloader and kernel, and those understand the ext4 filesystem. It turns out that there was a need to also use device tree parameters in earlier boot stages, and those earlier boot stages cannot read ext4…thus the content goes to a binary partition. Also, a mechanism was added for signing content in support of secure boot, so you really need to flash the content to the partition in order to get signing.

I won’t be able to tell you everything you need, but someone else can probably tell you the specific file to update and then the specific command to flash just the device tree. Some variations change depending on release and perhaps carrier board.

Please always follow the L4T document for build kernel and modules if you are not sure about the steps.
https://docs.nvidia.com/jetson/l4t/index.html

An important point here is that most users don’t know they need to add “LOCALVERSION=-tegra” during their cross compile. @linuxdev also pointed out this in #4.