How to use HIDdev device on Jetson Nano 4GB

Please read my answer once again, 4.9.299-usbdev has been booted, kernel is properly compiled, HIDDEV is enabled.

It can not be true, otherwise how old kernel did it? In the old kernel there was # CONFIG_USB_HIDDEV is not set, it is provided in my previous posts.

I did everything as per your instructions. Or please send me the screenshot what exactly I have to change in nconfig just to verify I did set all modules correctly.

So, are there any Nvidia employee available on this forum at all? How they provide support for their products?

I saw this:

Unfortunately, your suggested extlinux.conf file changes didn’t boot into new kernel 4.9.299-usbdev, but always to current 4.9.299-tegra

I interpreted your previous content to think that the “uname -r” still included the “-tegra” suffix. My mistake. Are all of your modules now found at here in addition to uname -r being “4.9.299-usbdev”?
/lib/modules/4.9.299-usbdev/kernel

If so, then this part is present. That means the HIDDEV is present and supported. The only step after that is for the device plugin to associate with the driver. This might imply a need for a udev trigger (udev is part of announcing USB devices to the hotplug mechanism, and can also customize associations or device names).

Does the working system have any software added to it for that device? On the working system can you post the output of:

cd /etc/udev
tree

(you might need to “sudo apt-get install tree”)

NVIDIA did not compile Ubuntu. They added drivers and boot content. Being that this is an embedded system with limited resources, there is a lot of content a PC would just add which is not added by default for reasons of limited resources. So it is a question of finding what content that device requires. The kernel itself now supports HIDDEV, so it becomes possible for the device to associate to the driver. I’m trying to find out on the working system if some file is associated with the device in its udev directory.

You are right, one file must be copied into /etc/udev folder, I did it and USB Dongle is working now. File content just add user access rights

#Enable user access to USB dongle
ATTRS{idVendor}=="1bc0", ATTRS{idProduct}=="8101", MODE:="0666"

The only issue now is that GUI is not loading anymore on compiled 4.9.299-usbdev kernel, I can access device only via network SSH connection. Where can I find boot log on Jetson Nano to show it here?

There is an NVIDIA GUI kernel module that needs to load. This is the parent directory in the old kernel:
/lib/modules/4.9.299-tegra/kernel/drivers/gpu/nvgpu/

This is the new parent directory:
/lib/modules/4.9.299-usbdev/kernel/drivers/gpu/nvgpu/

The file name is “nvgpu.ko”.

Before I say more I need to emphasize something about why we had to build a whole kernel and the modules, such that we did not reuse existing modules…

Whenever we load a module there is a mechanism by which a module’s signature is installed at some offset address. If we have two kernel which have the same integrated/base features (the “=y” features), then we can always add more modules and they’ll work with that kernel. In the case of deleting or adding a feature which is integrated, the loading of the modules might change. This can cause failure or odd behavior by inserting a module from one kernel’s (base feature set of “=y”) expectations of how to load and instead inserting to a different kernel. This is why we built not only the new kernel Image file, but also the modules (despite needing the same modules).

It sounds like (and this is common) you didn’t get the new nvgpu.ko file. If you check the old content:

cd /lib/modules/4.9.299-tegra/kernel/drivers/gpu/nvgpu/
ls

…you’ll find there is a version of nvgpu.ko. If you go to the new module location, it is very likely missing:

cd /lib/modules/4.9.299-usbdev/kernel/drivers/gpu/nvgpu/
ls

Do you still have your kernel source temporary output directory? You should be able to go to either the kernel output location, or the modules output location, and find nvgpu.ko. Using our previous abbreviations which we had set to environment variables (you’ll have to know where that is, they don’t save across boots or across command line terminals), you could go to one of these:

cd $TEGRA_KERNEL_OUT
cd drivers/gpu/nvgpu
ls
# Or you could:
cd $TEGRA_MODULES_OUT
cd lib/modules/4.9.299-usbdev
cd drivers/gpu/nvgpu
ls

Do you see “nvgpu.ko” in either of those locations? The $TEGRA_KERNEL_OUT would also have some intermediate .o files, while the subdirectory in $TEGRA_MODULES_OUT will just be the .ko file. If you have either of those, then copy it (the version built with “-usbdev”) to:
/lib/modules/4.9.299-usbdev/kenrel/drivers/gpu/nvgpu/nvgpu.ko

If not, then maybe we missed something in the configuration.

Both folders does contain nvgpu.ko file:

jetson@192.168.1.115:/lib/modules/4.9.299-tegra/kernel/drivers/gpu/nvgpu$ ls
nvgpu.ko

jetson@192.168.1.115:/lib/modules/4.9.299-usbdev/kernel/drivers/gpu/nvgpu$ ls
nvgpu.ko

nvgpu.ko in all three folders does have same size:

# System Folder
jetson@192.168.1.115:/lib/modules/4.9.299-usbdev/kernel/drivers/gpu/nvgpu$ ls -l nvgpu.ko
-rw-rw-r-- 1 jetson jetson 89684200 nvgpu.ko

# $TEGRA_KERNEL_OUT
jetson@192.168.1.115:~/output/drivers/gpu/nvgpu$ ls -l nvgpu.ko
-rw-rw-r-- 1 jetson jetson 89684200 nvgpu.ko

# $TEGRA_MODULES_OUT
jetson@192.168.1.115:~/modules/lib/modules/4.9.299-usbdev/kernel/drivers/gpu/nvgpu$ ls -l nvgpu.ko
-rw-rw-r-- 1 jetson jetson 89684200 nvgpu.ko

Maybe this information will be helpful for you:

jetson@192.168.1.115:~$ systemctl status proc-sys-fs-binfmt_misc.mount
● proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System
   Loaded: loaded (/lib/systemd/system/proc-sys-fs-binfmt_misc.mount; static; vendor preset: enabled)
   Active: failed (Result: exit-code) since Tue 2023-06-13 11:13:22 EEST; 34min ago
    Where: /proc/sys/fs/binfmt_misc
     What: binfmt_misc
     Docs: https://www.kernel.org/doc/html/latest/admin-guide/binfmt-misc.html
           https://www.freedesktop.org/wiki/Software/systemd/APIFileSystems
  Process: 4933 ExecMount=/bin/mount binfmt_misc /proc/sys/fs/binfmt_misc -t binfmt_misc (code=exited, status=32)

jūn 13 11:13:22 JETSON4GB systemd[1]: Mounting Arbitrary Executable File Formats File System...
jūn 13 11:13:22 JETSON4GB mount[4933]: mount: /proc/sys/fs/binfmt_misc: unknown filesystem type 'binfmt_misc'.
jūn 13 11:13:22 JETSON4GB systemd[1]: proc-sys-fs-binfmt_misc.mount: Mount process exited, code=exited status=32
jūn 13 11:13:22 JETSON4GB systemd[1]: proc-sys-fs-binfmt_misc.mount: Failed with result 'exit-code'.
jūn 13 11:13:22 JETSON4GB systemd[1]: Failed to mount Arbitrary Executable File Formats File System.
jūn 13 11:13:22 JETSON4GB systemd[1]: proc-sys-fs-binfmt_misc.mount: Start request repeated too quickly.
jūn 13 11:13:22 JETSON4GB systemd[1]: proc-sys-fs-binfmt_misc.mount: Failed with result 'exit-code'.
jūn 13 11:13:22 JETSON4GB systemd[1]: Failed to mount Arbitrary Executable File Formats File System.

I don’t think that matters, at least not for now. If you run “lsmod”, do you see “nvgpu”? If not, what do you see from using insmod on it (you might need to use the full path)? If the module is already loaded, then graphical mode should work; if not, and if the module can load, then it might just be a case of needing to run “sudo depmod -A” or “depmod -a”. If that module loads, then it is possible for graphics to succeed.

Should it need more debugging for the GUI, we’ll want this:

  • A new full serial console boot log.
  • A copy of the Xorg log. If you run this command it’ll tell you which log that is:
    ls -ltr /var/log/Xorg.*.log | tail -n 1

Noup, nothing:

jetson@192.168.1.115:~$ lsmod
Module                  Size  Used by

Here is the output:

jetson@192.168.1.115:~$ insmod /lib/modules/4.9.299-usbdev/kernel/drivers/gpu/nvgpu/nvgpu.ko
insmod: ERROR: could not insert module /lib/modules/4.9.299-usbdev/kernel/drivers/gpu/nvgpu/nvgpu.ko: Operation not permitted
jetson@192.168.1.115:~$ sudo depmod -A
depmod: ERROR: could not fstatat(/lib/modules/4.9.299-usbdev, modules.dep): No such file or directory

I do not have USB UART serial cable, are there any other way how to save this boot log?

Please see attached below:
Xorg.0.log (7.6 KB)

This requires using sudo. Was your user root?

You can first run “sudo -s” to drop into a root shell, or you can prefix with:

sudo insmod /lib/modules/4.9.299-usbdev/kernel/drivers/gpu/nvgpu/nvgpu.ko

(be sure to monitor “dmesg --follow” prior to this to see messages in case it does succeed)

This did the trick, you are true Linux GURU, Jetson Nano GUI is loaded now and USB Dongle is working as required.

Thank you a lot for your huge effort to find solution for this case. Very last question, is it the only way how to get it working, compile new kernel with USB HID support and boot from it? There is no option to extract driver now from working system and install them on another Jetson Nano like I can do it on Windows OS?

P.S. Please drop me PM with some crypto wallet address or other way I can donate you some beer or whatever you like for your help! :)

No donation required, I’d get bored if I didn’t do this, but the thought is appreciated. What follows isn’t a “rant”, but it’ll look like it. The long explanation below is more or less about how and why PCs and embedded systems differ on Linux as to what you find by default, and a comparison of both the cost and benefit of Windows for driver developers.


Windows is doing exactly what you just did. Their database is just better.

Just to illustrate, on your desktop PC, look what you have here, which is for a number of network cards:

# The $(uname -r) syntax will self-substitute...it'll fill that in for you:
cd /lib/modules/$(uname -r)/kernel/drivers/net/ethernet
ls

Look at all of those ethernet card drivers! Then look at the same location on a Jetson. The Jetson still has a lot of drivers, but not as many as the desktop PC. To see what package something belongs to, I’ll illustrate with the Intel e100 driver (this driver has been around a very long time):
dpkg -S /lib/modules/$(uname -r)/kernel/drivers/net/ethernet/intel/e100.ko
(Intel has several models, but this is for the specific chipset NIC)

Check the package on both the Jetson and the PC. You’ll find it shows up for the PC, but not the Jetson. It doesn’t mean the Jetson one does not exist.

Before more description, remember these details about building the driver:

  • You had to correctly set up the configuration to match the running system if you were to add a module. When you change the base integrated kernel Image, then you were relegated to rebuilding the entire kernel and all modules. Otherwise the module might not load.
  • You had to use the same kernel source code version to build a module which would load into an existing running kernel. We’re talking “kernel space” drivers, not user space programs.
  • Desktop PCs have a fixed amd64/x86_64 architecture. So they can easily use the mainline kernel source for all of those. You have modified kernel source because the hardware is unique. Look at other files on your system, e.g., look at commands in “/usr/bin/*”. You’ll see those all have a standard package on both the Jetson and the PC. We’re talking “user space” applications, not drivers.
  • User space has a more or less “generic” architecture to build against. Any arm64/aarch64 user space Linux program can run on any other arm64/aarch64 user space computer so long as drivers are present for access to the hardware. A PC’s user space program can be compiled and used interchangeably with any other PC of the amd64/x86_64 architecture, provided the drivers to access the hardware are present.
  • The key to really understand this is realizing that it is the kernel which provides the generic user space environment whereby it is easy for anyone to compile a program and get it it run in user space. However, the kernel itself inherits its environment on bare metal, without any help, and must inherit from what boot code sets up. All of that boot code is custom not only to the hardware drivers, but also to their physical address and other electrical layout setup. If you have a BIOS, like a PC, then it presents a uniform boot environment which makes it possible to standardize the kernel itself for packaging; if you do not have a BIOS, such as Jetsons, then a custom hardware bring-up on bare metal will also mean you need to customize some details of the kernel when compiling it. Both the source code for hardware and the layout differ on custom devices. So the modules are not interchangeable, and the carrier board hardware bare metal during boot which the kernel depends on demands the kernel itself be customized.

NVIDIA could have added all of the drivers you see on a desktop PC. There are 100 Gb/s (not Mb/s, but Gigabit) backbone network cards the Internet itself might use, costing thousands of dollars (maybe even tens of thousands or hundreds of thousands), which could have been added, but you’ll never use them. There are odd and relatively known hardware devices which PCs have inherited since the 1990s, and which were never removed, while 64-bit ARM first started life near 2015. The list for ARM 64-bit of inherited drivers is smaller. That in itself is not conclusive of anything, but it sets the stage.

You have a tiny computer on the Jetson. Because of the customizations and no BIOS, every driver might need custom setup. They’re all available on the kernel source you have, but to use them you must compile them against this custom source (if you remove part of the integrated configuration, or add new things to that configuration, then the interchangeable modules no longer guarantee the ability to interchange). But someone has to do it before you can load the module or build a new Image for the integrated kernel.

Windows for desktop PCs has drivers which are all built by them or their customers who paid money to Microsoft. Each kernel patch release for Windows is the same source code (Microsoft doesn’t exactly make all of their source code freely available the way Linux does, so Microsoft has to be the one to do most of this work). You have the kernel source for the Linux kernel, see what happens when you ask Microsoft for their kernel source. If you want to design and build a custom piece of hardware to run with Windows which does not have a BIOS, and which is to be minimal for an embedded tiny device, see what happens when you tell Microsoft you want them to build custom boot code for you. You won’t like the answer, although if you are a really big corporation and need a barrier to entry for people who don’t have a lot of money, then you might possibly like this.

The reality is that Microsoft controls the hardware requirements and is able to build all of the drivers for their kernel source in a standard environment. Linux for PCs does this too, although the kernel source is available and anyone can customize it. The user space part of custom devices can be uniform as well even on custom Linux devices. NVIDIA is the one which provides (configures and builds) a lot of device drivers in the kernel they ship, and many of those drivers you’ll never use. They’re the most popular drivers. NVIDIA makes that source and the base configuration available so you can build any driver they did not add.

Building drivers on Linux has an initial learning curve, but it isn’t actually all that bad. The Kconfig system does a lot to make it easier. You do have to understand some basics, e.g., knowing what your initial configuration is and matching that prior to making modifications, but once you’ve done it a couple of times it becomes trivial. This is kind of expected in the Linux community if you go beyond “standard” hardware, and sometimes even then. It lets you do things with odd hardware you cannot do with Windows unless you sign an NDA with Microsoft and pay a huge fee for access to their internals. It makes their “standardized” environment more user friendly for end users, at least when using common hardware or when working with big companies which have the money to develop drivers on Windows. As soon as you step into the custom hardware, you’re screwed on Windows as either a small company or a hobbyist.

As any company which wants to be able to audit code and find weaknesses and fix them, Linux is way ahead, Working in Windows either requires you to reverse engineer or pay lots of money and NDA (so you couldn’t fix bugs or flaws for everyone, just for you), or let Microsoft do all the work for something they might not care about because not enough people are suffering from the flaw.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.