Python3 Tainted kernel/module with Intel D415

Autonomous Machines Jetson & Embedded Systems Jetson TX2
#1

Could someone please explain to me what is the message I am getting?
It seems to me like a kernel module/driver crash, but I can’t really decode it.
It mentions v4l, so I am guessing it is the v4l driver that crashes?

[  +0.016904] CPU: 0 PID: 2646 Comm: python3 Tainted: G        W       4.4.38+ #7
[  +0.007294] Hardware name: quill (DT)
[  +0.003654] task: ffffffc1eae6e400 ti: ffffffc1eceb0000 task.ti: ffffffc1eceb0000
[  +0.007470] PC is at v4l_enum_fmt+0x1270/0x1608
[  +0.004520] LR is at v4l_enum_fmt+0x1270/0x1608
[  +0.004520] pc : [<ffffffc000752c10>] lr : [<ffffffc000752c10>] pstate: 60000145
[  +0.007378] sp : ffffffc1eceb3bf0
[  +0.003307] x29: ffffffc1eceb3bf0 x28: ffffffc0012ba2c0 
[  +0.005320] x27: 0000000000000000 x26: ffffffc1eceb3d20 
[  +0.005319] x25: 0000007fdbdf0450 x24: ffffffbffcf7a288 
[  +0.005319] x23: ffffffc15201b280 x22: ffffffc1579ec700 
[  +0.005320] x21: ffffffbffcf7a288 x20: 0000000000000000 
[  +0.005321] x19: ffffffc1eceb3d20 x18: 0000000000000014 
[  +0.005320] x17: 0000007fa6dc5290 x16: ffffffc0001d5974 
[  +0.005322] x15: 0000000000000028 x14: 595559560000001e 
[  +0.005320] x13: 000001e000000280 x12: ffffffc001218000 
[  +0.005318] x11: 0000000000000002 x10: 0000000000000000 
[  +0.005319] x9 : 00000000000004ba x8 : 0000000000000002 
[  +0.005321] x7 : 0000000000000000 x6 : 000000000000002e 
[  +0.005320] x5 : 0000000000000000 x4 : 0000000000000000 
[  +0.005320] x3 : 0000000000000000 x2 : 0000000000000000 
[  +0.005319] x1 : 0000000000000000 x0 : 000000000000001e 

[  +0.006805] ---[ end trace afbf1dd302f53e83 ]---
[  +0.004606] Call trace:
[  +0.002440] [<ffffffc000752c10>] v4l_enum_fmt+0x1270/0x1608
[  +0.005560] [<ffffffc00075375c>] __video_do_ioctl+0x224/0x298
[  +0.005734] [<ffffffc0007531d4>] video_usercopy+0x22c/0x574
[  +0.005561] [<ffffffc000753530>] video_ioctl2+0x14/0x1c
[  +0.005213] [<ffffffc00074e55c>] v4l2_ioctl+0xbc/0xcc
[  +0.005042] [<ffffffc0001d56b4>] do_vfs_ioctl+0x324/0x5e4
[  +0.005387] [<ffffffc0001d59f8>] SyS_ioctl+0x84/0x98
[  +0.004955] [<ffffffc000084ff0>] el0_svc_naked+0x24/0x28
[  +0.038996] Unknown pixelformat 0x00000000
[  +0.004125] ------------[ cut here ]------------
[  +0.004607] WARNING: at ffffffc000752c10 [verbose debug info unavailable]
[  +0.006772] Modules linked in: fuse ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat ip_tables uvcvideo videobuf2_vmalloc bcmdhd pci_tegra bluedroid_pm

I had opened an issue with Intel on Github, but all they did was tell me to update my L4T kernel.
Also, if I do dist-upgrade on L4T will I get a newer kernel from the L4T repository, or will I brake my system?

#2

I can’t answer everything, but a “dist-upgrade” will make the system unusable. A newer kernel might not be an option.

Did you custom build the kernel? I see the “uname -r” has changed to “4.4.38+”, which means the default kernel module search location has changed. If you did not build and/or copy modules to this new location, then anything which is a module will fail.

And yes, this is V4L crashing.

#3

At some point I built a kernel module for an Orbitty Carrier board, hasn’t given me any trouble other than with the D415. I don’t remember rebuilding the kernel my self, only that I downloaded some patches.

#4

Does “uname -r” show “4.4.38+”? If so, do you have a full set of kernel modules at this location?

/lib/modules/4.4.38+/
#5

Hmm, the plot thickens.

Yes uname -r shows 4.4.38+ and in those are the modules I have, with two of them being broken symlinks

ls -l /lib/modules/4.4.38+/
total 208
lrwxrwxrwx 1 root root    60 Nov  7  2017 build -> /disk2/parker/linux4tegra/branches/28.1_cam/tegra_kernel_out
drwxr-xr-x 8 root root  4096 Nov  7  2017 kernel
-rw-r--r-- 1 root root 40173 Nov  7  2017 modules.alias
-rw-r--r-- 1 root root 42139 Nov  7  2017 modules.alias.bin
-rw-r--r-- 1 root root 19500 Nov  7  2017 modules.builtin
-rw-r--r-- 1 root root 20834 Nov  7  2017 modules.builtin.bin
-rw-r--r-- 1 root root  6992 Nov  7  2017 modules.dep
-rw-r--r-- 1 root root 12534 Nov  7  2017 modules.dep.bin
-rw-r--r-- 1 root root    70 Nov  7  2017 modules.devname
-rw-r--r-- 1 root root  4884 Nov  7  2017 modules.order
-rw-r--r-- 1 root root   161 Nov  7  2017 modules.softdep
-rw-r--r-- 1 root root 10321 Nov  7  2017 modules.symbols
-rw-r--r-- 1 root root 13254 Nov  7  2017 modules.symbols.bin
lrwxrwxrwx 1 root root    85 Nov  7  2017 source -> /disk2/parker/linux4tegra/branches/28.1_cam/Linux_for_Tegra/sources/kernel/kernel-4.4
ubuntu@tegra-ubuntu:/lib/modules/4.4.38+$ file build
build: broken symbolic link to /disk2/parker/linux4tegra/branches/28.1_cam/tegra_kernel_out
ubuntu@tegra-ubuntu:/lib/modules/4.4.38+$ file source
source: broken symbolic link to /disk2/parker/linux4tegra/branches/28.1_cam/Linux_for_Tegra/sources/kernel/kernel-4.4
ubuntu@tegra-ubuntu:/lib/modules/4.4.38+$
#6

“build” and “source” are not actually modules and can be ignored. These can be used by package systems when building for the “current” kernel. This isn’t used with Jetsons.

What do you see via:

find /lib/modules/4.4.38+ -type f

What do you see from:

find /lib/modules/4.4.38-tegra -type f
#7

find /lib/modules/4.4.38+ -type f

outputs tons modules of (too many to copy-paste)

kernel/drivers/
kernel/fs/
kernel/net
etc...

Same with find /lib/modules/4.4.38-tegra -type f

Both have

/lib/modules/4.4.38-tegra/kernel/drivers/media/v4l2-core/videobuf2-vmalloc.ko
/lib/modules/4.4.38+/kernel/drivers/media/v4l2-core/videobuf2-vmalloc.ko

I see that

v4l_enum_fmt

is exported by

uvcvideo.ko

which exists in both directories.

I see both uvcvideo and videobuf2_vmalloc loaded, although i don’t know which version.
The v4l2_ioctl seems to be exported by the linux kernel? At which point I don’t know how to proceed.

I think this is the guide I had followed about a year ago:


and the repository with the scripts that built it and installed it:

I have Jetson 3.1 on the workstation so I’m assuming it is the one that is flashed, had to also use Orbitty’s custom firmware for the carrier board and to enable USB3.

I may have R28.1 or R28.2, don’t know how to find out from the board.
Looking at the archives I’d say R28.1

Intel on Github told me to update to 3.2:

But at this stage, 3 days before shipping off for testing I can’t do that.
I might make an image of it just for a back-up and try again in a month if this can’t be fixed.
I’ve managed to make the camera work by soft resetting the usb every time, but there’s obviously a problem somewhere.

#8

I’m not sure that the RealSense driver is even involved. The OOPS seems to be V4L2 triggered from a Python program. On the other hand, I’m not sure what it takes to get the RealSense driver working…I have seen many comments on this, but am not sure of the status.

It does look like you have kernel modules fully installed for both “uname -r” versions (original plus the one you are now using). Someone else may know more about RealSense status and whether this is possibly part of the V4L2 problem.

#9

Thanks for helping @linuxdev!
Yes the python app is what I’ve written, it uses pyrealsense, which uses the C++ bindings that rely on the intel driver which uses v4l.
I might just bite the bullet end of the month and try an upgrade to see if it fixes anything.