I have been investigating further with some colleagues and we discovered something interesting.
We also tested the exact same Jetson device with a flashed SD card with the Jetpack containing the Ubuntu 18.04 image and it is booting up just fine. So apparently, it is related to the software of the Jetpack and not (only) the hardware.
Question: does the installation of a system alter the hardware settings like firmware or anything else?
We also connected the UART pins to a UART-USB adapter and read the serial output. When we did this, somehow, we managed to get the splash screen with the Nvidia logo again, but then after showing the logo on a black background, it lights up quickly with a white screen and then it boes dark again.
Then, we also read the output of the serial port. This is now with a cleanly flashed SD card with the latest Jetpack containing Ubuntu 20.04, flashed with dd
. The last lines of the output before is crashes are:
[ 23.329411] x21: ffff800012014fe0 x20: 0000000000000005
[ 23.334577] hpd: switching from state 1 (Check Plug) to state 3 (Disabled)
[ 23.342877] x19: ffff8000120150b8 x18: 0000000000000010
[ 23.342891] x17: ffff800010efc870 x16: ffff800010ef9c70
[ 23.342902] x15: ffff800011d92b30 x14: ffffffffffffffff
[ 23.414720] x13: ffff8000900039e7 x12: ffff8000100039f1
[ 23.420315] x11: 616c70736944203a x10: 6572203336353637
[ 23.425651] x9 : ffff800010003d00 x8 : 79616c707369642e
[ 23.430988] x7 : ffff8000107456d0 x6 : 000000003f2588cb
[ 23.436070] x5 : 00ffffffffffffff x4 : ffff800012980000
[ 23.441664] x3 : 00000000ffffffff x2 : ffff800010236d10
[ 23.446758] x1 : ffff800011d925c0 x0 : 0000000000010001
[ 23.452078] Call trace:
[ 23.454972] tegra194_cbb_error_isr+0x198/0x1b0
[ 23.459273] __handle_irq_event_percpu+0x60/0x2a0
[ 23.463805] handle_irq_event_percpu+0x3c/0xa0
[ 23.468526] handle_irq_event+0x4c/0xf0
[ 23.472287] handle_fasteoi_irq+0xbc/0x170
[ 23.476313] generic_handle_irq+0x3c/0x60
[ 23.480354] __handle_domain_irq+0x6c/0xc0
[ 23.484280] efi_header_end+0xa8/0xf0
[ 23.488037] el1_irq+0xd0/0x180
[ 23.491032] cpuidle_enter_state+0xb4/0x400
[ 23.495211] cpuidle_enter+0x3c/0x50
[ 23.498456] call_cpuidle+0x40/0x70
[ 23.501969] do_idle+0x1fc/0x260
[ 23.505624] cpu_startup_entry+0x2c/0x70
[ 23.509131] rest_init+0xd8/0xe4
[ 23.512371] arch_call_rest_init+0x14/0x1c
[ 23.516824] start_kernel+0x4c0/0x4f4
[ 23.520174] Code: a9425bf5 a9446bf9 a8c67bfd d65f03c0 (d4210000)
[ 23.526282] ---[ end trace 83ea15fdedc9d963 ]---
[ 23.530743] Kernel panic - not syncing: Oops - BUG: Fatal exception in interrupt
[ 23.538283] SMP: stopping secondary CPUs
[ 23.542133] Kernel Offset: 0x80000 from 0xffff800010000000
[ 23.547710] PHYS_OFFSET: 0x80000000
[ 23.551317] CPU features: 0x8240002,03002a30
[ 23.555502] Memory Limit: none
[ 23.558653] trusty-log panic notifier - trusty version Built: 08:57:16 Feb 19 2022
[ 23.567033] ---[ end Kernel panic - not syncing: Oops - BUG: Fatal exception in interrupt ]---
nvidia-jetson-xavier-nx-boot-uart-j14_clean_flashed_ubuntu2004.log (80.7 KB)
Now, we also discovered, when using the UART-USB prompt, we can stop the auto-boot of the board with the Ubuntu 20.04 image when it is trying to boot. When we then enter boot
to boot the device, we get the following output:
[ 23.594372] ERROR: mmcblk0p1 mount fail...
[ 23.670398] usb 1-2.1: new high-speed USB device number 4 using tegra-xusb
[ 23.708886] usb 1-2.1: New USB device found, idVendor=0bda, idProduct=5409, bcdDevice= 0.02
[ 23.710366] usb 1-2.1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 23.711789] usb 1-2.1: Product: USB2.1 Hub
[ 23.713104] usb 1-2.1: Manufacturer: Generic
[ 23.716437] hub 1-2.1:1.0: USB hub found
[ 23.719285] hub 1-2.1:1.0: 5 ports detected
[ 23.931199] usb 2-3.1: new SuperSpeed Gen 1 USB device number 3 using tegra-xusb
[ 23.968794] usb 2-3.1: New USB device found, idVendor=0bda, idProduct=0409, bcdDevice= 0.02
[ 23.970257] usb 2-3.1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 23.971681] usb 2-3.1: Product: USB3.2 Hub
[ 23.973028] usb 2-3.1: Manufacturer: Generic
[ 23.981604] hub 2-3.1:1.0: USB hub found
[ 23.986527] hub 2-3.1:1.0: 4 ports detected
[ 24.038365] usb 1-2.1.1: new low-speed USB device number 5 using tegra-xusb
[ 24.066606] usb 1-2.1.1: New USB device found, idVendor=413c, idProduct=2113, bcdDevice= 1.10
[ 24.068046] usb 1-2.1.1: New USB device strings: Mfr=0, Product=2, SerialNumber=0
[ 24.069416] usb 1-2.1.1: Product: Dell KB216 Wired Keyboard
[ 24.077100] input: Dell KB216 Wired Keyboard as /devices/platform/3610000.xhci/usb1/1-2/1-2.1/1-2.1.1/1-2.1.1:1.0/0003:413C:2113.0001/input/input1
[ 24.138595] hid-generic 0003:413C:2113.0001: input,hidraw0: USB HID v1.11 Keyboard [Dell KB216 Wired Keyboard] on usb-3610000.xhci-2.1.1/input0
[ 24.143419] input: Dell KB216 Wired Keyboard Consumer Control as /devices/platform/3610000.xhci/usb1/1-2/1-2.1/1-2.1.1/1-2.1.1:1.1/0003:413C:2113.0002/input/input2
[ 24.202580] input: Dell KB216 Wired Keyboard System Control as /devices/platform/3610000.xhci/usb1/1-2/1-2.1/1-2.1.1/1-2.1.1:1.1/0003:413C:2113.0002/input/input3
[ 24.204450] hid-generic 0003:413C:2113.0002: input,hidraw1: USB HID v1.11 Device [Dell KB216 Wired Keyboard] on usb-3610000.xhci-2.1.1/input1
[ 24.286382] usb 1-2.1.2: new low-speed USB device number 6 using tegra-xusb
[ 24.311062] usb 1-2.1.2: New USB device found, idVendor=413c, idProduct=301a, bcdDevice= 1.00
[ 24.312599] usb 1-2.1.2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 24.314076] usb 1-2.1.2: Product: Dell MS116 USB Optical Mouse
[ 24.315966] usb 1-2.1.2: Manufacturer: PixArt
[ 24.322200] input: PixArt Dell MS116 USB Optical Mouse as /devices/platform/3610000.xhci/usb1/1-2/1-2.1/1-2.1.2/1-2.1.2:1.0/0003:413C:301A.0003/input/input4
[ 24.325186] hid-generic 0003:413C:301A.0003: input,hidraw2: USB HID v1.11 Mouse [PixArt Dell MS116 USB Optical Mouse] on usb-3610000.xhci-2.1.2/input0
[ 24.406374] usb 1-2.1.5: new high-speed USB device number 7 using tegra-xusb
[ 24.443649] usb 1-2.1.5: New USB device found, idVendor=0bda, idProduct=1100, bcdDevice= 1.01
[ 24.445251] usb 1-2.1.5: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 24.446817] usb 1-2.1.5: Product: HID Device
[ 24.448299] usb 1-2.1.5: Manufacturer: Realtek
[ 24.456425] hid-generic 0003:0BDA:1100.0004: hiddev96,hidraw3: USB HID v1.11 Device [Realtek HID Device] on usb-3610000.xhci-2.1.5/input0
[ 54.242370] vdd-1v8-sd: disabling
[ 54.243908] vdd-3v3-cvb: disabling
[ 54.245374] vdd-1v8-cvb: disabling
[ 54.246878] vdd-epb-1v0: disabling
[ 54.248317] vdd-fan: disabling
[ 54.249735] vdd_sys_en: disabling
[ 54.251177] avdd-cam-2v8: disabling
nvidia-jetson-xavier-nx-boot-uart-j14_clean_flashed_ubuntu2004_manual_boot.log (60.9 KB)
In this scenario, it still does not work, but it shows text on the screen connected to the Jetson Xavier HDMI port. It is actually the same text that is also visible on the serial prompt.
We need to have Ubuntu 20.04 for using ROS Noetic and ROS2 Foxy with a realsense camera. Using an Ubuntu 18.04 image and upgrading to Ubuntu 20.04 takes several hours to install and it is not working as well as a clean installation. The upgrade version will also not allow to use CUDA. (At least not when we did it 2 months ago). And using docker could be possible, but the additional setup and added complexity to operate the device is not interesting when it could work with a native installation.
So, is it the device or the image?