Xavier NX not starting nor flashing after apt-get update/upgrade

Robotics & Edge Computing Jetson Systems Jetson Xavier NX
,
1

Hello,

First and foremost, our issue is the same reported as this one

Today, after apt-get update/upgrade our Xavier NX was not starting anymore. We then tried to flash the device and got this exception :

[   5.3954 ] tegradevflash_v2 --oem platformdetails eeprom cvm /data/Linux_for_Tegra/bootloader/cvm.bin
[   5.3968 ] CPU Bootloader is not running on device.
[   5.5382 ] 
Error: Return value 4
Command tegradevflash_v2 --oem platformdetails eeprom cvm /data/Linux_for_Tegra/bootloader/cvm.bin
Reading board information failed.

Context

We use a custom docker based image to flash our Xavier NX through the eye4 baseboard. The rootfs used was customized for our needs for a previous L4T 32.5.0 and all nvidia prefixed .deb were installed on this custom rootfs from this version of JP.

We updated our Dockerfile multistage to take the latest L4T 32.5.2 and just reused our rootfs mentioned above. The dockerfile is formed as a set of stages building custom kernel, dtb, build custom drivers, update pinmux, customize l4t_initrd and then expose flash.sh as an entrypoint.

pls note, we did not apply_binaries.sh as the rootfs was already prepared with nvidia prefixed .deb version from 32.5.0.

The rootfs get the /etc/apt/source... configured with t194 r32.5

Flash was working every time during days using this method, and finally we decided to update our running and fresh flashed Xavier NX (32.5.2) with apt using basic apt-get update / upgrade as mentioned here

While doing it we noticed at this time the nvidia-l4t-bootloader updating (from 32.5.0 to 32.5.1) as mentioned in the linked post. After rebooting, the Xavier NX was not starting, I tried many times to switch to recovery mode and re-flash and reached again and again

[   5.3939 ] 
[   5.3954 ] tegradevflash_v2 --oem platformdetails eeprom cvm /data/Linux_for_Tegra/bootloader/cvm.bin
[   5.3968 ] CPU Bootloader is not running on device.
[   5.5382 ] 
Error: Return value 4
Command tegradevflash_v2 --oem platformdetails eeprom cvm /data/Linux_for_Tegra/bootloader/cvm.bin
Reading board information failed.

Nvidia ? Pls advice on this and confirm we need to RMA ?

2

Do you have NVIDIA devkit? Can you try to move this module to NVIDIA devkit and see if it can get flashed?

You shouldn’t directly do apt-get upgrade on custom board. Need to remove nvidia related apt source from it first.

And please make sure you know the correct number of release name… you just shared some release versions that don’t exist…

3

Thanks for your answer, corrected the typo on the version.

We don’t have devkits, we work on production module w/ EMMC on our carrier board. I can just confirm on other Xavier NX or TX2 NX with the same carrier board we can flash and/or start a new module.

As for the apt … Could you pls point any documentations mentioning that apt-get upgrade on custom board can’t be done using nvidia apt source related ? As this would be a major issue / breaker for many carrier manufacturers like us.

Could you please advise how to resolve this issue ?

4

  1. Is it possible to share the uart log during the flash on the problematic module?

  2. There is no explicit indication about the apt source on custom board. The developer guide won’t just tell you such answer boldly. But you can find it out easily if you know how apt get upgrade is working.

https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/updating_jetson_and_host.html

Just make it easier to understand, the apt upgrade will do the OTA update so it will install pure jetpack to your board again. Since most of custom board has their own kernel and dtb, such installation will just make your own dtb gone. For custom board, it means it has chance to not boot into kernel and bootloader and other I/O failure. The easiest way is just remove NV related source from apt source list.

However, this won’t make your board not able to get flashed. Also, for a vendor like you, it is always better to have a devkit with you. We won’t be able to have every vendors’ carrier board, thus, you need to reproduce issue over devkit so that we can help.

5

Thank you, We know basic principles about apt but not specific related to OTA and fact that kernel and dtb could gone ; but make sense after reviewing the nvidia packages supplied.

However, as you pointed out, this should not affect the device to get flashed. Therefore I’m a bit skeptical, as the issue reported today looks the same on the post linked following the same pattern.

We will try to address the 1. you mentioned with UART and will post the log here Meanwhile we will find a nvidia devkit to flash.

6

If even devkit cannot get flashed, then please RMA this module. Devkit can be kept for future debug use.