Jetson Nano in infinite reboot loop

Hello everybody,

I have a Jetson Nano since few weeks that I was using for astrophotography application with an IMX477 sensor. It is installed in a metal case with power on, reset and FR mode buttons. The kind of case very common on online shopping sites.

Suddenly few days ago it started to randomly power off, I had to manually push the power on button to restart it. After a while of this behaviour it never come up again.

I can power on, the green led turns on, but no system is booted.

I connected the serial debug port and I see the following repeated in loop:

[0000.125] [L4T TegraBoot] (version 00.00.2018.01-l4t-8728f3cb)
[0000.130] Processing in cold boot mode Bootloader 2
[0000.135] A02 Bootrom Patch rev = 1023
[0000.138] Power-up reason: pmc por
[0000.141] No Battery Present
[0000.144] pmic max77620 reset reason
[0000.147] pmic max77620 NVERC : 0x40
[0000.151] RamCode = 0
[0000.153] Platform has DDR4 type RAM
[0000.156] max77620 disabling SD1 Remote Sense
[0000.161] Setting DDR voltage to 1125mv
[0000.165] Serial Number of Pmic Max77663: 0x1f26e4
[0000.172] Entering ramdump check
[0000.175] Get RamDumpCarveOut = 0x0
[0000.179] RamDumpCarveOut=0x0,  RamDumperFlag=0xe59ff3f8
[0000.184] Last reboot was clean, booting normally!
[0000.188] Sdram initialization is successful 
[0000.192] SecureOs Carveout Base=0x00000000ff800000 Size=0x00800000
[0000.199] Lp0 Carveout Base=0x00000000ff780000 Size=0x00001000
[0000.204] BpmpFw Carveout Base=0x00000000ff700000 Size=0x00080000
[0000.210] GSC1 Carveout Base=0x00000000ff600000 Size=0x00100000
[0000.216] GSC2 Carveout Base=0x00000000ff500000 Size=0x00100000
[0000.222] GSC4 Carveout Base=0x00000000ff400000 Size=0x00100000
[0000.228] GSC5 Carveout Base=0x00000000ff300000 Size=0x00100000
[0000.234] GSC3 Carveout Base=0x000000017f300000 Size=0x00d00000
[0000.250] RamDump Carveout Base=0x00000000ff280000 Size=0x00080000
[0000.256] Platform-DebugCarveout: 0
[0000.259] Nck Carveout Base=0x00000000ff080000 Size=0x00200000
[0000.265] Non secure mode, and RB not enabled.
[0000.269] BoardID = 3448, SKU = 0x0
[0000.272] QSPI-ONLY: SkipQspiOnlyFlag = 0
[0000.276] Nano-SD: checking PT table on QSPI ...
[0000.281] Initialize FailControl
[0000.284] Read PT from (2:0)
[0000.314] *** crc32 (df5e36b4) checksum error (exp:ce24a198) ***
[0000.320] *** Set to failover in the next boot ***
[0000.325] NvTbootFailControlSetClobber:
[0000.328] *** Rebooting ***

I tried to write a new SD but I have the same behaviour.

Do you know what is happening and how to solve the problem?

Unfortunately I have ubuntu 20.04 and I have some difficulties running the SDKManager.

I managed to use the SDKManager docker image.

I am using the command (found here Jetson Nano unable to boot and flash ):

docker run -it --rm --privileged -v /dev:/dev --name JetPack_Jetson_Nano sdkmanager:1.6.0.8170 --cli install --logintype devzone --product Jetson --version 4.6 --targetos Linux --host --target JETSON_NANO_TARGETS --flash all --license accept --staylogin true --datacollection enable

During the flash process I have the following error on the serial console:

[0002.407] Processing in recovery mode
[0002.411] A02 Bootrom Patch rev = 1023
[0002.414] Power-up reason: software reset
[0002.418] Established communication link with host
[0003.422] NvTbootI2cWrite(): error code 0x00045100 Error while starting write transaction
[0003.430] NvTbootI2cDeviceRead(): error code 0x00045001 Error while sending the offset to slave
[0003.439] NvTbootI2c: Read failed for slave 0xac, offset 0x00 with error code 0x00045001

and the flash process fails

1 Like

Good to know the docker command worked for you. I ordered a serial cable to also check those logs. Hopefully, I will be able to see what the errors are then.

PS. I am also interested in astrophotography with the jetson. What software and equipment are you using? And what are your results?

Hi,

Could you add the -v /dev/bus/usb:/dev/bus/usb/ to see if it works?
You can find an example in the below document:
https://docs.nvidia.com/sdk-manager/docker-containers/index.html#additional-considerations

Thanks.

I followed the example in the document, commited the previous state and relaunched adding -v /dev/bus/usb:/dev/bus/usb/ and there was indeed some progress.

At first it progressed enough to show the same issue about libxml2-utils as Jetson Nano unable to boot and flash

Instead of doing a new fixed container I just launched a bash into the running container and installed it to progress, but I think it should be fixed upstream.

After installing libxml2-utils it progressed further and now the serial debug console is showing:

[0096.840] Processing in recovery mode
[0096.844] A02 Bootrom Patch rev = 1023
[0096.847] Power-up reason: software reset
[0096.851] Established communication link with host
[0097.837] Odmdata from BCT: 0x000a4000
[0097.841] DebugPort= 0x3
[0097.843] SkipQspiOnly= 0x0
[0097.895] BoardId read from EEPROM/NCT: 3448
[0097.899] BoardID = 3448, SKU = 0x0
[0097.926] NvTbootInit_Porg: Setting GPIO_A6 for HDMI ...
[0097.932] max77620 setting MBLPD bit
[0097.935] No Battery Present
[0097.938] RamCode = 0
[0097.940] Platform has DDR4 type RAM
[0097.944] max77620 disabling SD1 Remote Sense
[0097.948] Setting DDR voltage to 1125mv
[0097.952] Serial Number of Pmic Max77663: 0x1f26e4
[0097.959] Entering ramdump check
[0097.962] Get RamDumpCarveOut = 0x0
[0097.966] RamDumpCarveOut=0x0,  RamDumperFlag=0xe59ff3f8
[0097.971] Last reboot was clean, booting normally!
[0097.976] Sdram initialization is successful 
[0097.980] SecureOs Carveout Base=0x00000000ff800000 Size=0x00800000
[0097.986] Lp0 Carveout Base=0x00000000ff780000 Size=0x00001000
[0097.992] BpmpFw Carveout Base=0x00000000ff700000 Size=0x00080000
[0097.998] GSC1 Carveout Base=0x00000000ff600000 Size=0x00100000
[0098.003] GSC2 Carveout Base=0x00000000ff500000 Size=0x00100000
[0098.009] GSC4 Carveout Base=0x00000000ff400000 Size=0x00100000
[0098.015] GSC5 Carveout Base=0x00000000ff300000 Size=0x00100000
[0098.021] GSC3 Carveout Base=0x000000017f300000 Size=0x00d00000
[0098.037] RamDump Carveout Base=0x00000000ff280000 Size=0x00080000
[0098.043] Platform-DebugCarveout: 0
[0098.047] Downloaded BCT successfully

but on the sdkmanager container the process still fails, the first error is:

error: Board ID(3448) version(400) 

If I reset the board I’m still in the infinite loop of reboots as before.

Actually my error was before yours, I’m not sure I can say it “worked”. Now I encounter the same libxml2-utils issue and others later on.

I was still at the very beginning of the setup when the board stopped working. I was not experimenting any particular software on the jetson side, just a gstreamer pipeline to dump a video for later photo stacking on the host PC. I think the jetson can do much more.

1 Like

That doesn’t change the command really. /dev:/dev just maps the whole dev directory instead of only the usb.

Instead of launching the full CLI I tried to launch it directly from a bash in the container using the command suggested here ERROR : Flash Jetson Nano : Board ID(3448) version(200)

However the flashing fails with the following error:

Error: Return value 1
Command tegrarcm --download ebt cboot.bin.encrypt 0 0 --download rp1 tegra210-p3448-0000-p3449-0000-b00.dtb.encrypt 0
Failed flashing t210ref.

I attach here the full log of the flash.sh run. I think I ran out of tests I can do since I don’t know what the return value 1 means and the debug serial console do not show any further information.

nvidia@ac08af3bb3d3:~/nvidia/nvidia_sdk/JetPack_4.6_Linux_JETSON_NANO_TARGETS/Linux_for_Tegra$ sudo ./flash.sh jetson-nano-qspi-sd mmcblk0p1
###############################################################################
# L4T BSP Information:
# R32 , REVISION: 6.1
###############################################################################
# Target Board Information:
# Name: jetson-nano-qspi-sd, Board Family: t210ref, SoC: Tegra 210, 
# OpMode: production, Boot Authentication: , 
# Disk encryption: disabled ,
###############################################################################
./tegraflash.py --chip 0x21 --applet "/home/nvidia/nvidia/nvidia_sdk/JetPack_4.6_Linux_JETSON_NANO_TARGETS/Linux_for_Tegra/bootloader/nvtboot_recovery.bin" --skipuid --cmd "dump eeprom boardinfo cvm.bin" 
Welcome to Tegra Flash
version 1.0.0
Type ? or help for help and q or quit to exit
Use ! to execute system commands
 
[   0.0028 ] Generating RCM messages
[   0.0050 ] tegrarcm --listrcm rcm_list.xml --chip 0x21 0 --download rcm /home/nvidia/nvidia/nvidia_sdk/JetPack_4.6_Linux_JETSON_NANO_TARGETS/Linux_for_Tegra/bootloader/nvtboot_recovery.bin 0 0
[   0.0058 ] RCM 0 is saved as rcm_0.rcm
[   0.0067 ] RCM 1 is saved as rcm_1.rcm
[   0.0067 ] List of rcm files are saved in rcm_list.xml
[   0.0067 ] 
[   0.0068 ] Signing RCM messages
[   0.0092 ] tegrasign --key None --list rcm_list.xml --pubkeyhash pub_key.key
[   0.0099 ] Assuming zero filled SBK key
[   0.0159 ] 
[   0.0160 ] Copying signature to RCM mesages
[   0.0185 ] tegrarcm --chip 0x21 0 --updatesig rcm_list_signed.xml
[   0.0204 ] 
[   0.0204 ] Boot Rom communication
[   0.0229 ] tegrarcm --chip 0x21 0 --rcm rcm_list_signed.xml --skipuid
[   0.0239 ] RCM version 0X210001
[   0.0680 ] Boot Rom communication completed
[   1.0761 ] 
[   1.0764 ] dump EEPROM info
[   1.0822 ] tegrarcm --oem platformdetails eeprom /home/nvidia/nvidia/nvidia_sdk/JetPack_4.6_Linux_JETSON_NANO_TARGETS/Linux_for_Tegra/bootloader/cvm.bin
[   1.0849 ] Applet version 00.01.0000
[   1.1033 ] Saved platform info in /home/nvidia/nvidia/nvidia_sdk/JetPack_4.6_Linux_JETSON_NANO_TARGETS/Linux_for_Tegra/bootloader/cvm.bin
[   1.1803 ] 
[   1.1830 ] tegrarcm --reboot recovery
[   1.1845 ] Applet version 00.01.0000
[   1.2032 ] 
Board ID(3448) version(400) 
copying bctfile(/home/nvidia/nvidia/nvidia_sdk/JetPack_4.6_Linux_JETSON_NANO_TARGETS/Linux_for_Tegra/bootloader/t210ref/BCT/P3448_A00_lpddr4_204Mhz_P987.cfg)... done.
copying bootloader(/home/nvidia/nvidia/nvidia_sdk/JetPack_4.6_Linux_JETSON_NANO_TARGETS/Linux_for_Tegra/bootloader/t210ref/cboot.bin)... done.
copying initrd(/home/nvidia/nvidia/nvidia_sdk/JetPack_4.6_Linux_JETSON_NANO_TARGETS/Linux_for_Tegra/bootloader/l4t_initrd.img)... done.
Making Boot image... done.
Existing sosfile(/home/nvidia/nvidia/nvidia_sdk/JetPack_4.6_Linux_JETSON_NANO_TARGETS/Linux_for_Tegra/bootloader/nvtboot_recovery.bin) reused.
copying tegraboot(/home/nvidia/nvidia/nvidia_sdk/JetPack_4.6_Linux_JETSON_NANO_TARGETS/Linux_for_Tegra/bootloader/t210ref/nvtboot.bin)... done.
copying cpu_bootloader(/home/nvidia/nvidia/nvidia_sdk/JetPack_4.6_Linux_JETSON_NANO_TARGETS/Linux_for_Tegra/bootloader/t210ref/cboot.bin)... done.
copying bpffile(/home/nvidia/nvidia/nvidia_sdk/JetPack_4.6_Linux_JETSON_NANO_TARGETS/Linux_for_Tegra/bootloader/t210ref/sc7entry-firmware.bin)... done.
copying wb0boot(/home/nvidia/nvidia/nvidia_sdk/JetPack_4.6_Linux_JETSON_NANO_TARGETS/Linux_for_Tegra/bootloader/t210ref/warmboot.bin)... done.
Existing tosfile(/home/nvidia/nvidia/nvidia_sdk/JetPack_4.6_Linux_JETSON_NANO_TARGETS/Linux_for_Tegra/bootloader/tos-mon-only.img) reused.
Existing eksfile(/home/nvidia/nvidia/nvidia_sdk/JetPack_4.6_Linux_JETSON_NANO_TARGETS/Linux_for_Tegra/bootloader/eks.img) reused.
copying dtbfile(/home/nvidia/nvidia/nvidia_sdk/JetPack_4.6_Linux_JETSON_NANO_TARGETS/Linux_for_Tegra/kernel/dtb/tegra210-p3448-0000-p3449-0000-b00.dtb)... done.
Copying nv_boot_control.conf to rootfs
	populating kernel to rootfs... done.
	populating initrd to rootfs... done.
	populating kernel_tegra210-p3448-0000-p3449-0000-b00.dtb to rootfs... done.
Making system.img... 
	populating rootfs from /home/nvidia/nvidia/nvidia_sdk/JetPack_4.6_Linux_JETSON_NANO_TARGETS/Linux_for_Tegra/rootfs ... 	populating /boot/extlinux/extlinux.conf ... done.
	Sync'ing system.img ... done.
	Converting RAW image to Sparse image... done.
system.img built successfully. 
Existing tbcfile(/home/nvidia/nvidia/nvidia_sdk/JetPack_4.6_Linux_JETSON_NANO_TARGETS/Linux_for_Tegra/bootloader/nvtboot_cpu.bin) reused.
copying tbcdtbfile(/home/nvidia/nvidia/nvidia_sdk/JetPack_4.6_Linux_JETSON_NANO_TARGETS/Linux_for_Tegra/kernel/dtb/tegra210-p3448-0000-p3449-0000-b00.dtb)... done.
copying cfgfile(/home/nvidia/nvidia/nvidia_sdk/JetPack_4.6_Linux_JETSON_NANO_TARGETS/Linux_for_Tegra/bootloader/t210ref/cfg/flash_l4t_t210_max-spi_sd_p3448.xml) to flash.xml... done.
copying flasher(/home/nvidia/nvidia/nvidia_sdk/JetPack_4.6_Linux_JETSON_NANO_TARGETS/Linux_for_Tegra/bootloader/t210ref/cboot.bin)... done.
Existing flashapp(/home/nvidia/nvidia/nvidia_sdk/JetPack_4.6_Linux_JETSON_NANO_TARGETS/Linux_for_Tegra/bootloader/tegraflash.py) reused.
./tegraflash.py --bl cboot.bin --bct  P3448_A00_lpddr4_204Mhz_P987.cfg --odmdata 0xa4000 --bldtb kernel_tegra210-p3448-0000-p3449-0000-b00.dtb --applet nvtboot_recovery.bin  --cmd "flash; reboot"  --cfg flash.xml --chip 0x21    --bins "EBT cboot.bin; DTB tegra210-p3448-0000-p3449-0000-b00.dtb" 
saving flash command in /home/nvidia/nvidia/nvidia_sdk/JetPack_4.6_Linux_JETSON_NANO_TARGETS/Linux_for_Tegra/bootloader/flashcmd.txt
saving Windows flash command to /home/nvidia/nvidia/nvidia_sdk/JetPack_4.6_Linux_JETSON_NANO_TARGETS/Linux_for_Tegra/bootloader/flash_win.bat
assign_value: crc-flash.xml.bin 1 65520 1
printf '\x1' | dd of=crc-flash.xml.bin bs=1 seek=65520 count=1 conv=notrunc
1+0 records in
1+0 records out
1 byte copied, 8.6562e-05 s, 11.6 kB/s
assign_value: crc-flash.xml.bin 0 65521 1
printf '\x0' | dd of=crc-flash.xml.bin bs=1 seek=65521 count=1 conv=notrunc
1+0 records in
1+0 records out
1 byte copied, 7.5807e-05 s, 13.2 kB/s
assign_string: crc-flash.xml.bin PTHD 65528 4
echo PTHD | dd of=crc-flash.xml.bin bs=1 seek=65528 count=4 conv=notrunc
4+0 records in
4+0 records out
4 bytes copied, 7.2304e-05 s, 55.3 kB/s
*** Flashing target device started. ***
Welcome to Tegra Flash
version 1.0.0
Type ? or help for help and q or quit to exit
Use ! to execute system commands
 
[   0.0033 ] tegrasign --getmode mode.txt --key None
[   0.0048 ] Assuming zero filled SBK key
[   0.0068 ] 
[   0.0072 ] Generating RCM messages
[   0.0110 ] tegrarcm --listrcm rcm_list.xml --chip 0x21 0 --download rcm nvtboot_recovery.bin 0 0
[   0.0126 ] RCM 0 is saved as rcm_0.rcm
[   0.0156 ] RCM 1 is saved as rcm_1.rcm
[   0.0160 ] List of rcm files are saved in rcm_list.xml
[   0.0160 ] 
[   0.0160 ] Signing RCM messages
[   0.0186 ] tegrasign --key None --list rcm_list.xml --pubkeyhash pub_key.key
[   0.0198 ] Assuming zero filled SBK key
[   0.0263 ] 
[   0.0264 ] Copying signature to RCM mesages
[   0.0300 ] tegrarcm --chip 0x21 0 --updatesig rcm_list_signed.xml
[   0.0328 ] 
[   0.0329 ] Parsing partition layout
[   0.0357 ] tegraparser --pt flash.xml.tmp
[   0.0379 ] 
[   0.0384 ] Using default ramcode: 0
[   0.0385 ] Disable BPMP dtb trim, using default dtb
[   0.0385 ] 
[   0.0385 ] Creating list of images to be signed
[   0.0430 ] tegrahost --chip 0x21 0 --partitionlayout flash.xml.bin --list images_list.xml
[   0.0601 ] 
[   0.0605 ] Generating signatures
[   0.0628 ] tegrasign --key None --list images_list.xml --pubkeyhash pub_key.key
[   0.0641 ] Assuming zero filled SBK key
[   0.1397 ] 
[   0.1400 ] Generating br-bct
[   0.1452 ] tegrabct --bct P3448_A00_lpddr4_204Mhz_P987.cfg --chip 0x21 0
[   0.1528 ] 
[   0.1528 ] Updating boot device parameters
[   0.1553 ] tegrabct --bct P3448_A00_lpddr4_204Mhz_P987.bct --chip 0x21 0 --updatedevparam flash.xml.bin
[   0.1566 ] Warning: No sdram params
[   0.1571 ] 
[   0.1571 ] Updating bl info
[   0.1602 ] tegrabct --bct P3448_A00_lpddr4_204Mhz_P987.bct --chip 0x21 0 --updateblinfo flash.xml.bin --updatesig images_list_signed.xml
[   0.1630 ] 
[   0.1631 ] Updating secondary storage information into bct
[   0.1686 ] tegraparser --pt flash.xml.bin --chip 0x21 0 --updatecustinfo P3448_A00_lpddr4_204Mhz_P987.bct
[   0.1707 ] 
[   0.1708 ] Updating Odmdata
[   0.1779 ] tegrabct --bct P3448_A00_lpddr4_204Mhz_P987.bct --chip 0x21 0 --updatefields Odmdata =0xa4000
[   0.1814 ] Warning: No sdram params
[   0.1825 ] 
[   0.1827 ] Get Signed section of bct
[   0.1882 ] tegrabct --bct P3448_A00_lpddr4_204Mhz_P987.bct --chip 0x21 0 --listbct bct_list.xml
[   0.1921 ] 
[   0.1923 ] Signing BCT
[   0.2021 ] tegrasign --key None --list bct_list.xml --pubkeyhash pub_key.key
[   0.2048 ] Assuming zero filled SBK key
[   0.2055 ] 
[   0.2056 ] Updating BCT with signature
[   0.2084 ] tegrabct --bct P3448_A00_lpddr4_204Mhz_P987.bct --chip 0x21 0 --updatesig bct_list_signed.xml
[   0.2110 ] 
[   0.2111 ] Copying signatures
[   0.2145 ] tegrahost --chip 0x21 0 --partitionlayout flash.xml.bin --updatesig images_list_signed.xml
[   0.2218 ] 
[   0.2218 ] Updating BFS information on BCT
[   0.2250 ] tegrabct --bct P3448_A00_lpddr4_204Mhz_P987.bct --chip 0x21 0 --updatebfsinfo flash.xml.bin
[   0.2277 ] 
[   0.2278 ] Boot Rom communication
[   0.2308 ] tegrarcm --chip 0x21 0 --rcm rcm_list_signed.xml
[   0.2323 ] BR_CID: 0x321010016450f7050c00000004050200
[   0.3517 ] RCM version 0X210001
[   0.3522 ] Boot Rom communication completed
[   1.3592 ] 
[   1.3592 ] Sending BCTs
[   1.3623 ] tegrarcm --download bct P3448_A00_lpddr4_204Mhz_P987.bct
[   1.3652 ] Applet version 00.01.0000
[   1.3827 ] Sending bct
[   1.3838 ] [................................................] 100%
[   1.6094 ] 
[   1.6190 ] tegrahost --chip 0x21 --align cboot.bin
[   1.6263 ] 
[   1.6342 ] tegrahost --magicid EBT --appendsigheader cboot.bin cboot.bin_blheader
[   1.6552 ] 
[   1.6717 ] tegrasign --key None --list cboot.bin_list.xml
[   1.6803 ] Assuming zero filled SBK key
[   1.7208 ] 
[   1.7240 ] tegrahost --updatesigheader cboot.bin_blheader.encrypt cboot.bin_blheader.hash zerosbk
[   1.7274 ] 
[   1.7313 ] tegrahost --chip 0x21 --align tegra210-p3448-0000-p3449-0000-b00.dtb
[   1.7328 ] 
[   1.7358 ] tegrahost --magicid DTB --appendsigheader tegra210-p3448-0000-p3449-0000-b00.dtb tegra210-p3448-0000-p3449-0000-b00.dtb_blheader
[   1.7392 ] 
[   1.7482 ] tegrasign --key None --list tegra210-p3448-0000-p3449-0000-b00.dtb_list.xml
[   1.7526 ] Assuming zero filled SBK key
[   1.7638 ] 
[   1.7671 ] tegrahost --updatesigheader tegra210-p3448-0000-p3449-0000-b00.dtb_blheader.encrypt tegra210-p3448-0000-p3449-0000-b00.dtb_blheader.hash zerosbk
[   1.7692 ] 
[   1.7698 ] Sending bootloader and pre-requisite binaries
[   1.7729 ] tegrarcm --download ebt cboot.bin.encrypt 0 0 --download rp1 tegra210-p3448-0000-p3449-0000-b00.dtb.encrypt 0
[   1.7745 ] Applet version 00.01.0000
[   1.7926 ] Sending ebt
[   1.8976 ] 
Error: Return value 1
Command tegrarcm --download ebt cboot.bin.encrypt 0 0 --download rp1 tegra210-p3448-0000-p3449-0000-b00.dtb.encrypt 0
Failed flashing t210ref.

Just want to know that if it is possible to find a native ubuntu 18.04 to do the flash work?

I am just wondering if this issue is due to the docker or it is really a pontential hardware defect.

I managed to install ubuntu 18.04 but I think I have exactly the same behaviour. See the console and flash logs:

flash.log (39.3 KB)

console.log (1.7 KB)

Do you have other jetson devices there to validate that your host is able to flash them and only one nano has this problem?

For clarification: I didn’t install ubuntu 18.04 on the same system that I was using with docker, it’s a different machine I had around that I used to install a fresh copy of ubuntu.

I don’t have another Jetson device to test it.

Ok then please RMA this device and get a new one.