Jetson TX2 stuck in force recovery mode

Hi, I have a custom made carrier board and am trying to debug an issue where the Jetson TX2 module is stuck in force recovery mode. I have verified that the force recovery input signal is behaving correctly. Can someone suggest other areas of the hardware to investigate next that could be causing this?

Thanks,
Doug Burrell

Hi, can you please share more details? Do you have any log? How do you make sure it is in recovery mode? What’s the current value at this point?

I can see the unit enumerated over USB and when I run the flash script this is the output:

Bus 002 Device 070: ID 0955:7c18 NVidia Corp.
Bus 002 Device 050: ID 2109:2812 VIA Labs, Inc. VL812 Hub
Bus 002 Device 048: ID 067b:2303 Prolific Technology, Inc. PL2303 Serial Port
Bus 002 Device 046: ID 2109:2812 VIA Labs, Inc. VL812 Hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
root@l157:/usr/src/jetpack/64_TX2/Linux_for_Tegra_tx2# ./flash.sh smartsense-1 mmcblk0p1
flash.sh requires root privilege
root@l157:/usr/src/jetpack/64_TX2/Linux_for_Tegra_tx2# sudo ./flash.sh smartsense-1 mmcblk0p1
./tegraflash.py --chip 0x18 --applet “/usr/src/jetpack/64_TX2/Linux_for_Tegra_tx2/bootloader/mb1_recovery_prod.bin” --cmd “dump eeprom boardinfo cvm.bin” --skipuid
Welcome to Tegra Flash
version 1.0.0
Type ? or help for help and q or quit to exit
Use ! to execute system commands

[ 0.0030 ] Generating RCM messages
[ 0.0048 ] tegrarcm_v2 --listrcm rcm_list.xml --chip 0x18 --download rcm /usr/src/jetpack/64_TX2/Linux_for_Tegra_tx2/bootloader/mb1_recovery_prod.bin 0 0
[ 0.0059 ] RCM 0 is saved as rcm_0.rcm
[ 0.0071 ] RCM 1 is saved as rcm_1.rcm
[ 0.0072 ] List of rcm files are saved in rcm_list.xml
[ 0.0072 ]
[ 0.0072 ] Signing RCM messages
[ 0.0095 ] tegrasign_v2 --key None --list rcm_list.xml --pubkeyhash pub_key.key
[ 0.0121 ] Assuming zero filled SBK key
[ 0.0162 ]
[ 0.0163 ] Copying signature to RCM mesages
[ 0.0187 ] tegrarcm_v2 --chip 0x18 --updatesig rcm_list_signed.xml
[ 0.0201 ]
[ 0.0201 ] Boot Rom communication
[ 0.0209 ] tegrarcm_v2 --chip 0x18 --rcm rcm_list_signed.xml --skipuid
[ 0.0216 ] Boot Rom communication failed
[ 3.2072 ]
Error: Return value 3
Command tegrarcm_v2 --chip 0x18 --rcm rcm_list_signed.xml --skipuid
Reading board information failed.

Please look into the chapter of “strapping” in OEM DG to check if any strapping pins are wrongly pull-up or pull-down which could cause the boot failure.

Thanks for the tip on the strapping pins. I checked these all out and have verified that none of the strapping pins are being incorrectly pulled up or down.

I should mention that we have previously built some boards and they work successfully. Just recently however, we’ve built up another batch of boards and this is where we’re seeing the issue where the TX2 module is stuck in force recovery mode.

Thanks,
Doug Burrell

Another tidbit of info to share: if I put a previously flashed TX2 from another “good” board onto the “bad” board it boots but only partially. Here is the serial port output:

[ 6.822471] xhci-tegra 3530000.xhci: Firmware timestamp: 2017-03-31 06:21:56 UTC, Version: 55.07 release
[ 6.834011] xhci-tegra 3530000.xhci: xHCI Host Controller
[ 6.839433] xhci-tegra 3530000.xhci: new USB bus registered, assigned bus number 1
[ 6.847796] xhci-tegra 3530000.xhci: hcc params 0x0184fd25 hci version 0x100 quirks 0x00010810
[ 6.856436] xhci-tegra 3530000.xhci: irq 51, io mem 0x03530000
[ 6.862356] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
[ 6.869151] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 6.876376] usb usb1: Product: xHCI Host Controller
[ 6.881261] usb usb1: Manufacturer: Linux 4.4.38 xhci-hcd
[ 6.886662] usb usb1: SerialNumber: 3530000.xhci
[ 6.891522] hub 1-0:1.0: USB hub found
[ 6.895302] hub 1-0:1.0: 4 ports detected
[ 6.916084] xhci-tegra 3530000.xhci: xHCI Host Controller
[ 6.921516] xhci-tegra 3530000.xhci: new USB bus registered, assigned bus number 2
[ 6.929200] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003
[ 6.936005] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 6.943238] usb usb2: Product: xHCI Host Controller
[ 6.948125] usb usb2: Manufacturer: Linux 4.4.38 xhci-hcd
[ 6.953532] usb usb2: SerialNumber: 3530000.xhci
[ 6.958372] hub 2-0:1.0: USB hub found
[ 6.962148] hub 2-0:1.0: 3 ports detected
[ 6.966481] tegra-xotg xotg: otg: host 3530000.xhci registered
[ 7.064190] xhci-tegra 3530000.xhci: entering ELPG
[ 7.071606] xhci-tegra 3530000.xhci: entering ELPG done
[ 95.492563] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[ 99.608351] eqos 2490000.ether_qos eth0: Link is Up - 1Gbps/Full - flow control rx/tx
[ 99.617242] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Press Enter for maintenance
(or press Control-D to continue):
root@tegra-ubuntu:~#

No abnormal message found… Suggest to check the hardware change list to locate the issue, seems it is a circuit design problem.

What about the time difference between these 2 lines:

[ 7.071606] xhci-tegra 3530000.xhci: entering ELPG done
[ 95.492563] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready

and the fact that link is not ready? On a good board the link comes up immediately, there is not 90 second time difference.

Hi dburrell,

Following log looks normal as it is probing eth0.

[ 95.492563] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[ 99.608351] eqos 2490000.ether_qos eth0: Link is Up - 1Gbps/Full - flow control rx/tx
[ 99.617242] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready

Does system still hang after ethernet is ready?

No, the system does not hang after ethernet is ready but it only partially boots and I am not able to ssh into the unit.

On a working board the boot behaviour is different. There is no 90 second hang before setting up ethernet, it typically happens at about time 7 seconds. And there is no log entry stating that eth0 link is not ready, it just comes up fine right away.

It sort of feels like a timeout. Can you watch your router logs in real time to see when an address is requested and replied to via DHCP? It might be interesting if the router itself might say something about that MAC address.