Orin does not boot anymore "Error: VRS10 or VRS11 interrupt is detected, power sequence stops"

Autonomous Vehicles DRIVE AGX Orin DRIVE AGX Orin General
1

Please provide the following info (tick the boxes after creating this topic):
Software Version
DRIVE OS 6.0.10.0
DRIVE OS 6.0.8.1
DRIVE OS 6.0.6
DRIVE OS 6.0.5
DRIVE OS 6.0.4 (rev. 1)
DRIVE OS 6.0.4 SDK
other

Target Operating System
Linux
QNX
other

Hardware Platform
DRIVE AGX Orin Developer Kit (940-63710-0010-300)
DRIVE AGX Orin Developer Kit (940-63710-0010-200)
DRIVE AGX Orin Developer Kit (940-63710-0010-100)
DRIVE AGX Orin Developer Kit (940-63710-0010-D00)
DRIVE AGX Orin Developer Kit (940-63710-0010-C00)
DRIVE AGX Orin Developer Kit (not sure its number)
other

SDK Manager Version
2.1.0
other

Host Machine Version
native Ubuntu Linux 20.04 Host installed with SDK Manager
native Ubuntu Linux 20.04 Host installed with DRIVE OS Docker Containers
native Ubuntu Linux 18.04 Host installed with DRIVE OS Docker Containers
other

Dear Nvidia Team,

I have the exact same issue as described in Unable to poweron Orin. The Orin does not boot anymore, only the fan keeps working. Reading the serial messages with sudo minicom -D /dev/ttyACM1 from the host machine I obtain

Error: VRS10 or VRS11 interrupt is detected, power sequence stops

What is your recommended solution for this issue? From my point of view this indicates a serious hardware issue, which cannot be circumvented easily.

2

Dear @kristian.fischer ,
May I know if the board is used in car or office premises? Could you reflash the board to see if it fixes.

4

Dear SivaRamaKrishnaNV,

The board is used in the office and stands on the desk.

I tried to reflash the board but obtained the following error messages:

root@6.0.10.0-0009-build-linux-sdk:/drive# ./flash.py /dev/ttyACM1 p3710
2024-10-18 09:50:45 INFO : Initiating pre-flash check
2024-10-18 09:50:45 INFO : Checking Serial Port Connection
2024-10-18 09:50:45 INFO : Serial Port found /dev/ttyACM1
2024-10-18 09:50:45 INFO : Running basic open/close tests on Serial Port Connection
2024-10-18 09:50:45 INFO : Serial Port is open
2024-10-18 09:50:45 INFO : Flushing input and output buffer, discard all content
2024-10-18 09:50:45 INFO : Serial Port is closed
2024-10-18 09:50:45 INFO : Checking available targets
2024-10-18 09:50:45 INFO : Checking Orin board connection
2024-10-18 09:50:45 INFO : Orin board with Device ID -- 0955:7045 found.
2024-10-18 09:50:45 INFO : Orin board with Device ID -- 0955:7023 found.
2024-10-18 09:50:45 INFO : Orin board device path is /dev/bus/usb/001/016
2024-10-18 09:50:45 INFO : Orin board device path is /dev/bus/usb/001/017
2024-10-18 09:50:45 INFO : Pre-flash check complete
2024-10-18 09:50:45 INFO : Flashing DRIVE OS Docker
2024-10-18 09:51:13 INFO : driveinstaller v2.0
2024-10-18 09:51:13 INFO : 
2024-10-18 09:51:13 INFO : SDK/PDK Installer and Flasher
2024-10-18 09:51:13 INFO : Run `./driveinstaller --help` for usage information.
2024-10-18 09:51:13 INFO : 
2024-10-18 09:51:13 INFO : Initializing board and OS configuration to Autonomous Vehicle p3710 Linux Tegra A
2024-10-18 09:51:13 INFO : Manifest version 2.0
2024-10-18 09:51:13 INFO : 
2024-10-18 09:51:13 INFO : Invalid response for Aurix command!
2024-10-18 09:51:13 INFO : 
2024-10-18 09:51:13 INFO : Attempt 1 to re-send version command
2024-10-18 09:51:13 INFO : 
2024-10-18 09:51:13 INFO : Invalid response for Aurix command!
2024-10-18 09:51:13 INFO : 
2024-10-18 09:51:13 INFO : Attempt 2 to re-send version command
2024-10-18 09:51:13 INFO : 
2024-10-18 09:51:13 INFO : Invalid response for Aurix command!
2024-10-18 09:51:13 INFO : 
2024-10-18 09:51:13 INFO : Attempt 3 to re-send version command
2024-10-18 09:51:13 INFO : 
2024-10-18 09:51:13 INFO : Invalid response for Aurix command!
2024-10-18 09:51:13 INFO : 
2024-10-18 09:51:13 INFO : Could not fetch board revision. Re-try either by re-connecting board or make sure all serial port processes such as minicom are closed.
2024-10-18 09:51:13 INFO : Failed to bind partitions!
2024-10-18 09:51:13 INFO : driveinstaller cannot continue! Check the log file for details: /drive/driveinstaller/driveinstaller.log
2024-10-18 09:51:13 ERROR : Flashing failed, please contact provider with logs /drive_flashing/log_0z7kov8mcwluirj2nag3b49sdxh1fepq.txt and /drive/driveinstaller/driveinstaller.log.

Since I was able to see two Nvidia devices when executing lsusb, I expect the board was successfully put into recovery mode. Minicom was closed during the flashing process. A second try resulted in the same behavior.

The corresponding logs suggested in above error message are attached to this post.

driveinstaller.log (13.4 KB)
log_0z7kov8mcwluirj2nag3b49sdxh1fepq.txt (1.9 KB)

5

Make sure you set the board in normal mode before attempt to flash using flash.py. Also, power off/on both host and target to close any active minicom sessions. Is aurix and tegra accessible via minicom from host after hard rebooting?

6

Dear SivaRamaKrishnaNV,

After a hard reboot of the board, Aurix (/dev/ttyACM1) is accessible with minicom, but Tegra (/dev/ttyACM1) not. Is this the supposed behavior?

If you have to set the board to normal mode before flashing, than why do you even have to put it into recovery mode before? So what steps have to be performed before putting the board in recovery mode and back to normal mode?

On another board, I was able to flash it with the same hardware setup of cables, and with the same order of commands:

  1. put board into recovery mode
#sudo minicom -D /dev/ttyACM1
#tegrarecovery x1 on
#tegrareset x1 #reset tegra
  1. close minicom
  2. lsusb now shows two Nvidia devices
  3. run docker image
  4. run flash.py script within the docker image

However, these steps do not work on the issued, possibly broken, machine.

7

The flash.py script takes care of putting the board in recovery mode and flashes the board and puts back the board in normal mode. If you plan to use bootburn script instead of flash.py, then we have to take care putting the board in recovery mode/normal mode.

When you access the tegra via minicom make sure you put the board in normal mode.

Did you test flashing with flash.py when board is in normal mode? Please share the logs. This should confirm if it is hardware issue.

8

Dear SivaRamaKrishnaNV,

when executing flash.py in normal mode, I obtain the following error log:

root@6.0.10.0-0009-build-linux-sdk:/drive# ./flash.py /dev/ttyACM1 p3710
2024-10-24 12:02:07 INFO : Initiating pre-flash check
2024-10-24 12:02:07 INFO : Checking Serial Port Connection
2024-10-24 12:02:07 INFO : Serial Port found /dev/ttyACM1
2024-10-24 12:02:07 INFO : Running basic open/close tests on Serial Port Connection
2024-10-24 12:02:07 INFO : Serial Port is open
2024-10-24 12:02:07 INFO : Flushing input and output buffer, discard all content
2024-10-24 12:02:07 INFO : Serial Port is closed
2024-10-24 12:02:07 INFO : Checking available targets
2024-10-24 12:02:07 INFO : Checking Orin board connection
2024-10-24 12:02:07 INFO : Orin board with Device ID -- 0955:7045 found.
2024-10-24 12:02:07 INFO : Orin board device path is /dev/bus/usb/001/002
2024-10-24 12:02:07 INFO : Pre-flash check complete
2024-10-24 12:02:07 INFO : Flashing DRIVE OS Docker
2024-10-24 12:02:35 INFO : driveinstaller v2.0
2024-10-24 12:02:35 INFO : 
2024-10-24 12:02:35 INFO : SDK/PDK Installer and Flasher
2024-10-24 12:02:35 INFO : Run `./driveinstaller --help` for usage information.
2024-10-24 12:02:35 INFO : 
2024-10-24 12:02:35 INFO : Initializing board and OS configuration to Autonomous Vehicle p3710 Linux Tegra A
2024-10-24 12:02:35 INFO : Manifest version 2.0
2024-10-24 12:02:35 INFO : 
2024-10-24 12:02:35 INFO : Invalid response for Aurix command!
2024-10-24 12:02:35 INFO : 
2024-10-24 12:02:35 INFO : Attempt 1 to re-send version command
2024-10-24 12:02:35 INFO : 
2024-10-24 12:02:35 INFO : Invalid response for Aurix command!
2024-10-24 12:02:35 INFO : 
2024-10-24 12:02:35 INFO : Attempt 2 to re-send version command
2024-10-24 12:02:35 INFO : 
2024-10-24 12:02:35 INFO : Invalid response for Aurix command!
2024-10-24 12:02:35 INFO : 
2024-10-24 12:02:35 INFO : Attempt 3 to re-send version command
2024-10-24 12:02:35 INFO : 
2024-10-24 12:02:35 INFO : Invalid response for Aurix command!
2024-10-24 12:02:35 INFO : 
2024-10-24 12:02:35 INFO : Could not fetch board revision. Re-try either by re-connecting board or make sure all serial port processes such as minicom are closed.
2024-10-24 12:02:35 INFO : Failed to bind partitions!
2024-10-24 12:02:35 INFO : driveinstaller cannot continue! Check the log file for details: /drive/driveinstaller/driveinstaller.log
2024-10-24 12:02:35 ERROR : Flashing failed, please contact provider with logs /drive_flashing/log_6gp8odmuwhay29l01cebrnxz3vtfk4qs.txt and /drive/driveinstaller/driveinstaller.log.

Attached, you can find the other error logs.

driveinstaller.log (13.4 KB)
log_6gp8odmuwhay29l01cebrnxz3vtfk4qs.txt (1.9 KB)

9

Dear SivaRamaKrishnaNV,

there is still a broken DRIVE Orin on my desk. Any updates on this issue?

10

Dear @kristian.fischer ,
Firstly, Do you have access to Aurix via minicom and able to issue any command? If so, can you share the complete log on aurix after serial connection and hard reboot?
Also, Is tegra accessible via serial console? If so, please share the boot log.
Just double checking, Please confirm if host is connected to target’s debug USB port and left Type -C.

This confirm board is able to entering recovery mode. If you can enter commands on aurix console, please put the board in normal mode(tegrarecovery x1 off & tegrareset x1) before attempt to flash.

This error indicates either connections are not correct or other serial connections are open to target. Please hard reboot both host and target.

11

Dear SivaRamaKrishnaNV,

this is my procedure to conduct tests that you proposed:

  1. Starting the host PC

  2. Connecting the left USB-C Port of DRIVE Orin with USB-A Port of host PC

  3. Connecting DEBUG Port of DRIVE Orin with USB-A Port of host PC

  4. Starting minicom to Aurix with sudo minicom -D /dev/ttyACM1

  5. Powering on DRIVE Orin
    ==> infamous “Error: VRS10 or VRS11 interrupt is detected, power sequence stops” error occurs in minicom; see
    aurix_log_after_reboot.txt (5.7 KB)

  6. Closing minicom with Ctrl + A, then Q

  7. Starting minicom to Tegra with sudo minicom -D /dev/ttyACM0
    ==> no reaction

  8. Hard reboot of DRIVE Orin with power switch

  9. Still no reaction from Tegra minicom

  10. Closing minicom

  11. Powering off DRIVE Orin

  12. Starting minicom to Aurix

  13. Powering on DRIVE Orin

  14. Prompting tegrarecovery x1 off + Enter
    ==> “Command Executed” message

  15. Prompting tegrareset x1 + Enter
    ==> “Command Executed” message

  16. Closing minicom

  17. Starting docker image with sudo docker run -it --privileged --net=host -v /dev/bus/usb:/dev/bus/usb -v /drive_flashing:/drive_flashing nvcr.io/drive/driveos-sdk/drive-agx-orin-linux-aarch64-sdk-build-x86:latest

  18. Flashing with ./flash.py /dev/ttyACM1 p3710
    ==> Above mentioned error with “could not fetch board revision” occurs

12

Dear @kristian.fischer ,
What was the DRIVE OS version on board earlier? Was the board working earlier? Did you run any application before you hit this issue. Can you share some details. This looks like a HW issue. Can you check private message.

15

Dear @kristian.fischer,
As aurix console is accessible, could you confirm if you can boot into other bootchain using aurix console
Please try getdfltbtchain to know default bootchain and use setdfltbtchain and aurixreset to boot into other bootchain. Please see Using the MCU Console | NVIDIA Docs for more details.

16

Issue got fixed after flashing the aurix.

17

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.