Flash an Orin from a ARM CPU

Robotics & Edge Computing Jetson Systems Jetson AGX Orin
,
1

Hello,

As part of our project, our Orin devices must be able to be reflashed OTA by our embedded CPU (Texas Instruments CPU, ARM).

I was able to compile and prepare my image with the command: 'sudo -E ./tools/kernel_flash/l4t_initrd_flash.sh --no-flash --massflash 1 Orin-jetson-agx-orin-thelis internal"

I have my ‘mfi_Orin-jetson-agx-orin-thelis.tar.gz’ package, which I was able to copy and extract to my ARM CPU.

But as soon as I try to launch the flash with the command ‘sudo -E ./tools/kernel_flash/l4t_initrd_flash.sh --flash-only --massflash 1’, I get the following error:

 Entering RCM boot

[   0.1445 ] mb1_t234_prod_aligned_sigheader.bin.encrypt filename is from --mb1_bin
[   0.1446 ] psc_bl1_t234_prod_aligned_sigheader.bin.encrypt filename is from --psc_bl1_bin
[   0.1446 ] rcm boot with presigned binaries
Error: Could not find tegrarcm_v2
Cleaning up...

From what I understand, the tegrarcm_v2 binary is intended for an x86-x64 CPU, not ARM…

How can I flash my Orin from my ARM CPU?
Is there a specific option to pass to the flash tool?

Thanks in advance

3

Recovery mode for a Jetson turns the Jetson into a custom USB device. This is quite different than simply adding content to a USB mass storage device. The only way to use this custom USB device is with the driver, which happens to be the x86-64 flash software. Thus, there is no direct possibility of this. The only option is to use a VM or container to make the ARM flash device appear as x86-64.

4

Thank you very much for your reply. I understand completely. But can’t NVidia provide an ARM driver?
This issue is a big red flag for the rest of our project. We didn’t check at the outset because it seemed so obvious to us that USB flashing would work just as well on x86 as on ARM.
Our CPU, which manages the Orins, is fairly lightweight. It has an internal USB NVMe drive, so we have storage space to manage the Orin images, but in terms of performance, a VM seems to me to be completely beyond the capabilities of this small CPU.

I hope that Nvidia can provide a solution quickly.

5

There is no ARM version flash tool, and not plan yet.

6

Is there a way to obtain the source code for tegrarcm_v2 so that I can compile it myself for ARM?

With this answer, I can simply throw my 500 k€ project in the trash.

Or is there another way to flash an Orin AGX without using tegrarcm_v2 that would be ARM-compatible?

It’s quite surprising that an ARM CPU (the Orin) cannot be flashed by another ARM CPU…

7

Direct flash has no substitute. However, if you arrange an update as a live Over The Air (OTA) update, then the Jetson can update itself. OTA updates have dangers, and you’d have to test, but an x86 Linux host is otherwise mandatory. An OTA update has only a networking requirement (and if you had to you could use the host ARM system as a web server and/or NFS server).

This is likely to be unacceptable, but if you have a small x86 Linux host PC appliance, then that could be used with flash. Some appliances don’t work for this, so you’d have to test, and I have no doubt that packaging that with your Jetson would be a major problem, but the possibility is there.

The main things to consider:

  • The flash program must be on an x86 Linux system, preferably the correct Ubuntu (command line works on some other distros, but not JetPack/SDK Manager).
  • If you are building from the “Linux_for_Tegra/rootfs/”, then that must be an ext4 filesystem.
  • True flash means you have to control the recovery mode pin while rebooting or powering on (there are ways to go into recovery mode during a reboot via a command, but once such a thing is started, then there is no going back should there be an error during the process).
  • OTA does not have such requirements, but has its own risks. You would need extra caution surrounding any customizations, e.g., device tree, kernel, partition backup or encryption, so on.
  • Right now QEMU is used on the x86 host PC to modify the “Linux_for_Tegra/rootfs/” (such as changing a password), and if you got it working correctly, then maybe the reverse could be performed. This would not be simple, but maybe (with a lot of work) you could use QEMU to emulate x86 on the ARM system to run flash; then the modification by QEMU of anything in “Linux_for_Tegra/rootfs/” could be modified to be native, or oddly enough, use QEMU from the x86 QEMU to emulate ARMv8 from the “fake” Linux x86 host PC.

If you look at the script l4t_create_default_user.sh, then you get a human-readable example of how QEMU is used to emulate ARM. If you are considering feasibility, then try to do the reverse to set up very simple use of Linux commands which are binary x86 on the Jetson. One tool to consider while testing or imagining if this will work (a pretty big "if"™) is “ldd”. On your host PC, if you find a binary executable, then you could directly copy it to your control Jetson running QEMU and test with that command if the file itself has no linkage (it is statically linked). If not static, and if linked to other libraries, then your QEMU will need those libraries (and linker) installed to test (in the end you’d need to do this anyway, but QEMU has some linker ability I think…not positive, I’ve never done this).

Most of the Linux host PC binaries I’ve looked at link to a very minimum of other files, but if you were to build your own “hello, world” application which is statically linked (very simple to do), then you could try to run that x86 app on QEMU. Once it runs, you could then try a binary copy of simple commands which are binary executable, but have minimal linking. Examples to copy are “whoami” and “ls”. On one Ubuntu host PC I see this linkage when I run ldd on them:

#  ldd /usr/bin/pwd
        linux-vdso.so.1 (0x00007ffd441a5000)
        libc.so.6 => /usr/lib/x86_64-linux-gnu/libc.so.6 (0x00007f88b441c000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f88b467b000)
#  ldd /usr/bin/whoami
        linux-vdso.so.1 (0x00007ffc3a0f1000)
        libc.so.6 => /usr/lib/x86_64-linux-gnu/libc.so.6 (0x00007f3fb9932000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f3fb9b90000)

If you do manage to set up QEMU, then there is very little you couldn’t do on command line. GUI apps might be a different story. In the downloads for a release the “driver package” is the actual flash software, and this contains the actual binary tegrarcm_v2 and everything needed for flash. SDK manager is the “smart network” layer which you don’t need, and this in turn runs the JetPack application; this is just a GUI on top of the driver package and is also not needed. JetPack does offer download and post flash optional package install, but all of that can be skipped if you already know what packages you want: Just use the apt mechanism when the proper repository is listed in “/etc/apt/”.

8

Hello,

Thanks a lot for this very complete answer.

The entire build and “flash” (using the --no-flash command) is performed on a build server, in a Docker container based on Ubuntu 22.

I end up with a tar.gz file that contains everything I need to flash my Jetson (–flash-only).
My CPU (ARM) manages the Jetson startup and controls the force-recovery pin. I am therefore fully capable of restarting my Orin in recovery mode via my CPU (and it works very well).
My only obstacle is the fact that tegrarcm_v2 needs to be run on an x86 CPU.
I will try using QEMU.
All the applications and packages I need for my Orin are already present before the build process, because I use chroot to enter the Orin rootfs and do everything I need for my final turnkey image. So once the flash is made, I have nothing to do manually into the orin system.

Thanks

9

Keep in mind while working on this that if a flash starts out correct, but fails part way through, that often containers or VMs fail to always pass through USB. During a flash USB will disconnect and reconnect, so be ready to examine that if a flash starts out ok but then breaks.