Query on Backing Up and Restoring Jetson Devices Across Different Hosts

Dear NVIDIA Support Team,

We are exploring a method to back up and restore Jetson devices (Jetson Orin Nano/NX) from different host machines, without relying on BSP installation or setup on each host.

Could you please confirm:

  1. Whether it’s possible to perform backup and restore operations across two different Ubuntu host systems.
  2. How to achieve this in a BSP-independent way — ideally using raw image or device-level backup techniques.

We aim for a portable solution that doesn’t require re-installing or replicating the Jetson BSP on every host.

Looking forward to your guidance.

Best regards,
Sunil A P

@WayneWWW

A partial answer: In recovery mode a Jetson is a custom USB device, and does not have a BIOS. The interaction requires the flash software on the host. JetPack/SDK Manager is just a front end to the actual flash software, and the part which performs the flash is the “driver package” (appropriately named because it is the driver to the custom USB device). This is what creates almost all of the “Linux_for_Tegra/” subdirectory and content. The actual rootfs content requires the “sample root filesystem” package, along with the “apply_binaries.sh” command to prepare it. Technically, if you are only reusing an earlier flash image or rootfs clone image, then you don’t need the rootfs content.

The driver package has all of the firmware and non-rootfs content in it. If flashing something with eMMC, then this has the content of the other partitions; for an SD card model, this has the content which goes into the QSPI memory. A flash which reuses the system.img (such as from a previous clone) uses this other content as well (and if your partition sizes are not default, then it is mandatory that you specify a rootfs size to avoid those non-rootfs partitions overlapping).

One can manually install the driver package without JetPack/SDKM, and this will allow running the program from a wider range of hosts. You can then flash via a clone image if you don’t mind that non-rootfs content being installed from the default.

A true full backup and restore which also clones and restores the non-rootfs partitions or QSPI would require the backup and restore script. This too would require the driver package, and would use your backup image for both rootfs and non-rootfs content. Some environment variables within the boot stages might not be preserved, e.g., I think reordering boot options might need to be performed again if this was in the previous image (I have not actually done this so I am not certain).

You do need JetPack/SDKM if you are going to add or remove optional packages, e.g., CUDA.

Because the Jetson in recovery mode is a custom USB device there is no method of working with a recovery mode Jetson without at least the driver package. If you are backing up or restoring just rootfs content, then there are other “live” methods available which any Linux system can work with, but they only cover rootfs. For example, you can use rsync on a running system for a kind of backup or restore. There are advantages though to a clone image, e.g., rsync, by nature of working on a running system, would also capture things like lock files and temp files, but the clone image would capture a true image of the shut down state.

Hi,
We may not support this case. The supported case is as described in developer guide:

Flashing Support — NVIDIA Jetson Linux Developer Guide

Hi @DaneLLL

Thanks for the clarification.

We are looking for a way to share a bootable image with our customer so they can flash the Jetson device from a different host system.

Could you please advise if there is any supported or recommended method to package and share such an image for flashing on another machine?

Looking forward to your guidance.

Best regards,
Sunil

@DaneLLL

Hi @linuxdev

Thanks for the clarification.

We are looking for a way to share a bootable image with our customer so they can flash the Jetson device from a different host system.

Could you please advise if there is any supported or recommended method to package and share such an image for flashing on another machine?

Looking forward to your guidance.

Best regards,
Sunil

@linuxdev

Jetsons do not have a real BIOS. You can think of a BIOS as a micro operating system that brings up power rails and clocks in the correct order. Then it has minimal drivers for hardware on the motherboard, e.g., a driver for a USB keyboard and boot devices. On a Jetson this is entirely in software and there is no BIOS. There is no ability during boot to “self flash”, which is why a host PC is required which runs the custom USB driver and software which understands some of the layout of the Jetson.

As such, a full/normal flash of a Jetson is simultaneously flashing this or its equivalent:

  • BIOS
  • Bootloader
  • Operating System (rootfs)

Can you imagine how much more difficult it would be for something like a desktop PC to flash Windows or Linux on it if every installation was also a simultaneously installation of a BIOS? It changes a lot of details.

Regarding the Orin itself, it can be flashed with either L4T R35.x (Ubuntu 20.04) or L4T R36.x (Ubuntu 22.04). It isn’t until you get to R36.x where there is a full UEFI compliant boot. Before that many parts of boot were entirely custom and non-standardized (that’s the history of U-Boot and basically all embedded devices in the past and isn’t just a “Jetson” thing). What UEFI does is to provide an abstract object oriented interface. During the first part of boot there is nothing “standard”, but the software eventually reaches a point where the UEFI interface is truly interchangeable with something that might boot on a PC. Everything up to that point is custom software. However, once you reach this point, it is possible in R36.x to more or less drop in somewhat interchangeable boot code. I think you will see some better options maybe in a future release because of this, but it isn’t there yet where you can just create a bootable image. One still has to flash the content up to UEFI at the same time as the o/s.

A lot of "and"s: If two Jetsons are of the same model, and if both Jetsons have had all of that content which gets to the point of loading the rootfs installed, and if both Jetsons use the same major release of that content, and if the rootfs image is the same size of partition, then it becomes possible to just flash the rootfs image. The content which supports booting to that is already in place and is compatible which makes this possible. But there are a lot of things that can go wrong up to that point. Still, it is useful.

One thing which can go wrong flashing just the rootfs even if all of the boot content is valid and matches the rootfs is that some things like the UUID of a disk or partition are sometimes part of that boot content, and if your new rootfs is otherwise an exact match, but the UUID is not the same, it could still fail to boot. The point of mentioning this is that there are degrees of backup and restore in different ways, but most all of them require some knowledge and some time.

Going down a different route, if you were to flash an Orin with the same major release, but use the default o/s, then traditional rsync backup and restore works. The backup can be a cloned raw partition image which is loopback mounted on another computer on the same network, or it can be created by rsync itself. Scripts could be made relatively painless such that if someone has a Jetson flashed with the same major release it will update the Jetson to whatever image you happen to have, but it would be a file-based update. This has an advantage that so long as the rootfs partition is capable of holding that much content it won’t matter if the rootfs partition size has changed, it should “just work”.

If you are interested in some particular scheme of backup and restore, go ahead and add more details of how you want this to work with your customer and maybe we can come up with something. Just beware that none of this will be entirely simple, and that unless the device being updated/restored has the same kind of boot device (e.g., both using eMMC, or both booting to an NVMe, or both using QSPI memory), then even that gets more difficult. What boot device type is it? eMMC? NVMe? Hopefully eMMC because that is by far the easiest to deal with.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.