Reliable Way to Transfer Linux for Tegra Flashing Setup Between Ubuntu PCs

Hello,

We successfully implemented a complete working JetPack 6 image and set up a flashing environment on one Ubuntu PC using the Linux for Tegra (L4T) tools. The setup works perfectly, and we can flash devices without issues. The setup also includes a custom rootfs that we integrated as part of the flashing process.

Now, we want to replicate this flashing setup on other Ubuntu PCs. However, the size of the current setup is around 150GB, which makes transferring the folder between PCs quite challenging.

Is there a reliable and efficient approach to achieve this? Ideally, I’d like to copy the folder to the other Ubuntu PCs and be able to flash devices by running the same flash commands, without needing to re-download or re-setup everything from scratch.

Any guidance or best practices for transferring and reusing the Linux for Tegra setup across multiple PCs would be greatly appreciated!

Thank you!

If this has a single parent directory containing everything, then you might try to use scp -r (recursive) of that directory. Add the “-C” option for compression (scp -r -C). Use sudo as well since some of this is owned by root. Lastly, use the -p option to preserve permissions and times. For example, the Linux_for_Tegra/ folder has a parent folder somewhere under “~/nvidia/nvidia_sdk/JetPack...version.../”; you could cd there, and then:
sudo scp -r -p -C ./Linux_for_Tegra someone@the_other_computer

Barring that, if working with individual files, I’d go to rsync. Just ask if you want information for that.

Also, note that if you have an external USB hard drive formatted as ext4, then you could recursively copy to the drive, move the drive to the other computer, and copy there. This is a lot of data, so no matter how you do this, it is going to take a lot of time. There are all kinds of variations on how to copy this, but all of them will be slow.

hello @linuxdev

Thank you for the valuable information! I appreciate the detailed explanation. However, what I’m looking to do is create a production image that is immutable since the SDK is finalized, and no more modules will be added or removed. I want an image that can simply be flashed, and the person flashing it shouldn’t be able to modify or access the root filesystem.

In the past, on the TX2, we created such an image, but it was a long time ago, and no one remembers the exact process. I’ve also looked into the backup and restore methods, but those still rely on the root filesystem and the flashing command checking the rootfs folder.

I’m relatively new to this, so I apologize if I’ve misunderstood or said anything incorrect.

There are some details you will need to be aware of. To start, any model with eMMC memory will differ from a model without eMMC. You will probably find that the small form factors, such as the Nano series (Orin isn’t the only Nano), have a “developer’s kit” version. That version has only an SD card and no eMMC. The SD card itself mounts to the module, and not to the carrier board for dev kits. Any model which has eMMC, such as commercial modules (or larger form factor dev kits with eMMC), has a lot of content related to boot within partitions. Any model without eMMC has that same content in QSPI memory on the module itself. Just to emphasize, only one partition contains the operating system, but depending on model, other memory outside of the rootfs partition (which is the operating system), must be compatible with what the rootfs partition has on it.

The L4T release version can be found with “head -n 1 /etc/nv_tegra_release”. L4T is just what you call Ubuntu after it has the NVIDIA content added to it. The major release of the rootfs L4T content must be the same as the major release on that other content which I mentioned above. An Orin is compatible with L4T R35.x or R36.x. See:
https://developer.nvidia.com/linux-tegra

When you clone the rootfs partition, then you’ve cloned the entire operating system. This is normally what one would call a backup image. However, if you flash this backup image from one Jetson Orin Nano to another, the boot will succeed only if that other content matches. One can usually clone on command line with the flash.sh tool to get the clone of the rootfs. Unless the non-rootfs content is customized, then flashing the clone while allowing non-rootfs content to flash with default for that L4T release will work. That non-rootfs content is fairly standard and often does not need to be cloned in many cases. An example from the TX2 you mentioned:
sudo ./flash.sh -r -k APP -G my_backup.img jetson-tx2 mmcblk0p1

The above example, when used on a TX2 with R32.x L4T, would produce two files:

  • my_backup.img (a “sparse” clone)
  • my_backup.img.raw (a “raw” clone)

The sparse clone is good for flashing, but tends to not be too useful for other purposes. The raw clone is a bit-for-bit exact clone of the rootfs and is the exact size of that entire partition. The sparse clone does not have the “empty” space on the partition, and so it is smaller, reaching the size of actual content. As the filesystem fills and approaches 100% full, the sparse clone approaches the size of the raw clone. Cloning can require (just for the image) twice the size of empty space on the host machine as the size of the partition being cloned. This is also quite a long process. The purpose of the sparse clone is for faster flash; the raw clone can be loopback mounted (read-only if desired), read, manipulated, and saved (if you want to rsync to the raw clone it is easy).

Note that JetPack/SDK Manager is just a front end to the actual flash software. flash.sh is part of the “driver package” which is the actual flash software. This is tailored to the TX2 on R32.x, but is representative of restore (locations are relative to the “Linux_for_Tegra/” directory on the host PC):

  1. Place the raw or sparse cloned image in “Linux_for_Tegra/bootloader/”.
  2. Rename the image “system.img”.
  3. Flash on command line, use the option to “reuse image”:
    sudo ./flash.sh -r jetson-tx2 mmcblk0p1

What this would do on a TX2 (which has eMMC memory) is to flash the entire Jetson, but instead of generating a default image, the rootfs content would be 100% from the clone. All of the non-rootfs content would be flashed as well, and so if you used the same L4T release which produced the clone, then that other content would be compatible with your clone.

Sometimes size has to be specified if your clone is not from a default size. The “-S <size>” argument is for that purpose. Try “./flash.sh --help” from “Linux_for_Tegra/”.

There is a more complete way of doing this. Within “Linux_for_Tegra/” there is a “tools/backup_restore/” directory. The README file there explains this. The purpose of this more complete system is to also backup and restore non-rootfs content in addition to rootfs. This method works with many custom situations, e.g., when you use an NVMe for rootfs.


All of the above require the Jetson to be in recovery mode. You could use dd, but dd has some weaknesses:

  • Normally the rootfs is mounted read-write, and is a moving target which can change while running dd. Things like temp files are difficult to stop.
  • A user has to be logged in. This means you would clone in a state of a logged in user.
  • Remounting the rootfs read-only is helpful, but it will still contain content from a booted system with a login. Still, this is better than read-write since it is no longer a moving target.

The clone methods also give a perfect binary image copy of the rootfs partition. Clone methods do so for a filesystem in the shut down state, not with intermediate and temporary files related to a booted system. This does not suffer from the dd methods.

However, for illustration, let’s pretend you have a running Jetson. You want to clone only the rootfs partition. As a result, you switch the filesystem to read-only. Now you run dd on the device (if it is eMMC, then /dev/mmcblk0p1; if it is an SD card, probably /dev/mmcblk1p1). As your destination you might pipe dd through ssh and clone to your host PC. This gives a bit-for-bit exact copy of the rootfs. This is the same as the raw image from a clone, although the clone is a superior quality for being in an “offline” state during clone. This clone could also be renamed to system.img, placed in “Linux_for_Tegra/bootloader/”, and cloned “reusing” the rootfs image with the “-r” option. You might adjust the “-S <size>” if the image is not a default size.

If you want to clone everything, including non-rootfs content (perhaps including QSPI or other partitions), then you should use the tools/backup_restore/ method. This method also creates a bit-for-bit exact image of the rootfs, but it also combines it with non-rootfs content.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.