How to automate/speed up jetson flashing process?

We are using Jetson Xavier NX on a commercial product.

This product requires a modified pinmux configuration and all the software runs on docker with the nvidia-runtime.

Currently our workflow for each jetson is:

  1. Boot in recovery mode and connect via USB.
  2. Flash JetPack 5.1 with the SDK Manager, choose NVM installation, install Image and SDK Components (because we need nvidia docker)
  3. Wait ~10 minutes, flashing is done, connect a screen and keyboard and press “OK” on the language selection, then wait a few minutes and go back to the SDK manager to input the IP and click on next for the SDK installation.
  4. Wait another 20-30 minutes for flashing.
  5. Flash again from command line with sudo ./flash.sh jetson-xavier-nx-devkit-emmc mmcblk0p1 to install custom pinmux
  6. Use ansible to configure the rest of the environment, pull image dockers etc…

The whole process takes over an hour, with multiple manual inputs and what seems like redundant steps due to our lack of understanding of what is going on under the hood.

Ideally, I’d like to flash once with the correct pin configuration, then install the rest of the software via ansible.
Also, can I skip the SDK components and additional libraries if I am running everything inside a docker using the matching l4t-base image?

A default flash (which is what you described) generates a new rootfs image each time it runs. If you are using the same image each time, then you can stop generating the new image and just flash.

During flash a raw image is created at “Linux_for_Tegra/bootloader/system.img.raw”. From this a smaller image is created, “Linux_for_Tegra/bootloader/system.img” (it is a “sparse” image). The raw image is mostly generated from the “Linux_for_Tegra/rootfs/” content, although some adjustments are made for the target and device chosen (such as the extlinux.conf, and if you changed from flashing the particular target to a different target, then it would also change PINMUX/device tree and kernel). If you are modifying rootfs/ each flash, then you need to generate a new image. If you are modifying kernel and device tree between flashes, then you also need a new image each time.

The "-r" option says to reuse the "system.img" file. No system.img.rawwould be created, and no sparse image generated...it just uses what is there from the previous flash (incidentally, if you had a clone, you could put this in place assystem.imgand you'd be flashing the exact clone). Example:sudo ./flash.sh -r jetson-xavier-nx-devkit-emmc mmcblk0p1`

The above will flash everything, but it won’t generate a new system.img.

I want to point out that if you are at “Linux_for_Tegra/”, and you use this command:
ls -l jetson*.conf
…then what you will find are all flash targets. the “jetson-*.conf” targets are just the human readable names. The alias which points at more obscure file names are just the combination of module specification and carrier board specification. These, in turn, are all human-readable script. If you were to copy one of these, and rename it slightly, then that file (without the .conf suffix) becomes a flash target.

If you were to copy jetson-xavier-nx-devkit-emmc.conf to a new name, e.g., jetson-xavier-nx-devkit-emmc-custom.conf, you could edit that file. If you follow what files are included in that file, and then copy those files to a “-custom.conf” name, then you could also edit those files (the files become more specific to module, or SoC, or carrier board). If you find the location which names which device tree is copied into the “rootfs/” (or kernel Image), then the -custom.conf file could be edited to point directly at this. From then on you wouldn’t need to flash separately for PINMUX. Or kernel Image.

In fact, the device tree (PINMUX) rarely has to be flashed at all. It depends on some details. Some content is available both in partitions for boot and in the rootfs. The partition content (only on eMMC models, but yours is that) must be signed before it is accepted, and this is performed automatically during flash. If you have not burned security fuses, then the default NULL key is used. During boot signed partitions for kernel and device tree can be used, and those partitions are updated only via flash most of the time. However, content in “/boot” takes priority over partitions if the fuses are not burned.

Examine your “/boot/extlinux/extlinux.conf”. If there is an FDT key/value pair entry, then it names the location of the .dtb file to be used (overridden only if security fuses are burned or the file is not found). The same is true of the kernel Image file: If it is listed in the extlinux.conf, then that takes priority over any partition content (unless security fuses are burned).

If you were to log flash on command line (I very highly recommend you do this once to keep a record) you would see exactly which kernel and device tree and extlinux.conf is copied into “rootfs/” before generating a new rootfs image. Example to log:
sudo ./flash.sh jetson-xavier-nx-devkit-emmc mmcblk0p1 2>&1 | tee log_flash.txt

Example to log which reuses the rootfs image (“bootloader/system.img”):
sudo ./flash.sh -r jetson-xavier-nx-devkit-emmc mmcblk0p1 2>&1 | tee log_flash.txt

If you were to clone one of your Jetsons after doing everything you want, e.g., updating packages, maybe adding custom programs or new libraries, you could then copy the raw or sparse clone to “bootloader/system.img” and this would be used again. You could skip package updates and any customization that was in the clone.

A raw clone is the exact bit-for-bit image of the rootfs partition. It is the exact size of the partition, so if the partition is 20 GB, then the clone will take up 20 GB on the host PC. That’s an enormous file, but it can be loopback mounted, examined, edited, used as a sysroot for cross compile, so on. A sparse image is generated from the raw image and excludes “empty” space. Thus, a sparse clone size approaches the size of a raw clone as the space is filled. If you were to clone a full filesystem, then the raw plus sparse size would be double the size of the partition. Flash does not care if you flash the raw clone or if you flash the sparse clone, the result is the same, but you can’t use a sparse clone the way you can a raw clone.

There is a tool available to convert a raw clone to a sparse clone, “Linux_for_Tegra/bootl,oader/mksparse”. Just use the NULL byte fill pattern (“0”).

Flashing a raw clone takes more time than flashing a small sparse clone. All of that content goes over the USB cable. This is why the sparse clone exists: A faster flash. On the other hand, you’re generating a new system.img.raw and a new system.img is being derived from that every time you flash with the command you provided. Then the sparse clone is being used to flash.

If you were to clone, then you would need to beware that this is a full exact clone. User accounts and their passwords would be the same on all of the flashes. If you have a udev rule for a custom USB device setup, e.g., one that depends on a MAC address of a specific network adapter, then that too would be cloned. If you ship to the state of California, then you are not allowed to use default admin passwords, which is why the first boot account setup method was added and Jetsons long ago stopped using default accounts and passwords.

If you are ok with using the file version of your device tree, just copy it into the location named by the FDT key/value pair of extlinux.conf and don’t bother flashing (the old tree would still be in the partition, but if you adjusted flash target specification, the new tree would already be there anyway). Burning of security fuses will limit options.

Thank you for this incredibly quick and lengthy response.

I will work on it.

Thank you.

I have done as instructed and now will just flash once reusing the previous rootfs.

I believe there’s a wrong path on the second paragraph Linux_for_Tegra/rootfs/system.img.raw, I belive the system.img files are on bootloader and not on rootfs path.
Just in case someone stumbles upon this in the future.

You are correct…I’ve edit the bad path to fix this. rootfs/ is a source of content creating images in bootloader/.

I have another related question.
Can I use the same dtb or dts file from one unit to another?

I flashed device A, used jetson-io.py to configure the spi pins and rebooted.
Then I copied the /boot/kernel_tegra194-p3668-0001-p3509-0000-user-custom.dtb from device A to device B, copied the /boot/extlinux/extlinux.conf from A to B and rebooted B.

Device B did not have any ethernet interfaces and other errors.
I applied the same changes using jetson-io.py to device B, rebooted and all was fine.

I decompiled both A and B dtb files which should be identical, and they are extremely different.

I am attaching them.

Am I missing something?
A.txt (391.6 KB)
B.txt (394.8 KB)

If two modules use the same carrier board design, then you should be able to use the same device tree. A device tree is more or less a set of arguments to pass to drivers as they load for non-plug-n-play devices. One of the biggest uses of a device tree is to tell the kernel the address to find hardware at, and which driver to use. If the kernel on one is the same as the kernel on the other (so far as which drivers are present), then it should be portable. Drivers which don’t load simply ignore device tree nodes and won’t cause an error. As soon as carrier boards differ, things change because then the tree must change.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.