This is just some general talk about the topic, and is definitely not a comprehensive list, but I must warn you this is long (you asked for it :P, but a lot of people ask about some part of this anyway). YMMV. If you search for the “dd if=”, then you’ll get to the actual parts about using dd to clone an SD card. The dd command has a summary at the end if impatient.
At power up, every computer has a need to set up power rails, and to enable clocks in some specified sequence. On a PC much of this is done by the BIOS/UEFI/CMOS. For a PC the completion of the BIOS stage presents a uniform interface for operating systems expecting install or boot, and thus the CMOS and other pre-boot content does not need to be customized by every operating system for every model of motherboard. Embedded systems do not have a BIOS/UEFI/CMOS, and thus this is all done in software separate from the operating system.
If you wanted to perform a backup of a PC, then you wouldn’t bother with backing up the BIOS/UEFI/CMOS. When backing up a Jetson, you do not bother backing up the partitions outside of the rootfs, as these are somewhat equivalent to BIOS/UEFI/CMOS. Granted, some of those partitions contain a bootloader binary which is somewhat equivalent to GRUB, but unless something there is customized and in need of special attention, then any standard flash would restore these anyway. Most installs to a Jetson are just a single rootfs partition, with label “APP”. This is your real content and what you would back up (and there are additions if your hardware is custom, but you’d still always start with the rootfs).
If you flash and say to “reuse the rootfs”, then all of that side content gets flashed again, but this is standard (unless you have modified hardware). There is no reason to avoid flashing this other surrounding “non-rootfs” content.
You could clone this extra non-rootfs content, but this can have some problems you won’t expect. In earlier days this would not have been an issue, but once secure boot started development, the content of these other partitions had to be signed. What happens if you install signed content to hardware with a different key? It will fail to boot. Those partitions are the same, except their signing may differ. Unless the Jetson fuses have been burned to prevent reading the hardware, then the flash will “do the right thing” relative to what key is being used (once fuses have been burned you have to know what key to pass, or the hardware is useless).
If you do have custom hardware, and thus you have modifications to some other partition outside of the rootfs, then you’d place this on the host PC prior to flash, and let flash do the signing. A direct clone of one of these partitions would only work if you (A) removed the signing, and then (B) put the unsigned equivalent onto the host PC before flashing. Why bother doing this if the content (other than signature) is standard and unmodified?
One issue which people can sometimes fail to account for is that the non-rootfs content and the rootfs have some strong dependencies. This is why you cannot mix something like an R28.2 rootfs with an R32.4 flash tool. Cloning won’t care since it only reads content, but once flashed, the rootfs and non-rootfs have to be compatible. Still, a clone of R28.2 rootfs can usually be ported to R32.4, so the clone is useful.
There were some minor patch releases which did not generate a new JetPack/SDKM. Those are compatible, but are not nearly so common.
Clone the rootfs, and then flash everything is how I’d normally say to do it. For a Nano which is a dev kit SD card model, then I will add that you can clone the whole card if the content is going back on the same Nano. There will also be a big exception for a dev kit Nano or NX SD card model since they were designed from the ground up to run entirely from the SD card, and no eMMC is present for some of the secure boot. Notice how there is a generic image available for the SD card model of Nano and NX? There can’t be a signing requirement or this would not exist. I do not know all of the differences between SD card and eMMC models, but life should be simpler for this.
Does your PC have a lot of spare disk space? For example, if you have a 128GB SD card, then a clone will consume 128GB. If your host PC has an extra TB of disk, then that won’t matter (aside from massive files requiring a massive amount of time to manipulate). You can try the following with two SD cards which are the same size, and it should “just work” for a Nano (and if not you can post results/errors). I will assume that the PC names the SD card “/dev/mmcblk1”, but be absolutely certain you name the correct device to avoid cloning or overwriting the wrong partition/drive (note that the “block size” or “bs” used, is just a buffer and for increased performance…regardless of size chosen the result will be the same, but time required will change…dd works on damaged filesystems, and for that case the block size chosen would match the block size of the device, but you don’t need to worry about that):
- Create a clone on the PC:
sudo dd if=/dev/mmcblk1 of=sd_clone.bin bs=1M
- Remove that SD card, plug another card in, presumably with ID “
/dev/mmcblk1” since the old card is now removed.
- Send the clone to the new SD:
sudo dd if=sd_clone.bin of=/dev/mmcblk1 bs=1M
- Try it out for boot.
Alternately, you could just clone the rootfs, and then use flash software when performing an actual flash. The label for that partition is “APP”. While the SD is plugged in to the PC, this will list most everything about your SD card:
sudo gdisk -l /dev/mmcblk1
lsblk -f /dev/mmcblk1
In some cases you will car about the partition’s UUID, and this makes it useful to save a copy of the lsblk -f for later reference. In the gdisk -l the label “APP” will show up, and if this is referring to “/dev/mmblk1p1” (the first partition of “mmcblk1”, corresponding to the “Number” column of gdisk -l), then this means you’d dd copy just “mmcblk1p1”, and not the whole “mmcblk1”. Without the “p1” this is the entire SD card. With the “p1” it is a clone of a single partition. Example:
sudo dd if=/dev/mmcblk1p1 of=app_partition_clone.bin bs=1M
If you’ve ever flashed (versus using a preexisting image without any true flash) you will have this directory on your host PC:
~/nvidia/nvidia_sdk/JetPack...verison.../Linux_for_Tegra/
Normally this would be used to flash a connected SD card model Nano on command line (I think there are a few different targets which can be used, but I have not experimented so I’ll stick to jetson-nano-qspi-sd):
sudo ./flash.sh jetson-nano-qspi-sd mmcblk0p1
Normally the “Linux_for_Tegra/rootfs/” contains a full Ubuntu operating system plus some NVIDIA drivers, and an image is created based on this prior to flash. The actual argument of “jetson-nano-qspi-sd” triggers some copies into the “rootfs/boot/” area, e.g., might edit the extlinux.conf file, but otherwise the image generated is an exact match to the “rootfs/”.
If you had modified the “rootfs/”, for example by adding content to “rootfs/usr/local/”, then the flash would also contain that content. If “rootfs/etc/” had a user’s name and pass added, and perhaps “rootfs/home/” had a user’s home directory, then this too would be 100% preserved every time an image is generated and flashed.
If you have a clone, then you don’t need to generate this image. The generated image normally goes to “Linux_for_Tegra/bootloader/system.img”. If you have an image there already (regardless of whether it is from a clone or from a previous regular flash), then the “-r” option to flash.sh will avoid overwriting what is there.
If you were to place a copy of your clone in “Linux_for_Tegra/bootloader/system.img”, and then flash like this, then your clone would be installed (a clone of the rootfs, not the entire SD card):
sudo ./flash.sh -r jetson-nano-qspi-sd mmcblk0p1
(the only difference is the addition of “-r”)
Note that there might still be one wrinkle in this: The commands have all assumed the image is some default size. You might need to specify the size of the APP/rootfs partition with “-S size”.
Regarding size, this is the size of the fully expanded raw APP/rootfs partition. In a normal flash, and in a normal flash.sh generated clone (this does not apply to using dd and only applies to cloning from flash.sh software), you will actually generate two files: One is a “raw” file, and is the exact byte size of the partition. The other is a “sparse” file, and so long as the filesystem is not filled, this will be a lot smaller than the “raw” image. A normal flash puts the raw image at “Linux_for_Tegra/bootloader/system.img.raw”, and the sparse image at “Linux_for_Tegra/bootloader/system.img”. Either results in the same end flash, but the sparse image flashes faster.
If you look at the actual byte size of the raw clone, then this will be divisible by 1024 either twice (MiB), or three times (GiB). For example, if dd of APP/rootfs is size “30064771072” bytes, then this can be divided evenly by 1024 three times and is “28”. This is “-S 28GiB”. This would be the flash command to use this image when also specifying size:
sudo ./flash.sh -r -S 28Gib jetson-nano-qspi-sd mmcblk0p1
The dd image and the raw image can be loopback covered and lots of nice tricks used, e.g., you can mount the loopback device as if it is a real hard drive partition, and then use QEMU to run various dpkg operations on it. Even without QEMU you can copy files to/from the loopback mounted raw clone (dd copies are raw). You cannot do this with a sparse image…a sparse image is only good for flashing.
Summary to clone APP/rootfs with dd from a host PC when the SD card is “/dev/mmcblk1" and the APP label is the first partition, "/dev/mmcblk1p1": sudo dd if=/dev/mmcblk1p1 of=clone_of_rootfs.bin bs=1M ("bs`” is optional)