When you look at a hard drive directly as bits the method of using it is different than when looking at a file system and using the file system to access it. During normal run the rootfs is accessed by the methods of an ext4 file system, and this in turn talks to the driver which directly accesses as bits. If you were to read bits, then there is no such thing as having any meaning, e.g., permissions, users, files versus directories, so on. Copying a mounted file system’s content causes interpretation of what is there to become permissions, files, directories, so on…on the other hand, copy of raw bits means the partition and the file system itself are also copied.
One can manipulate partitions on a real disk, and as a result of a clone being a bit-for-bit exact copy (versus files and directories), one can use loopback to make those bits appear as a disk and allow the bits to be manipulated exactly the same as if it is a raw disk or partition (you can use gparted to resize a disk partition…and to resize a loopback covered clone). When a Jetson is flashed the driver package has no concept of ext4…it merely copies bits which by coincidence were previously formatted as an ext4 partition. If you copy files directly, you can’t flash it. If you copy bits, then the file system and every aspect is an exact mirror, and this can flash to become a complete restoration down to the last bit.
Because clone is a bit-for-bit exact copy, this means your entire file system can be mounted on your host the same as if it were an SD card or external disk drive…you just have to cover it first with loopback so the kernel thinks it is a disk instead of a file. A clone is better than a file copy since it has the file system itself copied into it. Loopback mounting this implies you can use rsync from your Jetson to update the loopback partition as if it were a real disk. Future flashes can use this. This is the usual reason for cloning. See:
You can also use “dd” to clone a partition. However, the partition should not be mounted (because you don’t want it changing in the middle of a copy) when using dd. A clone works with the Jetson in recovery mode, and thus the operating system is not running and the clone is guaranteed to be an exact copy of your system. You could use dd correctly if and only if you boot to alternate media and do not mount the partition, e.g., by having an SD card rescue boot set up.
In the above URL pay close attention to file names. If you clone “backup.img”, then you will have created both “backup.img” and “backup.img.raw”. Only the raw image is useful…the “backup.img” would be a “sparse” file which is essentially a cheap kind of compression…and it can be flashed…but it can never be updated or examined without restoring it onto a Jetson. I throw away the “backup.img” and keep only the raw “backup.img.raw” (as a file system gets filled with content the size of the sparse image will approach the size of the raw image…after a fresh flash a raw image is about 15GB, and a sparse image is about 2GB).
When the driver package running on the host PC flashes a Jetson (which can be done on command line or via JetPack running the driver package) it can use either the raw format or the sparse format…the driver package doesn’t care and both will correctly flash the Jetson (the Jetson knows how to unpack raw or sparse ext4 images). Originally there was only the raw version, and sparse had no support. Sparse was added to speed up flash time. Sparse is much smaller than raw if the raw is mostly empty…raw is the exact and full size of a partition. If your Jetson had been flashed with “-S 14GiB” size argument, then this makes the rootfs partition “1410241024*1024” bytes (15032385536 bytes). That’s a huge file. Transferring it over USB takes longer than something which is perhaps 20% of that size. Your host would have to be able to hold files in the 15GB range (the host needs a lot of spare disk space for clones) for the raw image, and perhaps 2 or 3GB for the raw image.
If you take a raw image, e.g., named “backup.img.raw”, then you can explore it or use it like a real partition:
mount -o loop /where/ever/it/is/backup.img.raw /usr/local/my_arm64
ln -s /usr/local/my_arm64/usr/lib/aarch64-linux-gnu .
# Now your PC has the entire arm64 library at "/usr/lib/aarch64-linux-gnu/".
# The content on the host is a 100% exact match of your complete Jetson "/usr/lib/aarch64-linux/gnu/".
# Now the aarch64 is empty.
# Have a different clone from a different Jetson with some difference? Mount it at
# the same location...your build environment is now an exact match to that other system.
# Have other people who need to share builds? Pass around the clones and mount this way...
# you'll now have an exact duplicate build environment.
# Not enough space? Put a clone on an SD card or external USB disk...or even an NFS server...
# your environment can now become any image you want without modifying the host (as you go
# through lots of embedded devices and lots of cross-tool versions this becomes more valuable).
There is a subtle detail to think about when comparing dd or clones to rsync or other file copy mechanisms. If you were to mount another file system on top of the original file system, e.g., some subdirectory, then file copy will only see what is mounted over the top…it can’t see what was originally underneath that new mount. As an example, when you first boot, there is content in “/dev” which is used for booting. Later on at some stage of boot, sysfs and udev mounts a pseudo file system on top of “/dev” and creates its own content. rsync and other backup methods are generally designed to not cross file systems, and thus would leave “/dev” empty. If you get around this by file copy, then you’ve just contaminated your backup image with files not on the original partition…those files are hidden by being mounted over. What you really want is the partition content, not the pseudo file content. Clone does this without any rescue disk. dd does this if you have a rescue disk. File copy can never see or copy the full partition when a pseudo file system covers up parts.
For the purpose of developing you could copy every file which you know is needed (e.g., all of the “/usr/lib/aarch64-linux-gnu/” and nothing else). When you do a system update of the Jetson, then you will need to repeat this if you want the build to be for that version. If you have two Jetsons with different versions, then you need to do this again each time you build for the different version. Every…single…time.
Or…you could mount a clone which is sym linked and be done. And have the added security that if you lose the content on a Jetson you can absolutely restore the rootfs.
Do beware that restoring and cloning may have differences depending on which release you use. rsync doesn’t care about that, so there rsync and file copy wins.
If you do clone, then I suggest you save a pristine copy. I use “bzip2 -9” on images I’m not currently using…this takes a lot of time. Any image mounted with “mount -o loop,ro …” is read-only and protected…a good way to mount to library directories you don’t expect to modify. Any “restore from clone” directions do eventually have you copy the image to “Linux_for_Tegra/bootloader/system.img”, and in that case I would never ever put my original there…it would get overwritten the first time you forget to use the “-r” option to flash.sh.
Note that you could use file copy to an SD card instead of cloning and it would probably be good for that purpose. You could then mount the SD card somewhere and sym link to the lib directory the same as if it were a clone. I prefer the clone because it doesn’t matter how I might mess up the copy…it’ll always be perfect, and there all kinds of things which can go wrong with recursive file copies.
Tip: If your host PC already has “/usr/lib/aarch64-linux-gnu/”, then it means you’ve added foreign architecture environment for cross compile. Like algebra, there are a lot of ways to avoid conflicts…some are more work than others and some have more problems than others. You could use symbolic links to individual files in the loopback mount location’s aarch64-linux-gnu instead of linking the entire directory. Or you could add “/usr/lib/alt/” to your cross-linker path, and then sym link aarch64-linux-gnu in there. You can always ask if you see alternatives and want help picking one.
Caution: You can never restore a Jetson with a clone from a different L4T version than what is on the rest of the Jetson. However, a clone makes it much safer and easier to restore content onto a newly upgraded flash and take your time with it.