I have developed some software running on my Orin dev kit and I now want to be able to “freeze” the environment (with all systemd, crontabs, files, users, ssh keys etc) on the current devkit, to be able to deploy to new jetson orin series devices such as the NX8 and NX16.
Would there be any problem if I just ran on the devkit: sudo dd if=/dev/mmcblk0p1 of=/path/to/destination/image.img bs=4M
And then on each new jetson devices (after booting up ubuntu): sudo dd if=/path/to/your/image.img of=/dev/mmcblk0p1 bs=4M
Or is there any better way to do this? I don’t want to have to configure/setup every single new Orin device from scratch. The best thing would be if I could overwrite the default “recovery mode” to boot straight from that image, not sure if this is possible?
You can clone mmcblk0p1 on a recovery mode Jetson. Basically shut the Jetson down, put it in recovery mode, and use command line on the Ubuntu host PC (probably Ubuntu 20) to get a “raw” and “sparse” image. Then, when you flash, use the “-r” option during flash to “reuse” that image (you place the image as “Linux_for_Tegra/bootloader/system.img” and it will use that instead of generating a new default image).
Incidentally, a raw image is a bit-for-bit exact copy of the partition, just like dd, but the system is shut down and read-only. One can use Magic SysRQ keys to cause the Jetson to become read-only before dd, but temporary files from a running system would also be copied, so clone is somewhat superior. I don’t think you can put the rootfs into a read-only mode without putting other storage into read-only, e.g., if you have an SD card or thumb drive you are copying the clone to (but dd can be used over an ssh tunnel if the tunnel is already established before turning read-only; this too would be copied by dd). If you really must clone a running system, then the way to do it is to use rsync since it understands a read-write filesystem and different file types (you’d basically create an ext4 formatted loopback image on the other computer, and remote rsync to that loopback device).
A “sparse” image is the same as a raw image except that it does not copy blank space. As a filesystem fills the size of the sparse image approaches the size of the raw image (which is the exact byte size of the partition…better have a lot of spare disk space on the host PC). Since cloning produces both it is possible that there will be two files approaching a total of double the size of the partition. Moving (if crossing storage devices) or copying these files can take a very long time. Sparse images cannot be loopback mounted (I tend to delete the sparse image and use only the raw image), but they can be flashed, and due to (normally) smaller size they flash faster (one can also create a sparse image from a raw image with “bootloader/mksparse” and a NULL fill pattern).
There are also some scripts (which use recovery mode cloning) you might examine in “Linux_for_Tegra/tools/backup_restore/”. Check the README_backup_restore.txt".
Do note that when creating a system which duplicates the original that you still need to flash the “other” content which is not the rootfs. This content is essentially what a BIOS would have (there is no BIOS on a Jetson, it uses software in partitions), plus boot content. That content must be of a compatible release for a given rootfs. When you flash with “restore” via something like the flash.sh “-r” option it reuses that partition, and also flashes the non-rootfs content. If you do this using the same release that flashed the system you cloned, then it should work. If you have a different release on the recipient Jetson, then it might or might not work.
Ok so if I understand you correctly: Put my AGX Orin devkit into recovery mode, connect a ubuntu pc via the usb dongle and then clone the AGX Orin image. Once I have it I boot up a new device that I want to flash (like the Orin NX16), replace the system.img on it with the one I cloned and then flash it with the -r option?
More or less yes. Both have to be the same model, you can’t directly swap an AGX and NX model rootfs. You can probably come close if they use the same release. There would be some trial and error involved. But this is true regardless of whether you use rsync, dd, or cloning. Recovery mode cloning is the best way to get the image clone. rsync is next best if the system is read-write. dd has some advantages over rsync (not much) if the system is read-only.
Whenever you flash you will use recovery mode. This can be using a “Linux_for_Tegra/bootloader/system.img” which is from a clone, or if properly set up, rsync, or it can be directly from dd (dd and clone produce the same raw image, but clone works without booting).
Being able to use an image from an AGX on an NX is a different story. There is a minor variation you can use which is kind of fun to think about: You can loopback mount a raw clone over the “Linux_for_Tegra/rootfs/” directory and not tell the flash to reuse the system.img. The generated content will then be based on the clone, but it will be edited slightly depending on flash parameters. Your clone would be edited, so you don’t want to use the original. You would expect some of the “/boot” content and firmware to be edited prior to generating an image which is otherwise a match for the loopback mounted clone.