I’d like some advice on the best way to clone a Jetson TK1 for deployment to many Jetson TK1 modules in a production environment. I can’t quite figure out a nice way to achieve what I want.
So, we have a TK1 running Ubuntu with our own custom Linux kernel, device-tree and many many packages installed via a setup script. These additional packages range from Qt5, ROS and CUDA through to gstreamer plugins. In total, about 1GB of downloads to be pulled from the internet and takes approximately 40 minutes to install.
Moving our product into production, what I want to avoid is having to run this script on every single TK1 so ideally, I’m looking for a way of “cloning” our development TK1 with all relevant packages, kernel and device-tree modifications on then deploying this onto subsequent Jetson TK1 modules.
I’m not really sure about the implications of cloning directories such as /sys and /proc to other modules as my Linux experience still leaves a lot to be desired. Any suggestions would be really welcome.
You can clone and then use the clone image to flash. Probably the down side is that (1) it looks like you can only flash to one TK1 at a time (it looks like R28.1 for TX1/TX2 can clone multiple devices), and (2) if you clone just the rootfs partition you may find it incompatible with the hidden partitions if there is a base version difference between what is being written to by the clone and what was originally flashed. Restated, not all rootfs partitions are compatible with hidden partitions from a different release.
Some TK1 clone information is here…you can clone individual partitions or “all”:
[url]http://elinux.org/Jetson/Cloning[/url]
Something useful to emphasize is that a flash procedure normally produces “system.img” as a raw file system image, moves this to name “system.img.raw”, and then creates a sparse (compressed) image again by the name “system.img”. The flash.sh script will work with “system.img” being compressed or raw. A raw image takes longer due to size, but can be loopback mounted; a sparse/compressed image is smaller and faster, but cannot be loopback mounted (there may be exceptions to this for sparse image support, but I wouldn’t depend on it).
A normal generated image is a combination of the rootfs and some boot files which are generated at each flash. A clone is an exact copy of the resulting image complete with boot edits. If you “reuse” a system.img then no edits occur to the rootfs boot files. A raw image can be loopback mounted and examined or updated (e.g., a reference TK1 can use rsync to update a loopback mounted image on another machine just like a normal backup).
We routinely clone K1 boards using a method close to what’s described by linuxdev.
The contents of /sys and /proc are dynamically created by the kernel, and not written to disk. You won’t be cloning it because the cloning process reads the flash while the board is in recovery mode and Linux is not running.
This just reminded me of something…if you do want to rsync to a PC loopback mounted clone image you will find rsync and other tools (such as “find”) have options to not cross file systems. This allows just updating actual eMMC changes without digging into device special files, SD card mounts, “/proc”, “/sys”, so on.
Think of a clone as a full backup. You can do fast/easy automated rsync incremental backups to a spare copy of a clone as well. Any flash with the “-r” reuse option would include the rsync updates.
Thanks for your comments.
I think based on what you’ve commented, the rsync approach seems more attractive do to its incremental backup type nature. But then again, creating a raw (compressed) system.img seems to also be a good option.
Sorry for my next question, it may well be trivial.
Looking at the printenv out put from u-boot, thinks like serial number and mac address are present. If I did a direct clone of another TK1 via the system.img route, would the serial number and mac (etc) be duplicated or is this information taken from elsewhere (hardware)?
Thanks again.
I’m not sure about that…I think some part of U-Boot is capable of overriding MAC address setup, and some values would indeed be cloned. Someone from NVIDIA will have to answer which serial numbers, MAC address info, or other relevant info are part of hardware, and which would be cloned (this is for cases of cloning everything, and not just the rootfs…cloning rootfs would not clone boot environment).
If you clone only the root file system then each flashed unit gets its own boot loader flash, so I would predict there is no issue if the clone is purely rootfs. There might be an issue if you clone all parititons. Wish I knew more about MAC address and serial number generation.
Not sure it is still relevant, but before making the rootfs clone image on R19.2, I had to delete file
/etc/udev/rules.d/70-persistent-net.rules
that records the MAC address, otherwise when booting the cloned image on a different board, eth0 was associated to old MAC and new MAC got eth1, resulting in NetworkManager being confused.
I think this is more related to Ubuntu14.04 than to L4T (TX1/TX2 with Ubuntu16.04 do not need this).
So, if you have this file, delete it before shutting down and making the image. It will be automatically created at boot if it doesn’t exist.
Hi,
What do you want?? Need to know where r8169 driver loads the serial numbers and mac addr?