Clone entire TX2

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

Is there one command to clone the entire storage of a TX2 to another and not just the app partition? I am using a Connecttech Rudi which has some mods to the image which are not coming over when I run:

sudo ./flash.sh -r -k APP -G system.img jetson-tx2 mmcblk0p1

I need to copy the entire thing, all 32 GB and clone it to another one as we have a few of these we need to build and going through and reinstalling all the packages would take forever. I am reading that I might be able to call mmcblk0 or use dd to copy it all.

2

There was a way on the TK1, but I do not believe the flash software has this ability on a TX1 or TX2. If you have a running system you can use dd. You can clone individual partitions, and other than the metadata hidden in about the first 4096 bytes, this should work…unfortunately, the hidden data at the start is necessary for correct boot, and there seems to be no way to do this other than via dd.

This is one of the features I’d consider very useful in a number of environments (developing or commercial production)…flash.sh being able to clone or write via byte or sector offsets instead of only via partition name/label.

3

In doing some more reading, there may be some partitions that I need to leave alone. The modifications to the BSP for the Rudi are described here: http://connecttech.com/resource-center/cti-l4t-nvidia-jetson-board-support-package-release-notes/ what I’m fighting is CTI says that you need to start fresh every time but we have made quite a few customizations and installed so much software that we need to clone it rather than build from scratch as we didn’t document as much as we should have and since there are mutiple units, it gets easier to clone…

4

Is this R28.1 or newer (there is a pre-release R28.2)? If so, then I suspect device tree modifications plus rootfs would probably suffice if they have the same starting base install version. This would imply cloning mmcblk0p15 (“device-tree”) and mmcblk0p1 (rootfs, or “APP”). It might just be a case of testing it out…the rest of the file system space, e.g., the first 4KB, should be the same even with most customizations.

5

Jetpack 3.1 so I am pretty sure 28.1

Could you share how to structure those commands?

My first guess would be: sudo ./flash.sh -r -k [Partition name]-G system.img jetson-tx2 [Partition Location]

Name being: device-tree or APP
Location being: mmcblk0p15 or mmcblk0p1

I worry about trying things that I don’t really understand as I have no good backup right now and don’t want to accidently erase or brick it.

6

Making a clone to backup a Jetson has no risk…recovery mode does not edit anything, though it does essentially make it possible (if you use the wrong command it could destroy the rootfs partition…no command at all or read-only commands won’t write to the Jetson). A typical command:

sudo ./flash.sh -r -k APP -G backup.img jetson-tx2 mmcblk0p1

The “-r” says to not generate a new system.img. If the command works with another partition, then it also keeps system.img from being flashed to the Jetson. system.img is the rootfs partition, a.k.a., mmcblk0p1…the GPT partition label for this is “APP” which is visible if you list via “sudo gdisk -l /dev/mmcblk0”. So the “-r” combined with -G implies this will only read the Jetson, and not write it. The “-k APP” and mmcblk0p1 both name the rootfs partition, but probably different parts of the software label it differently.

Note that if you do restore from a clone that the use of “-r” is what keeps your cloned image from being regenerated…if you put system.img in “bootloader/” subdirectory and did not use “-r”, then it would get overwritten. During a regular flash you get both system.img and system.img.raw in the “bootloader/” subdirectory…during a clone to read the Jetson you would get both backup.img and backup.img.raw. The “raw” images are the most useful…you can loopback mount them, read them, use them with rsync, edit them, and reflash if you rename it system.img. The non-raw “sparse” image is only useful for flashing…this cannot be converted back into a full image except by flashing (it appears the sparse image is incompatible with both regular Linux sparse file system programs and Android versions of sparse file system tools). Be careful about throwing away your old raw images…if you really must archive and save space take the time to use something like “bzip2 -9 backup.img.raw”.

A typical restore of rootfs/APP/mmcblk0p1 might be to copy backup.img.raw (or just the backup.img which is smaller and faster) to “bootloader/system.img”, then:

sudo ./flash.sh -r jetson-tx2 mmcblk0p1
7

So based on this, can someone suggest a command to copy them all or would I need to increment the mmcblk0p1 to mmcblk0p2 etc and add the name after -k. to get each one of these and have 17 different files to flash?

Number Start (sector) End (sector) Size Code Name

1 4097 58724352 28.0 GiB 0700 APP

2 58724353 58732544 4.0 MiB 0700 mts-bootpack

3 58732545 58733056 256.0 KiB 0700 cpu-bootloader

4 58733057 58733568 256.0 KiB 0700 bootloader-dtb

5 58733569 58739712 3.0 MiB 0700 secure-os

6 58739713 58739716 2.0 KiB 0700 eks

7 58739717 58740924 604.0 KiB 0700 bpmp-fw

8 58740925 58741924 500.0 KiB 0700 bpmp-fw-dtb

9 58741925 58746020 2.0 MiB 0700 sce-fw

10 58746021 58758308 6.0 MiB 0700 sc7

11 58758309 58762404 2.0 MiB 0700 FBNAME

12 58762405 59024548 128.0 MiB 0700 BMP

13 59024549 59090084 32.0 MiB 0700 SOS

14 59090085 59221156 64.0 MiB 0700 kernel

15 59221157 59221668 256.0 KiB 0700 kernel-dtb

16 59221669 59745956 256.0 MiB 0700 CAC

17 59745957 61071326 647.2 MiB 0700 UDA

from:Which partition is the EBT partition? - Jetson TX2 - NVIDIA Developer Forums

8

I only know of either running dd directly on the Jetson, or else to clone individual partitions using the GPT partition label. Does the Jetson you want to clone run? If so, what is the output of “sudo gdisk -l /dev/mmcblk0”? (replace the “-k APP” with “-k <whatever_label>”)

Note that cloning all partitions will not clone the entire disk…there is a 4KB block at the start of the disk which does not have a partition label.

9

I think I would be good missing the first 4kb. I’m worried about missing some of the BSP modifications to the kernel and other areas that might be touched. I’m trying to get this to be as simple as possible. I’m going to run the command now.

10

Output is:

nvidia@tegra-ubuntu:~$ sudo gdisk -l /dev/mmcblk0
[sudo] password for nvidia:
GPT fdisk (gdisk) version 1.0.1

Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: present

Found valid GPT with protective MBR; using GPT.
Disk /dev/mmcblk0: 61071360 sectors, 29.1 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): 00000000-0000-0000-0000-000000000000
Partition table holds up to 17 entries
First usable sector is 4097, last usable sector is 61071327
Partitions will be aligned on 1-sector boundaries
Total free space is 1 sectors (512 bytes)

Number Start (sector) End (sector) Size Code Name
1 4097 58724352 28.0 GiB 0700 APP
2 58724353 58732544 4.0 MiB 0700 mts-bootpack
3 58732545 58733056 256.0 KiB 0700 cpu-bootloader
4 58733057 58734080 512.0 KiB 0700 bootloader-dtb
5 58734081 58740224 3.0 MiB 0700 secure-os
6 58740225 58740228 2.0 KiB 0700 eks
7 58740229 58741436 604.0 KiB 0700 bpmp-fw
8 58741437 58742436 500.0 KiB 0700 bpmp-fw-dtb
9 58742437 58746532 2.0 MiB 0700 sce-fw
10 58746533 58758820 6.0 MiB 0700 sc7
11 58758821 58762916 2.0 MiB 0700 FBNAME
12 58762917 59025060 128.0 MiB 0700 BMP
13 59025061 59090596 32.0 MiB 0700 SOS
14 59090597 59221668 64.0 MiB 0700 kernel
15 59221669 59222692 512.0 KiB 0700 kernel-dtb
16 59222693 59746980 256.0 MiB 0700 CAC
17 59746981 61071326 646.7 MiB 0700 UDA

11

Also:

nvidia@tegra-ubuntu:~$ cat /proc/mounts
/dev/root / ext4 rw,relatime,data=ordered 0 0
devtmpfs /dev devtmpfs rw,relatime,size=7973052k,nr_inodes=1993263,mode=755 0 0
sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0
devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
tmpfs /run tmpfs rw,nosuid,nodev,mode=755 0 0
tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0
tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0
cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd 0 0
pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0
cgroup /sys/fs/cgroup/debug cgroup rw,nosuid,nodev,noexec,relatime,debug 0 0
cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0
cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0
cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0
debugfs /sys/kernel/debug debugfs rw,relatime 0 0
configfs /sys/kernel/config configfs rw,relatime 0 0
tmpfs /run/user/1001 tmpfs rw,nosuid,nodev,relatime,size=803988k,mode=700,uid=1001,gid=1001 0 0
fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0
gvfsd-fuse /run/user/1001/gvfs fuse.gvfsd-fuse rw,nosuid,nodev,relatime,user_id=1001,group_id=1001 0 0
nvidia@tegra-ubuntu:~$

12

You should be able to clone everything in the “Name” column of that “gdisk -l” output. You may want to save a copy of this for future reference as well since it shows offsets and size…if you were to manually reconstruct something at a later date it might come in handy.

In R28.1 of a TX2 the mmcblk0p15 “kernel-dtb” seems to be the device tree where both U-Boot and the Linux kernel look, but I don’t know if other stages in boot prior to U-Boot might look at device tree in other directories.

Most of what you see in “/proc/mounts” involves pseudo file systems which don’t exist except in RAM. The ones you get with “grep ext4 /proc/mounts” are the ones with files you can back up. If you clone the “APP” (rootfs) partition I believe this will include all real file system mounts unless you’ve customized something (such as mounting a different file system type from a custom eMMC partition).

13

when i’m using :

sudo ./flash.sh -r jetson-tx2 mmcblk0p1

the image always was overwritten by the original image, how i can make sure that i’m flashing my backup !

anyone can help me please ?

Thanks
Ebraheem

14

The command “sudo ./flash.sh -r jetson-tx2 mmcblk0p1” will flash without generating a “new” rootfs…the file currently existing in “bootloader/system.img” will be the source. If this file is not valid or is missing, then the flash program probably won’t tell you…it just copies bytes present. This command does flash brand new all of the other support partitions, e.g., the boot loader.

You must create a clone with the Jetson in recovery mode. If R28.1, then the command is something like:

sudo ./flash.sh -r -k APP -G backup.img jetson-tx2 mmcblk0p1

…which would produce files “backup.img” and “backup.img.raw”. A raw image is uncompressed and quite useful for editing and examining, and can be used directly as a replacement to system.img (flash would take significantly longer). The non-raw image can also replace system.img…you can’t edit or decompress this (neither Linux nor Android sparse file system tools will work on it), but the flash is much faster.

Note that clones and system.img.raw (any raw file) is extremely large. Check disk space with something like “df -H -t ext4” before flashing. A single raw image can easily approach 30GB. A sparse image depends on file system content, but a basic sparse image is typically another 2GB.

1 Like
15

We are working on jetson tx2 28.2.1 and we followed above steps to clone the image. Clone was successful and system.img created and that we replaced in “/bootloader”.

So with following command:

sudo ./flash.sh -r -k APP -G backup.img jetson-tx2 mmcblk0p1

It flashes successfully and previous cloned data retains when we flash cloned image.

But we want to add some logs using printk(), in Linux driver files and want these modified changes should reflect when we flash with cloned image, but we are not able to get those changes. So what flash commands or steps needs to be followed so that these changes reflected with cloned image.

16

The clone process would have produced both “backup.img” and “backup.img.raw”. The “raw” image is the full image without any kind of pseudo compression. This “raw” image is the one you can edit, but it will be much larger and slower to work with. If you don’t use the raw image you can’t edit since the sparse image is not compatible with any available tools. So long as the image is renamed “system.img” before flashing to restore the flash tools won’t care if the base image is raw or sparse…raw just takes longer.

The raw image can be loopback mounted. I’m going to assume you named it “backup.img.raw”. You can mount loopback devices anywhere, but I’ll use the traditional “/mnt”.

First a comment on “losetup”, the command for loopback: If you use “losetup --find”, then the command will name the first unused loop device. If the device does not yet exist, then the device will not be created unless you use sudo. Some commands both find and create…take note of when you need sudo…query does not require sudo, but create does.

For example:

# Find first unused device and create:
sudo losetup --find
# Cover the file and echo the name...create if needed, use existing if not needed:
sudo losetup --show /where/ever/the/file/is/backup.img.raw /mnt
# Now browse "/mnt" as if it were "/" of the image.
# Perhaps edit content somewhere, e.g., add a README.txt file mentioning edits you've added.
# Now be sure nothing is still holding "/mnt" and detach:
cd
sudo umount /mnt
# Alternative umount if loop0 is the device:
sudo umount /dev/loop0
# Now detach all unused loop devices just for safety...don't want a loop device leak:
sudo losetup -d /dev/loop0
# Now do your flash of clone in usual manner after putting "backup.img.raw" in the right place as name "system.img"

Incidentally, if you were to update that system with apt or add something, then you could rsync to the loopback mounted raw file to update it as well without a full clone.

Almost forgot, but very important: The kernel itself may be named via extlinux.conf in “/boot”, but newer releases tend to add components to partitions when those components are needed prior to mounting the rootfs. As such you will need to specifically find out if the kernel is actually used from the rootfs on the version of L4T you use, or if the flash tool is needed to place it in a partition. For L4T version see “head -n 1 /etc/nv_tegra_release”. It doesn’t hurt to try to see if a clone updates the kernel, but if it doesn’t, then probably the kernel isn’t the one in “/boot” and you need to investigate if it is instead in a partition.

17

I have upgraded to JetPack 4.2 via SDK Manager on ubuntu 18.04 because old linux versions as host systems via VM weren’t very stable for me.

SDK Manager doc says there is no CLI for Jetson devices. Does anybody know how to clone an entire TX2 with the SDK Manager toolchain on a 18.04 host?

18

I have not yet tried cloning via SDK Manager, but do know that the SDK Manager itself does not do flashes…like JetPack, it is a front end. The driver package does the actual flash and any cloning. Somewhere within your SDK Manager directory will be a “Linux_for_Tegra/” subdirectory, and within that “flash.sh”. You can try that with the TX2 in recovery mode and the micro-B USB cable connected.

For details see:
[url]https://devtalk.nvidia.com/default/topic/1028867/jetson-tx2/clone-entire-tx2/post/5234090/#5234090[/url]

19

Thank you, it worked, on R32.

For reference: I used the backup method as descriobed here: Jetson/TX2 Cloning - eLinux.org
And then I symlinked my compressed backup.img as bootloader/system.img and used the following command on a blank TX2:

sudo ./flash.sh -r jetson-tx2 mmcblk0p1
20

I used these instructions to get a clone of my existing Tx1, and it works flawlessly, with one exception.

Eventhough it clones the Tx1 rootfs, it’s not a “whole” or “master” image. It doesn’t clone/flash the kernel, device tree and uboot with these commands.

Are there any command to both “clone” and “flash” the tx1 completely(including dt and kernel as well)?
This is still useful to clone the rootfs, but I need to flash the kernel and devicetree everytime after. Then configure u-boot env. variables.