Thank you very much for your reply.
rsyncwould need further steps to use as a clone if you were to flash with it, but the content would be available and safe so long as you had that copy，
You mentioned it above, could you please explain it further
I’m the engineer so you just ask me.
Put your raw image on a USB drive, then run the following command.
(I’m not sure if this command is available on 34.1, but you may give it a try.)
sudo ./tools/kernel_flash/l4t_initrd_flash.sh --initrd <board-name> <rootdev>
The device will boot into initrd, then mount the USB drive, and use dd to put the image back to /dev/mmcblk0p1.
Still, you should not stay at any developer preview version once a stable version is released.
You really should consider upgrading.
This isn’t specific to your case, and you likely do need a more recent release, but to aid in backing up and not losing anything I will mention both some standard Linux backup tools, plus talk about clone. Lots of people can use this so I’m adding more detail than expected.
Any time you are backing up files you hope they do not change in the middle. An ordinary cp (copy) command is not able to properly anticipate changes, so it is normally not used. Additionally, there are different types of files, and different types of filesystems, and not all software understands this (even if the software can use those files, it isn’t always understood by that software what it means to copy or preserve such a file).
The number one tool for this on a Jetson is cloning. This is performed in recovery mode, and so you know for a fact that no file will change. Better yet, it also means there are not temporary files from running programs. Most of those temp files go away upon shutdown, and the Jetson clone mechanism is as close to perfect as you can get regarding read-only mode and the proper shut down state.
If you happen to have a raw clone, which is the exact size of the partition itself, e.g., named system.img.raw, then you can use loopback to fool the system into thinking this is a disk partition. Once mounted somewhere, you can read, examine, copy, or alter that content at will. Example if you are located in a directory with raw clone system.img.raw:
sudo mount -o loop ./system.img.raw /mnt
cd /mnt
ls
cd -
sudo umount /mnt
You can’t do that with the “sparse” file; the sparse file removes the “empty space” (reminds me of an unrelated book, “The Physics of Empty Space”). So a sparse clone is the size of the content within the partition; as the partition is filled, the size of the sparse clone approaches the size of the raw clone. Sparse clones are faster to flash with, but either can flash the rootfs. Only raw clones can be manipulated. There are tools for converting sparse to raw, but I’ve not had luck with those tools in the Jetson version of a sparse partition (admittedly, I have not tried for a long time).
Even if your raw clone says there is an error, if you can mount it on loopback and see content, then your raw clone is useful for rescue or study. I personally throw away sparse clones, and if not using the raw clone, I run bzip2 -9 on it to compress (which takes many hours, as these are enormous files).
Regarding rsync, this is a very reliable and flexible tool for copying all filesystem content. rsync understands things like symbolic links, device special files, crossing mount boundaries, so on. rsync also gracefully handles changing content during the backup. “Gracefully handling” is not as good as working with a read-only filesystem, but it works. rsync does have to have a running system, so you might find the state preserved is that which includes temporary content. A very very nice part of rsync is that it can run through an ssh tunnel to a remote host…that means the backup media (and restore media) do not have to be on the same computer. I discourage doing this over Wi-Fi, but Wi-Fi can work; wired ethernet is quite good though at backup.
Do remember that when you run rsync you will need to be admin (root, via sudo). So likely you will want to just drop into a root shell via “sudo -s” or “sudo su”. The option “--dry-run” is wonderful as it will give you the messages of what it would as if it is actually doing it, but no change or file write will ever exist; you can see what rsync would do if it were actually running that command.
Important: Will you back up to media on the Jetson, e.g., a USB hard drive? Or will you back up to a remote host over ethernet whereby the remote host has the backup media? I can be more specific if I know that. Describe your backup media and location with as much detail as possible.
Some example rsync, with --dry-run, and which might be typical of a PC backup from one Linux system to another over ssh (note that despite putting output to ~regular_user/ as a destination, this is going through user root on that remote system; more on that below):
# Assumes the backup host has user "regular_user", with directory "~/Downloads/backup/jetson/":
rsync -avczrx --dry-run --numeric-ids --info=progress2,stats2 -e 'ssh' --delete-before / root@another_computer:~regular_user/Downloads/backup/jetson/
Whenever two separate Linux systems are copying back and forth between them, they need a user to give ownership to. User names are not what the system actually stores. Instead there are numeric user groups and user names, and the name itself is just an alias for the numeric ID. If the list of users are not an exact match between the systems, then translation will occur. If you try to restore with translated IDs, things will only appear to go well, but the backup and restore won’t be completely valid. If you use the option “--numeric-ids”, then no translation is allowed; this is an exact and perfect copy of security. Only root is allowed to do this, which is why you are backing up to the remote host’s root account; the placement of the output in “~regular_user/Downloads/backup/jetson/” is just a convenient location. More likely it will be something like an external USB hard drive mounted there.
One could substitute the ssh part and back up with that same command to an externally mounted USB drive. For that example, I am going to assume you have user nvidia as your regular user on the Jetson. I’m going to assume that the external USB hard drive is mounted at “/mnt/backup”:
rsync -avczrx --dry-run --numeric-ids --info=progress2,stats2 --delete-before / ~regular_user/Downloads/backup/jetson/
The above is more or less the same backup. Note that you can also rsync to and from loopback mounted partitions. If you have as your source the mount point of a loopback partition, then that can be source or destination. Or both if you have two loopback partitions.
Let’s say that you create a blank and completely empty file. You then cover the file with loopback, which means you can format it as if it were a hard drive partition. If that file is the exact size that the system.img.raw would be during a flash (more on that below), and if you have covered that file with losetup such that it is now device “/dev/loop0” (the number changes), then you could format it:
sudo mkfs.ext4 /dev/loop0
Next you could mount it (assuming loop0 is still attached):
sudo mount /dev/loop0 /mnt
You now have an empty partition of the exact size as the underlying file, and it is awaiting population. Perhaps population is via rsync from your other clone which was not a convenient size. The original rsync command can be across two loop devices, e.g., perhaps the clone is /dev/loop0, and the new partition is /dev/loop1. That’s a lot of disk space, and this won’t be fast. This might take most of the day. Keep in mind that when copying one partition to another by any means, even if it is rsync it is best when read-only; the best read-only is if the filesystems are not even mounted. So you should have loop0 attached to your cloned source media, and loop1 should be attached to your destination, but try this without mounting the source device (you need to mount destination, it i–info=progress2,stats2s being written; don’t mount source); if mounted, try to mount read-only.
rsync -avczrx --dry-run --numeric-ids --delete-before /dev/loop0 /dev/loop1
And of course then remove the “--dry-run” if it looks correct. I also recommend this option for some feedback during each run:
--info=progress2,stats2
(I probably should have just used this in all of the examples; I’ll append that to one of the first examples; if I miss this and don’t have that option, add it in)
Note on “--delete-before”: rsync works on existing backups. If a file is the same and not changed, then rsync will avoid unnecessary copy. If you cloned a system, and have a loopback remote mounted clone, and then you do something like “sudo apt-get upgrade” to update a few packages, then you could use the same rsync, with the destination to the existing clone, and rsync would update only the changed files. The time to do so would be very fast in most cases if you’ve only made a few changes (it does have to checksum all content to know if there was a change, so it isn’t instant). Without “--delete-before” an update via rsync is “safer” in the sense that the old file is left in place until the update finishes; if there were a power failure or something in the middle of an rsync without “--delete-before”, then your old copy is still there, but if failure occurs with “--delete-before”, then you’ve lost the old file which was being backed up. The win is that using “--delete-before” does not require as much disk space (consider that at any moment without --delete-before you have two copies of each file).
About empty files…
Usually flash requires certain multiples of a power of 2. The earlier mentioned “not a multiple of 4096” is wanting 2^12 bytes. This might be a bug so far as the original message, I’m not really sure if all such similar bugs have a failed size. However, let’s assume you want to create a destination loopback mountable file which is a multiple of 4096 in size. There are a number of ways to create the empty file, e.g., truncate or fallocate. Just for giggles (hey, it’s close to giga!) we will assume you are creating this size:
# 4096 is 4 KiB
# 1024 is 1 KiB
# 1024*1024 is 1 MiB == 1048576 bytes
# 1 MiB * 1024 is 1 GiB == 1073741824
# We will create a 100 GiB partition:
1073741824 * 100 == 107374182400 bytes
# Example 1:
truncate -s 107374182400 destination.bin
# Example 2:
fallocate -l 107374182400 destination.bin
rsync -avczrx --dry-run --numeric-ids --delete-before /dev/loop0 /dev/loop1
The above will take a long time, but not actually too long since the content is empty.
Let’s format as ext4 after creating destination.bin:
sudo mkfs.ext4 ./destination.bin
We could just mount this with “mount -o loop”, but we don’t necessarily want it mounted. We can cover it with loopback and find the name of the loopback device like this (which does not mount):
# Most of this requires root, thus `sudo`.
sudo -s
losetup --show --find ./destination.bin
# You need to know which loop device was mentioned. I'm using "`/dev/loop16`"
# because my system already has the first 15 devices used. Adjust for your case.
# verify it is a block special device (hard drives are block devices, not serial devices):
file /dev/loop16
# Test mount and umount via the loop device name (no "-o loop" because it isrsync -avczrx --dry-run --numeric-ids --delete-before /dev/loop0 /dev/loop1 itself
# a loop device):
mount /dev/loop16 /mnt
df -H -T /mnt
umount /dev/loop16
# We don't want to stay as root forever:
exit
Note that you can run out of loopback devices if you keep creating them. Some commands release them. You could do this to release a specific loopback device (or reboot releases it):
sudo losetup -d /dev/loop16
As long as we have loop16, you can use this as a destination if you mount it somewhere. If your original clone is “/dev/loop17” after this (“sudo losetup --find --show”), then you don’t want the source mounted if and only if the software used to read this does not need a mount. I have not mentioned the dd command, but it works with unmounted partitions. In the case of two loopback mounted partitions for rsync, both must be mounted. However, the source could be read-only. I am going to assume loop16 is the destination empty ext4 file, and that loop17 is the source file. If you ever get confused by the number of loop devices check sudo losetup --list. Example:
sudo -s
# Even if I don't use all of these I will create them:
mkdir /mnt/source
mkdir /mnt/dest
mount /dev/loop16 /mnt/dest
mount /dev/loop17 /mnt/source
rsync -avczrx --dry-run --numeric-ids --delete-before /dev/loop17 /dev/loop16
You’ve just created a clone of loop17 superimposed on loop16, but loop16 can be a different size. Space in a partition is not 100% available in terms of the byte size of the partition since formatting takes space. However, roughly speaking, if you have a system.img.raw which is the wrong size, then so long as the destination is large enough for the files within the system.img.raw, you can put it in any size of destination via rsync. The original “not a multiple of 4096” issue might actually be a bug, but if you do run into a need to resize safely to another file which is of a particular multiple, then that is one way to do it.
I also will suggest that although it is not as safe, you can directly use apps which work with partitions directly on the loopback device. If for example you want to see what the specs are of /dev/loop16:
sudo gdisk -l /dev/loop16
lsblk -f /dev/loop16
# You could even change the partition label or ID just like a real partition.
To that end, programs which resize a partition, e.g., gparted, work with the loop device. Programs such as gparted can easily reduce a partition size, but the real fun is trying to increase size. You can in fact create an empty file far larger than the partition, cover that as a disk, and then make subpartitions within the disk; if the first partition matches your original partition, then you can rsync to the new first partition (losetup can be a bit trickier, and dd is easier to use for some of those purposes), and suddenly your gparted can also enlarge the partition because you have extra room at the end of the loopback file. This gets more complicated, but you can always ask if you want that information. It is easier to just create a big enough file in the first place and rsync, but when doing data recovery with dd the gparted method might work (basically you are repairing metadata and missing data, and dd is superior to rsync for that purpose).
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.