I’ve inherited several TX1 images that were compressed using bootloader/mksparse. I don’t have the raw image files. I’d like to turn these back into files I can mount on my desktop. I know I could flash them onto a board, then clone the image, but was hoping there was a faster way.
“Sparse” is a way to store a file system which isn’t just for Jetsons, there are places throughout Linux where this is supported. The trouble is that it may be supported differently (or not at all) on different Linux distributions. The mksparse file has the ability to turn uncompressed raw into sparse, but I do not believe it can turn sparse back into raw…someone from NVIDIA knowing the internals of mksparse would need to comment.
The indirect (slow) method would be to flash with the sparse file (flash produces a raw file on the Jetson), and then clone…clone will give you back both a raw and a sparse file.
Let me ask though, are the images you acquired all from the same L4T release version? And if so, which version (e.g., “R28.1” is the most recent)?
Depending on your host Linux version this might differ, but try this just to see if it works (file systems are large…expect this to take a long time and need many GB of space):
cp --sparse=never ./the_sparse_file.img not_sparse.img.raw
…this stands a good chance of working, but I have not tried (and even if it works for me I’m running Fedora 27 at the moment and it would likely differ from your results).
If this works you should be able to mount on loopback (this example is read-only to protect the image):
sudo mount -o loop,ro ./not_sparse.img.raw /mnt
cd /mnt
ls
cd -
sudo umount /mnt
Do keep in mind that the hidden partitions which were present on the original system which was cloned can make a difference in whether or not your clone works, e.g., you wouldn’t want to restore a clone from R24.2.1 onto a system which had been flashed with R23.1 or R28.1.
1 Like
Thanks for your help. I’m at least learning something about sparse files in Linux.
All images are the same L4T version: R24.2.1. I’m running Ubuntu 16.04.
cp --sparse=never produces a file identical in size and contents - I checked with diff. The error I get on mounting is what you would expect:
mount: wrong fs type, bad option, bad superblock on /dev/loop0,
missing codepage or helper program, or other error
Nothing is printed to dmesg or syslog.
My first thought was to try to reverse-engineer mksparse, but it’s not human-readable. I also hope Nvidia chimes in.
Creating sparse files isn’t specific to NVIDIA, although the tool is. Sparse files are described here:
[url]https://en.wikipedia.org/wiki/Sparse_file[/url]
The fill byte specified is NULL, which could be anything (it’s customizable), but when the sparse file is converted to raw the Jetson specifically marks fill bytes as NULL. Knowledge of this, and perhaps of ext4 is basically all of the customization via standard configuration.
The sparse file was not originally used for flashing earlier Tegra devices. Using a non-sparse raw image still works without any effort if you use this as system.img (it seems that it is the Jetson itself which knows the difference between raw and sparse…the driver package just sends bytes and does not care about content). The reason why this was introduced is because of the amount of time it took to flash when for example you downloaded 15GB instead of 2GB…addition of sparse support saved a lot of flash time. The conversion from sparse to raw is done on the actual Jetson during flash, so there was no need to convert from sparse to raw on the host. Similarly, old clones used to only clone the raw partition, whereas more recent flash clone software produces both system.img and system.img.raw (and in fact you might want to check if you have a “.raw” version of all of your “.img” versions). So far as I can tell a clone sends the raw image, and the PC side host uses mksparse to create the sparse version of the image (this is why clone takes longer than flash).
I tried a number of methods to test converting sparse to raw on my PC host, but so far do not have an answer. I’m researching, it seems there should exist some method to do this.
Actually I found a solution(after 4 years):
you can use the android simg2img tool, in ubuntu you can install it by
sudo apt install android-sdk-libsparse-utils
then just
simg2img <spare.img> <uncompressed.img.raw>
3 Likes
Nice to know about that…I’ll give that a try one of these days when I have just the sparse image. Sounds quite useful.