My Application is working on drm. So I want reduce the size of rootfs.
I find a webpage “Jetson/FAQ/BSP/RootFS Reduction - eLinux.org”. But I find that the “usr” folder is much larger In reality. Similarly, vlc chromium ffmpeg gstream libreoffice is not useful for me.
Is there a minimum rootfs packet or a script?
Hi,
Please check this post:
DS app running on SD, not running on eMMC - Part II - #5 by DaneLLL
CUDA and TensorRT have large size. if you don’t need the package in your use-case, may not install these to save the space.
cuda and tensorrt is not large. But the ubuntu is. Ubuntu is about 5GB. I see ubuntu graphic mode is waste the space, similarly, x11. Because it is a produce. If the image is too large. We will waste more time to flash it.
I know I can apt remove --purge and pack the image from jetson. But I think it’s not flexible. I want to know if there is any method or scriptor for redefining the rootfs before the image flash into jetson?
Two shortcuts:
- Replace the content used to generate the image in “
Linux_for_Tegra/rootfs/”. - “Reuse” the image and have a custom image as “
Linux_for_Tegra/bootloader/system.img”.
Method 1 will generate its image from that. Do beware though that some content in “rootfs/boot/” will be added prior to image generation, e.g., the Image file and extlinux.conf may change because the arguments passed to the flash script puts the proper kernel and extlinux.conf in place based on the name of the target.
Method 2 is an exact method, e.g., via using a cloned image. That cloned image can be created after removing packages, updating, so on. Beware that the passwords and any customization will be a 100% exact match, so if you want first boot setup to remain, then you’d need to add that back in just prior to cloning (or using QEMU on the loopback mounted raw clone image, and then to shorten flash, make the sparse image out of the raw clone).
Method 1 can actually be achieved by mounting a loopback image on “rootfs/” prior to a normal flash, but it would have to be read-write, and this would modify your loopback mounted image’s “/boot” content. One could also create the method 1 “rootfs/” via something like rsync (which can maintain permissions and numeric user IDs…regular copy won’t necessarily work for this).
Do you have the link about method 2? I think it’s comfortable to me ^_^
Method 2 assumes you have a clone. There is more than one method to clone, but the best one is with the Jetson in recovery mode and using the flash software to clone instead of alter the Jetson. The following will produce “raw” file “my_backup.img.raw” and also “sparse” image “my_backup.img”:
sudo ./flash.sh -r -k APP -G my_backup.img jetson-tx2 mmcblk0p1
You could then copy either backup image to “Linux_for_Tegra/bootloader/system.img” and “reuse” that image via command line flash with the added “-r” option. Example:
sudo ./flash.sh -r jetson-tx2 mmcblk0p1
(use the same release of flash software as the release which created the Jetson being cloned)
The sparse image is smaller and faster if the filesystem was not full at the time of clone, whereas the raw image is slower, but also allows loopback mounting and editing. I highly recommend doing all updates and customizations you are interested in prior to cloning. Note that the loopback mounted sparse image can be updated later with rsync against a running system just like any other backup method.
Hello, I have done. But after I do sudo ./flash.sh -r -k APP -G tx2nx_20220607.img jetson-xavier-nx-devkit-tx2-nx mmcblk0p1 , I found that size of tx2nx_20220607.img(14GB) is equal to tx2nx_20220607.img.raw, why? tx2nx space useage is about 5GB.
If the filesystem is not full, then they should not be the same size. Did the host PC have enough space on the filesystem to continue? If you are in the “Linux_for_Tegra/” directory, do you have sufficient free space via the command “df -H .” (where the “.” is important)? What is the exact byte size of the image? If it is a valid raw image, then it will be a multiple of 1024 (for an Orin I think it will be a multiple of 4096, but this too is a multiple of 1024).
On the tx2nx, I df -h the mmcblk0p1 free space is about 7GB
Filesystem Size Used Avail Use% Mounted on
/dev/mmcblk0p1 15G 6.2G 7.9G 44% /
and I sudo ./flash.sh -r -k APP -G tx2nx_20220607.img jetson-xavier-nx-devkit-tx2-nx mmcblk0p1.
I found that the tx2nx_20220607.img is 13GB. tx2nx_20220607.img.raw is 14GB.
-rwxr-xr-x 1 root root 13G 6月 7 16:25 tx2nx_20220607.img
-rw-r--r-- 1 root root 14G 6月 7 16:12 tx2nx_20220607.img.raw
In the Linux_for_Tegra/ directory, I run df -H .and show that
/dev/sda1 492G 435G 32G 94% /media/alfred/data
In my opinion, I think tx2nx_20220607.img.raw is about 15GB, tx2nx_20220607.img is about 6GB.
I recovery the tx2nx_20220607.img. I can see that the free space is exactly 7GB.
I don’t know if iflash.sh works well.
if it works well. How can I compress the img?
if it dose not work well. How to fix it?
Unless this is the space after a successful clone, this will not be enough space since the raw image should be the exact byte size of the partition in question (it is necessary to know the exact byte size of that clone to know if the raw clone is valid…so far I’ve only seen an approximation of size). The raw image is created first, and then the sparse image is created from the raw image. Do you have an exact size of the raw image?
Also, if the raw image is valid, then you should be able to loopback mount it and examine it. Example:
sudo -s
mount -o loop ./tx2nx_20220607.img.raw /mnt
cd /mnt
ls
cd
exit
You can only compress the image after clone is finished. The sparse image will normally fail to gain anything by compression, but the raw image can be compressed by any compression method, e.g., gzip or bzip2 (and it will take a long time to compress).
I see that system.img is just 5GB. When I sudo ./flash.sh -r jetson-xavier-nx-devkit-tx2-nx mmcblk0p1.Then I df -h on the tx2nx, I see the usage space is 5GB too.
In my case, I think tx2nx_20220607.img is 6GB. But why it is 13GB?
The img.raw must be valid. Because the img is from img.raw and the img can be used. I have flashed it to the tx2nx. And works well, and the usage space is 6GB.
Please give an exact size for the raw image, e.g., “ls -l tx2nx_20220607.img.raw” (approximations won’t answer that question). It is necessary to know if it is an exact multiple of 1024 bytes, but likely if loopback worked, then the image is valid (I’m not sure if it is possible for loopback to work with a truncated image…even if it is, then it should complain about a corrupt filesystem). I am very suspicious as to whether the host PC performing the clone has enough space.
Note that a raw clone is an exact bit-for-bit copy of the partition itself. That partition is only valid if it is a multiple of 1024 bytes. An example is that many of the smaller systems have approximately 14 GiB bytes (a GB is a multiple of 1000*1000*1000, a GiB is a multiple of 1024*1024*1024), so 14 GiB would clone to a raw file of size 15032385536 bytes (which is 14 times 1024 three times).
When calculating space requirements on the host PC keep in mind that there may be temporary files or files which are momentarily increasing space requirements. For a smaller partition like 16GB in magnitude you probably need about 25GB of free space before starting the clone. You can manually delete the old system.img and system.img.raw from old clones and from “Linux_for_Tegra/bootloader/system.img*” before starting. I don’t know that space is related to your original issue, but there isn’t much of a way to continue debugging without knowing space is sufficient. So I am looking at exact sizes to see if what is really below recommended free space in order to find out if it is really part of the problem or not.
Yes, it is15032385536. tx2nx_20220607.img is 13687922688.
In the Linux_for_Tegra/ directory, I run df -H . and show that
/dev/sda1 492G 435G 32G 94% /media/alfred/data
32GB, it seems to be enough.
mount -o loop ./tx2nx_20220607.img.raw /mnt
ls /mnt/
bin dev home lost+found mnt opt README.txt run snap sys usr
boot etc lib media nv_preseed.cfg proc root sbin srv tmp var
df -h /mnt/
/dev/loop10 14G 5.5G 7.6G 42% /mnt
I found that if I rm -r rootfs and then mount the img.raw and the sudo ./flash.sh jetson-xavier-nx-devkit-tx2-nx mmcblk0p1 The new img is about 5.7GB
This verifies the raw clone is valid (and likely, unless there is an error indicated from mksparse, so too would the sparse clone be valid).
Mounting to “rootfs/” will work, but beware that it will possibly change the kernel and extlinux.conf. Directly on the loopback mounted partition (these are picked via flash arguments just before generating “bootloader/system.img”). Did flashing with this do as you want?
An interesting bit of trivia is that you don’t need to delete the original “Linux_for_Tegra/rootfs/” content. If you loopback mount (or just mount) a partition on “rootfs/”, then the original content goes away until the partition mount is removed. It does of course save disk space.
Yes, this is my image what I want. The extlinux.conf is not changed.
I have sudo mkdir rootfs then mount, so the rootfs is empty.
I want to know why the “flash.sh” can’t work well and will you fix it in the future? JP4.6.2 and tx2nx
I am confused by this…are you speaking of flash.sh modifying the “rootfs/boot/” content? If so, it is intended to do that. In fact it will possibly modify the kernel and extlinux.conf on the loopback mounted system as well, but if the argument used to create the original cloned system from flash matches the arguments passed to flash from the loopback image, then the files will be the same ones, and so appear to be unchanged (but a timestamp is probably changed).
Only reusing the system.img was intended to be a perfect match. “system.img” is the data being flashed, while “rootfs/” is for the specification of what occupies that image during creation. Since flash.sh works with more than one Jetson module, and more than one carrier board, the final flash.sh target must change some final details (e.g., a different device tree is needed for different Jetsons, and a different extlinux.conf content is needed for an SD card model versus an eMMC model, and versus different types of Jetsons from TX1 through Orin).
I confuse that if I use sudo ./flash.sh -r -k APP -G backup.img jetson-tx2 mmcblk0p1 The backup.img is too large. But when I use mount backup.img.raw rootfs and ./flash.sh jetson-tx2 mmcblk0p1 The image is the right size.
If I use mount backup.img rootfs and ./flash.sh jetson-tx2 mmcblk0p1. When the flash is complete, I find that the image in the tx2nx is as the same as backup.img. I have redefined the content of the extlinux.conf in the backup.img. I compare the extlinux.conf in tx2nx and in the system.img.raw. It’s the same. Maybe I just have one Jetson.
I know what you mean. I change the rootfs and flash and it shows that
populating kernel to rootfs... done.
populating initrd to rootfs... done.
populating kernel_tegra186-p3636-0001-p3509-0000-a01.dtb to rootfs... done.
Making system.img...
nvidia/nvidia_sdk/JetPack_4.6.2_Linux_JETSON_TX2_TARGETS/Linux_for_Tegra/rootfs/boot/extlinux/extlinux.conf is not found, exiting...
and How can I fix it? ~_~
Two backup images are created: A “raw” clone (“backup.img.raw”) and a “sparse” clone (“backup.img”). The raw clone is an exact match of the partition. If the partition is 28GB, then so is the raw clone. The sparse clone is not useful for much except flash, but it is smaller unless the filesystem is filling up (the sparse clone approaches the size of the raw clone as file content in the filesystem fills up to 100%). When you say “too large”, if you are just referring to being so big it is a pain to work with, then you are correct, but this is how it is intended. I throw away the sparse image, and after using the raw image, I compress it with “bzip2 -9” (which in itself takes a long time…simply copying such a file from one place to another takes a lot of time).
Yes, the result is “in most cases” the same whether you reuse the image with the “-r” option or mount it on “rootfs/”. If the kernel and device tree are updated on the clone, then flashing via a clone mounted on “rootfs/” could lose the kernel and device tree updates…but this only matters if such updates exist. When flashing with an image reused as “bootloader/system.img”, then nothing will ever be changed or lost in the rootfs. Restated, if you copy a reference of a kernel to a location which already had that kernel, then it will appear unchanged.
I am a bit confused by the error you mention. At the time of running the command “sudo ./flash.sh jetson-tx2 mmcblk0p1”, was the clone mounted on “rootfs/”? If so, can you see “extlinux.conf” at “rootfs/boot/extlinux/extlinux.conf”?
But in fact that, my file content is about 5GB 1/3 of the img.raw.
No. Here is all my steps
sudo rm -r rootfs
sudo mkdir rootfs
sudo mount backup.img rootfs
sudo ./flash.sh jetson-tx2 mmcblk0p1
Wait for it completing
sudo umount rootfs
sudo rm -r rootfs
sudo mkdir rootfs
sudo tar xvf Tegra_Linux_Sample-Root-Filesystem_R32.7.2_aarch64.tbz2 -C rootfs
sudo ./flash.sh jetson-tx2 mmcblk0p1
All of the steps with “rm” of “rootfs/” were unnecessary. While something else is mounted there that content is inaccessible and the system has only the mounted content available.
This was incorrect:
sudo mkdir rootfs
sudo tar xvf Tegra_Linux_Sample-Root-Filesystem_R32.7.2_aarch64.tbz2 -C rootfs
sudo ./flash.sh jetson-tx2 mmcblk0p1
After unpacking the rootfs content, but before flashing, you missed the step which installs NVIDIA driver content:
sudo ./apply_binaries.sh
(this only needs to be done once)
Without the apply_binaries.sh your native (not from a separate mount) content will not be valid.
Please note that the backup.img.raw of the clone will never be “the wrong size”, or at least it will be an exact match to the system it came from down to the last bit. If this is too large for some other system it is going to, then this can be dealt with, but other than convenience of size there should be no issue.
The “backup.img” (sparse) is not guaranteed of any size. Replacing NULL content with a formula which expands into empty ext4 space is what this is for. When that space expands, then it should be an exact match for the raw image. It isn’t really possible to completely estimate what its size will be, but the installed image, regardless of starting as raw or sparse, should be the same thing if you have not altered the raw image. The way you mounted the raw image in “rootfs/” it is possible there were minor edits.
Just prior to generation of system.img from “rootfs/” some “rootfs/boot/” content is altered by copying in things like kernel Image and the extlinux.conf which the system believes is correct for the arguments passed to the flash software. If this happens to be the same exact files which were already there, then then only possible difference will be a time stamp. If those are different, then it has just overwritten that content from the clone and turned it into the default content.
If you want an exact match each time, then you should only use the clone image as “bootloader/system.img” and “reuse” that image. It wouldn’t matter if you use the raw or sparse image there, but if it is raw, then you can choose to loopback mount this and edit it prior to the flash (you wouldn’t want to use the file as system.img while it is actively mounted).
It isn’t clear what it is you actually want to accomplish when comparing the two methods of flashing from a clone as “system.img” versus mounting on “rootfs/”.