I’m using Yocto (with meta-tegra) and flashing my TX2i with the deployed tar.gz.
I’ve made some customizations and I have U-Boot->Kernel->User_root booting flow, where each component has a backup partition as well.
The kernel partition is a read-only ext4 (I call tune2fs -O read-only when creating the ext4 in Yocto). This ext4 contains the initramfs and extlinux.conf as well.
After flashing the Jetson, I compare the data in the relevant partitions (for the kernel-ext4 and kernel-ext4-backup) with the original file I flashed.
I find a small difference (only 2 bytes out of 50MB partition) and it looks like some kind of changed ext4 header since it looks when using hexdump that the change is in a meta-data area of the file-system.
It is worth mentioning that dumpe2fs on the data in the partitions and the original file yields the same output.
This is an issue to me since I want to use rdiff with SWUpdate to update those partitions from time to time, yet the change that happens on flash prevents me from calculating diff from the original file I flash.
Hi,
Honestly speaking, your test has too many factors that are not from our official boot flow so we cannot make sure what is going on.
Is this is only 2 byte difference, I would suggest you could check the dtb or image is being signed or not.
The description of the boot system was just a way to explain my motivation.
The problem itself is much more specific - using the standard flashing procedure with a slightly modified flash.xml.in, a partition (unsigned) that is flashed with an ext4 file-system is modified slightly after flashing.
Hi,
Does this issue cause any fatal side effect? Could you then use the original flash procedure and share the steps to guide us how to find out this problem?
This does not cause any side effects and the Uboot successfully boots this Kernel partition.
I do think this has to do something with the ext4 attributes.
For example, using a standard JetPack, meaning flashing a sparse image to APP will result in the data in the APP partition to change after boot due to journaling in ext4.
Regarding my ext4 read-only file, I can mount it locally on my host, and than unmount it and the checksum does not change.
Well it look like after the flashing, and using dumpe2fs that the following attributes were added:
Journal features: journal_64bit_journal_checksum_v3 (was none before)
Journal checksum type: crc32c (did not appear before)
Journal checksum: 0x00000000 (did not appear before)
Using tune2fs and disabling journaling solved the problem, but I still don’t know why this happened.
I do not know if what follows is related or not, but there are some 64-bit ext4 features which are not supported in the default L4T flash. This could be completely unrelated to what you are observing, don’t know.
Previously, when a file needed to be read in bootloader stages from an ext4 partition, the bootloader chain did not have an ability to read 64-bit extensions of ext4 even though the Linux kernel itself could read this. Whenever someone had a host PC creating the ext4 partition and using those extensions, then boot would fail. If you take a look at your formatting host’s “/etc/mke2fs.conf”, within the ext4 block, look for “metadata_csum” and “64bit”. Those would become part of your ext4 filesystem if formatted from that host without options to disable those.
I’ve not looked at the 64-bit journal options, but it is possible similar options might be related to the journal. If software working on the journal handles 64-bit in one case, but does not handle the same way in another case, then I might expect some of the related checksums to change or fail when alternating between boot stage software and the Linux kernel.
In particular, note that one of those attributes which caused issues in the past is “metadata_csum”. I think some of the 64bit is ok now, but metadata_csum, even if it is handled correct in boot software now, might be related to different checksums. On your host PC which performed the ext4 format, and the computer which performed any tune2fs, what do you see for available options via:
gawk '/ext4 =/,/[}]/' mke2fs.conf
…if you see “metadata_csum”, then try formatting and/or tune2fs to disallow metadata_csum and examine if results change.
Disabling the journal is a really bad idea (other than for testing). It means the only repair of the filesystem no longer uses the journal and must use fsck for every single time there is an issue.
Thank you very much for your reply.
First, I checked the data content of a partition I did not boot from (no Uboot/Cboot tried to load) and saw a difference.
Second, the attributes were added AFTER the flash, which is opposite of what you try to say, if I understand correctly (you talk about attributes possible in the host but not by boot logic).
This leaves me wandering if the flash process changes anything
Each flash generates a new rootfs partition. It starts with a file of the right size, covers it with loopback, and then runs mkfs.ext4 (or its equivalent) on the loopback device. Finally, it copies files from “Linux_for_Tegra/rootfs/” into that (with a few “rootfs/boot/” file edits prior to doing so). The exact bit-for-bit copy of that loopback file is then flashed into mmcblk0p1. This means whatever your “Linux_for_Tegra/bootloader/system.img.raw” has in it (after loopback mount) is exactly what your Jetson will have.
I do not know what was actually added after the flash from the tune2fs. I am wondering if something added was incompatible during boot, prior to the Linux kernel itself running.
Sorry for confusing you. I’m not using JetPack I’m using Yocto and building a custom ext4 by creating a file (by myself), creating a file-system using mkfs.ext4 and tune2fs (to make it read only) and doing those copies you mention by myself. Them I edit the flash.xml.in to flash the custom partitions I created with those files.
This means I control the process entirely.
When I compare the file in the tar.gz which I flash (by explicitly using it’s name in the flash.xml.in) with the content of the custom partition, I see those journal attributes added.
This what makes me wonder whether the flashing process itself changed something.
- Did you create “
Linux_for_Tegra/bootloader/system.img.raw” yourself? Are you copying content into “Linux_for_Tegra/rootfs/” and instead letting the image be created from that? - Are you seeing differences in files, or differences in partition size?
- If there is a partition size difference, what is the exact byte size of the partition prior to flash?
- Are you running
tune2fsdirectly on the Jetson, or on a loopback file?
Note: You can temporarily remove a feature from the host PC’s “/etc/mke2fs.conf” ext4 section and this should at least test if a filesystem is being created on the host with a different set of options.
I will test altering /etc/mke2fs.conf.
Regarding your other questions:
Again, I’m using Yocto and meta-tegra. I’m using custom recipes to manually create an ext4 which I copy the Image, initrd and extlinux.conf by myself.
The partition is manually set to size 50MB by me in the flash.xml.in, and I manually edit the <file></file> tag of that custom partition to flash my custom kernel.ext4 50MB file.
The read-only attribute is set by me on the host, and that the only flag I set using tune2fs in the host.
I see no change at all in the paririon size after flash, only the journal attributes change after the flash (attributes ADDED after flashing).
It looks like attributes are ADDED compared to the file I have on my host and is supposed to be flashed unchanged using meta-tegra do-flash.sh (which calles tegra186-flasher-helper.sh which in turn invokes tegraflash.py).
The flash itself will demand a size evenly divisible by 1024 at least twice (“MiB”), or three times (“GiB”). If the image does not fit within that requirement I would expect the image to be appended to. In “flash.sh” the “-S size” is used to set alternate sizes, e.g., “-S 28GiB” or “-S 50MiB”. Make sure your 50MB is a match for “50MiB” where MiB is 1024*1024.
The size of the partition is not the problem (I set it by setting <size>52,428,800<\size> (1024 * 1024 * 50) in the flash.xml.in and creating an ext4 file with the exact same size.
The rest of the data between the flashed file and the eventual content of the partition is EQUAL but those 2 bytes that represent the journal attributes.
I couldn’t say for sure, but each time a filesystem with a journal is mounted there will be a journal entry for that mount. It is possibly the journal adding information about mount count.
First of all, I though that setting an ext4 to be read-only should prevent that.
I though of mounting changing the file-system, that’s why I read the data of a partition that supposedly was not mounted (the Jetson did not boot from).
Anyway, I tried to mount on my host the ext4 file I flash and it did not change (no change in the journal attributes).
Only if the UBoout mounting an ext4 results in a different behaviour than mounting in my host AND I’m wrong, meaning this partition was mounted (although I built my booting logic to do otherwise), that the change in the partition can be explained.
I don’t know for certain, but it is possible U-Boot or other early boot stages may have momentarily mounted the filesystem prior to Linux ever running. I’m kind of grasping at the moment, but that is a slight possibility. Considering it is the journal which changes, then it just seems logical that a mount operation was involved (even if the mount was not under Linux).
Thank you very much for all of the responses.
I think U-Boot can know boot squashfs so I think I switch to that and will use your advice and not use a journal-less ext4 file-system.
This is a good idea. Ext4 without a journal is an accident looking for a place to happen.