rootfsAB and test kernel/dtb

On JP 4.6.1, does extlinux.conf still work if rootfs AB is enabled?
I am on A, and updated /boot/extlinux/extlinux.conf file to add a new entry with a test kernel (LINUX) and test dtb (FDT). I made this new entry the default, but reboot still goes to old kernel.

If rootfsAB doesn’t support extlinux.conf, what is the alternative way to test new kernel/dtb? Is there a tool to copy the kernel/dtb to boot B rootfs?

ok, I am able to create the encrypted dtb file

sudo ./flash.sh --no-flash -k kernel-dtb jetson-xavier mmcblk0p1

and copy the Linux_for_Tegra/bootloader/kernel_tegra194-p2888-0001-p2822-0000_sigheader.dtb.encrypt to the board, and dd it to the kernel-dtb partition corresponding to my current boot partition. Reboot and I can see the new dtb tree.

The question remains why extlinux.conf doesn’t work in JP 4.6.1. I also tried on another board without rootfs AB enabled, booting alternative kernel in extlinux.conf also doesn’t work. I am pretty sure this used to work in earlier JP versions.

hello user100090,

when rootfs redundancy is enabled, the kernel and kernel-dtb partitions are belong to the user-space instead of the bootloader. they’re selected by their rootfs slots.
for example,
Rootfs Slot-A: kernel, kernel-dtb, and APP
Rootfs Slot-B: kernel_b, kernel-dtb_b, and APP_b

Are you saying the extlinux.conf is ignored if rootAB is enabled?

To confirm, I just flashed JP4.6.1 on my board without rootAB, the extlinux.conf still doesn’t work if I add another kernel image (/boot/test/Image) and set it as default boot.

If I flash JP 4.3 on my board, adding another entry in extlinux.conf for /boot/test/Image works fine. I also tried flashing JP4.6 and test extlinux.conf and that didn’t work either, like JP 4.6.1

BTW, in JP 4.6.1 with rootAB enabled, I see /dev/mmcblk0 has 40+ partitions. You only mentioned 6. What are the other ones for? We need to have a small partition to store a few text config files that can be shared between A and B. Out of those 40+ partitions, is there one that we can use to format as ext4 and put a few small files on?

Some updates. I have an NVME disk for data only (PCIe). Apparently there are some changes in how bootloader works. In 4.3, it reads /boot/extlinux/extlinux.conf from mmc, but in 4.6, it tries to read from my nvme data disk. Since I don’t have a /boot on my nvme data disk, bootloader doesn’t try to use /boot on mmc and directly boots using kernel/dtb partitions.

Regarding my other questions, I see Linux_for_Tegra/bootloader/flash.xml has below:

        <partition name="UDA" type="data">
            <allocation_policy> sequential </allocation_policy>
            <filesystem_type> basic </filesystem_type>
            <file_system_attribute> 0 </file_system_attribute>
            <allocation_attribute> 0x808 </allocation_attribute>
            <percent_reserved> 0 </percent_reserved>
            <align_boundary> 4096 </align_boundary>
            <description> **Required.** Automatically takes all remaining space on the device except space
              occupied by `secondary_gpt`. Allocation attribute must be set to 0x808. This
              partition may be mounted and used to store user data. </description>
        </partition>

Does that mean I can freely mkfs.ext4 on this partition and use it in /etc/fstab for both my rootA/B? I assume future flash.sh will still clear it, but other than that, will nvidia tools/scripts/system ever delete my data on UDA partition?

hello user100090,

please see-also Topic 210011. you should “see” the same data exists in both rootfs-A and rootfs-B if you saves some data to UDA partition, and mounts it automatically.

hello user100090,

we cannot reproduce issue by adding a new entry with a test kernel (LINUX) and test dtb (FDT) with r32.7.1 on Xavier. (this is test with/without enable rootfsAB).
could you please check the path definition, how about having test kernel image under /boot/ instead of /boot/test/..
thanks

You can ONLY reproduce if you have NVME drive (PCIe) that is mounted as data. My /boot directory is on the mmc, but in JP 4.6/4.6.1, bootloader tries to read /boot from NVME. Since my NVME is just data disk and doesn’t have /boot, bootloader fails to find /boot and decides to boot from kernel/dtb partitions. It never bother to try mmcblk0. In JP 4.3, bootloader reads mmcblk0 first and can find /boot.

I see this other user reported the same issue.

hello user100090,

FYI, we’re able to reproduce this.
we’ve arrange resources for investigation.

Thanks for the investigation!
BTW, I tried to modify Linux_for_Tegra/bootloader/cbo.dts and change the line:
boot-order = "sd", "usb", "nvme", "emmc", "net";

to

boot-order = "sd", "usb", "emmc", "nvme", "net";

to swap the order of emmc and nvme. After re-flash the board, it won’t boot up anymore. Reverting the change and reflashing, the board boots again.

I saw the new 32.7.2 just released, and tried it. It is even worse now. The bootloader is neither reading the /boot/extlinux/extlinux.conf from nvme, nor emmc. In the last release, as mentioned above, bootloader at least reads the one from nvme.

Could you check the boot log from UART to check it.

I need to capture and post later, but what I did yesterday was to put a long TIMEOUT in the /boot/extlinux/extlinux.conf on emmc and nvme, I watched the booting fly through without any pause. I also added something in APPEND, but didn’t see it in /proc/cmdline after system boots up.

Here are the lines from uart logs:

[0005.376] I> Look for boot partition
[0005.377] I> Fallback: assuming 0th partition is boot partition
[0005.383] I> Detect filesystem
[0005.410] I> Loading extlinux.conf …
[0005.410] I> Loading extlinux.conf binary from rootfs …
[0005.410] I> rootfs path: /sdmmc_user/boot/extlinux/extlinux.conf
[0005.447] I> ext4_read_data_from_extent:298: Total file read should not be larger than file stat size
[0005.448] E> file /sdmmc_user/boot/extlinux/extlinux.conf read failed!!
[0005.448] W> Failed to load extlinux.conf binary from rootfs (err=202113305)
[0005.449] E> Failed to find/load /boot/extlinux/extlinux.conf
[0005.450] I> Loading kernel …
[0005.453] I> No kernel binary path
[0005.456] I> Continue to load from partition …

Based on this security bulletin, 32.7.2 is a security patch with cboot fixes. It seems the cboot fixes accidentally break the read of extlinux.conf, as shown in my uart logs.

Has anybody else tried 32.7.2 and noticed extlinux.conf is no longer being read, with the same error in the uart log as I mentioned above?

Hi,

If you find some new issue, please open a new topic. I am not sure what is happened here. The first one was on jp4.6.1 and now you have something else in 32.7.2.

The issue has nothing to do with 32.6.1.

Originally the issue was reported for 32.7.1 (JP4.6.1), and was told to be fixed in next release. When the next release (32.7.2) was released, I was just testing the fix and report back the finding. That’s why you see them in the same thread.

There is no fix for anything based on what I saw. At least for this topic, we don’t have any fix.

Better sharing new log again to clarify the issue.

ok, opened a new thread.