I totally agree with you. In a very basic “student project” setup that workflow would be okay.
The extlinux probably will not work as we use the A/B feature. Afaik the A/B selection is done in UEFI, and not in extlinux.
My point though is that I need a flash procedure which works for production.
I want the flash tools to flash what I asked it to flash.
If I tell the scripts to flash QSPI + NVMe, I’d like to end up with a system which boots from NVMe and not from emmc, just because it already was there before. I can’t ask our production team to do the UEFI stuff manually. We assemble the systems with robots, there is no way we can add a manual step in there which the scripts should perform.
It can’t be that hard to modify the uefi partition before flash to be properly configured?
Maybe I have to compile it with some other options?
I’ve not experimented with massflash, so I am not very useful there. You are probably correct that A/B changes things, but when UEFI determines if a partition is bootable, then it probably also uses extlinux.conf to some extent. Do you have a working setup (regardless of whether it was mass flashed or manually configured) which you can post the extlinux.conf from all related partitions (eMMC and NVMe)? Also, post the output of “/proc/cmdline” in the working case, along with “lsblk -f”. I don’t know if I can spot anything, but it is simple to look at with that information, plus a serial console boot log of the working system. The goal is to see what is used from where, and why it transfers control from one step to another.
Additionally, when you flashed this working example, the target of a command line flash would have a “.conf” file. For example, if I flashed to target “jetson-xavier”, then this would have used “jetson-xavier.conf” for configuration. Whatever target you used, post a copy of that conf file as well.
FYI, flashing a Jetson and telling it NVMe versus eMMC is not about flashing the NVMe…it is about configuring for search.
I am glad for your help, and I think you’re right there is the A/B mechanism which selects if the extlinux from APP or APP_b is used. Afterwards the system continues with the extlinux config.
Extlinux:
TIMEOUT 30
DEFAULT primary
MENU TITLE L4T boot options
LABEL primary
MENU LABEL primary kernel
LINUX /boot/Image
FDT /boot/dtb/kernel_tegra194-p3668-0000-p3509-0000.dtb
INITRD /boot/initrd
APPEND ${cbootargs} root=PARTUUID=c75e9baf-ec9a-49d9-acf4-9cb0be129e74 rw rootwait rootfstype=ext4 console=ttyTCU0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0
Anyway, I think getting your head into our use case might be a bit too complicated. Since Nvidia does not support A/B OTA Updates I’ve created a workflow myself. We create a tar file to be copied to the unused A/B partition. Afterwards the extlinux.conf is modified to match the UUID of the next rootfs, the bootcontrl is changed to boot from the other slot. Like this uefi uses slot A/B, reads the extlinux and boots the correct UUID. When you flash the system with ROOTFS_AB the l4t scripts automatically modify the extlinux.conf on each of the system images to match their future UUID.
But the issue is not with here. It’s the default boot order.
I’ve done some research. It looks like no matter if you use flash.sh or initrd_flash.sh the uefi_jetson.bin is always the same. It has not updated in my system since I unpacked the jetpack. Could it be that the default boot order is set here and cannot be changed unless I modify the source here? GitHub - NVIDIA/edk2-nvidia: NVIDIA EDK2 platform support
I don’t know enough to work in detail, but since this is scheduled to work with device tree overlay in the next release, I suspect you are on some deadline to make this work with the current release.
Is that extlinux.conf the same on all partitions, including A/B/eMMC? I didn’t see “lsblk -f” on the running Jetson, but it would be interesting to see UUID information. One of the reasons I mention “/proc/cmdline” is to see if the PARTUUID relates to that in bothextlinux.confand what the kernel actually sees (cmdline in part inherits “${cbootargs}” which can be created and edited or deleted before passing to the Linux kernel…in this case “cmdline” reflects that there was no “${cbootargs}”, at least not at the time when the Linux kernel was loaded to overwrite boot software).
I am curious because you said the uefi_jetson.bin is always the same, and this leads to two questions: Was the binary content actually not changed, or was the binary content actually changed, but not behaving as you wish (in which case it was something the binary inherited determining the issue rather than the binary itself)? By what means did you determine uefi_jetson.bin did not change? I don’t know which partition it would be in, but in theory you could use dd to copy the partition, cut off any signature, and search for strings. That string search might indicate if any edits actually did take place. I suspect though that one would have to change the code to not use inherited environment if it is a case of inheritance changing boot order to the wrong order (the next release changes of .dtbo would be to provide an alternate environment inheritance of boot device search).
After adding this, build the dtbo out, put it to Linux_for_Tegra/bootloader and Linux_for_Tegra/kernel/dtb.
Also, need to add dtbo name to your board config file.
Checking your solution, I wanna make sure I get this right
I expected that the uefi binary would have to be replaced. But it seems like these dtbo are responsible for the boot order.
In the images directory I get this: BootOrderNvme_Jetson_RELEASE.dtbo, I copy ONLY this file? Does the name matter or will the scripts in 35.1 automatically pick this up? Do I need to copy the UEFI too to make sure that everything works?
For my solution I’d like to be able to flash emmc and nvme depending on a variable. Can I simply copy the required file or delete it before flashing?
my apologies, I did overread a part of your answer. My third question was nonsense of course.
Adding the DTBO to the config file of all “NMVe” configs is sufficient. Thus no variable is required on my side.
I tested flashing only the QSPI with the unmodified NVMe boot order overlay. That is sufficient to make the system boot from NVMe by default.
Thanks a lot for the support!
Now as this all works I’ve got two questions:
L4TRootfsInfo_Jetson_RELEASE.dtbo and L4TRootfsABInfo_Jetson_RELEASE.dtbo also already exist in the current bootloader directory of the L4T. Do I need to copy these or will the files from the R35.1 be compatible with the UEFI from 35.1-updates?
will this dtbo be added automatically when flashing nvme with 5.0.2 or will we have to compile the dtbo by hand even after the update?
Obviously it does not work without copying the AB Info files too? I tested without the files copied and the change to rootFS B gets ignored by the system? Will test tomorrow if it works without copying. A confirmation would be nice anyway.