Corrupted ESP after OTA update on JP 5.1.5

Robotics & Edge Computing Jetson Systems Jetson AGX Orin
, , ,
1

Hello,

I recently ran into a issue where my ESP was corrupted/erased after an OTA update, causing my unit to fail and boot into the UEFI shell.

I’ve tracked down the issue to the update_l4t_launcher() function in nv_ota_common.func, where the ESP is mounted and the BOOTAA64.efi is overwritten.

Has this issue been encountered before? I’m wondering if there are any ways to add more safety to this function to ensure safe OTA updates.

3

Hi jmelucci,

Are you using the devkit or custom board for AGX Orin?

How did you confirm the boot issue is caused from ESP?
Do you recover the board after updating BOOTAA64.efi in ESP manually?

Do you have any customization there?

4

We are using a custom carrier board. We confirmed that the boot issue is caused from ESP via the UART debug logs, where we boot directly in the UEFI shell and UEFI doesn’t know how to proceed. Within this shell, we were able to inspect the contents of the ESP and saw that it was empty with no BOOTAA64.efi. Both rootfs partitions looked fine.

`FS3:> ls
Directory of: FS3:
0 File(s) 0 bytes
0 Dir(s)

FS3:> vol
Volume (rw)
66059264 bytes total disk space
66058752 bytes available on disk
512 bytes in each allocation unit`

5

Do you know what operation(the exact step in OTA?) in your case causes BOOTAA64.efi missing in your case?
Please provide the full logs when you perform OTA update. (including OTA payload generation from host and serial console log from the device)

6

Hi Kevin,

These lines in update_l4t_launcher() within nv_ota_common.func seem to be the culprit for the ESP corruption

	mkdir "${esp_mnt}"
	mount "${esp_partition}" "${esp_mnt}"
	if [ -f "${esp_l4t_launcher}" ]; then
		ota_log "Updating L4T Launcher at ${esp_l4t_launcher}"
		cp "${l4t_launcher}" "${esp_l4t_launcher}"
	fi
	umount "${esp_mnt}"
	rm -rf "${esp_mnt}"
	return 0

I’m currently experimenting with adding a sync before unmounting to ensure proper flushing to the partition. I’m also trying to add sha1sum validation to BOOTAA64.efi to make sure that this file is not being corrupted during our OTA deployment process.

One question I have is if I can skip this update_l4t_launcher() step altogether during an OTA update. Is the ESP necessary to update every OTA deployment? I am not jumping JetPack versions and sticking with 5.1.5.

Here are the logs from the OTA update. They seem normal and don’t throw any errors. I’ve also attached logs from the serial console that show booting into the UEFI shell.

serial_logs.log (108.4 KB)

ota_logs.txt (14.2 KB)

7

Thanks for the details. This looks like an ESP / L4TLauncher update issue, not a rootfs A/B issue.

If you are staying on the same JP5.1.5 release, it is reasonable to test skipping update_l4t_launcher() during OTA.

For safety, please also add sync, verify BOOTAA64.efi after copy, and log the exact ESP partition being mounted on your custom carrier board.