Hi,
I’m using jetpack6.0.
We have encountered several cases where the OTA upgrade completed successfully but the slot switch failed. When the issue occurs, manually switching the slot also fails. The serial console shows :
However, we do not use an SD card in our hardware design.
From the logs, we can see that the system attempts to boot from slot 1 on the first boot, but it enters a busy spin state and then falls back to slot 0 to continue booting.
The serial logs have been uploaded. Could you please help analyze the root cause of this issue?
*** Please note that this reply is generated by LLM automatically ***
Based on the information provided, it appears that you are experiencing issues with the OTA (Over-the-Air) upgrade process on your Jetson AGX Orin device, specifically with the slot switch failing after a successful upgrade.
The error message you provided suggests that the system is trying to boot from the SD card, but since you mentioned that you don’t use an SD card in your hardware design, this could be a red herring.
To analyze the root cause of this issue, let’s break down the possible causes:
Bootloader configuration: The bootloader might be configured to boot from the SD card by default. You can try checking the bootloader configuration to see if it’s set to boot from the SD card. If so, you can try changing it to boot from the eMMC or NVMe storage instead.
Partition layout: The partition layout might be incorrect or corrupted, causing the system to fail to boot from the correct partition. You can try checking the partition layout using tools like fdisk or parted to see if it’s correct.
Slot switch failure: The slot switch failure could be due to a hardware issue or a software configuration problem. You can try manually switching the slot to see if it works, and if not, you can try checking the hardware connections to see if there’s a problem.
Image-based OTA update: Since you’re using JetPack 6.0, you might be using the image-based OTA update feature. This feature writes pre-generated images to the partitions, which could potentially cause issues if the partition layout is incorrect or corrupted.
Comparing your issue to the ones listed in the provided context, it seems that your issue is similar to the one described in the “Issue that OTA works on some Jetsons but not others” topic. This topic mentions that the OTA update works on some Jetson devices but not others, and it’s related to the UEFI firmware.
To troubleshoot this issue further, you can try the following:
Check the bootloader configuration to see if it’s set to boot from the SD card.
Verify the partition layout to see if it’s correct.
Try manually switching the slot to see if it works.
Check the hardware connections to see if there’s a problem.
Try updating the UEFI firmware to the latest version.
If none of these troubleshooting steps resolve the issue, you may want to try seeking further assistance from NVIDIA support or the Jetson community forums.
*** Please give the thumbs up if you get answers you like. Or provide feedback to help us improve the answer. ***
Additional information: The same OTA package successfully upgrades and boots from partition B on some AGX devices without encountering this issue. Currently, several devices have exhibited this problem.
may I know your base/target version for running OTA update. please also share the commands you’ve executed for reference.
besides.. since it’s related to some AGX devices, please share the board info for comparisons.
The base/target version is both 36.3.
The command to generate OTA package: sudo -E ROOTFS_AB=1 ./tools/ota_tools/version_upgrade/l4t_generate_ota_package.sh --external-device nvme0n1 -S 45GiB jetson-agx-orin-devkit R36-3
The command to start OTA sudo ./nv_ota_start.sh /data/ota/ota_payload_package.tar.gz
The AGX devices that upgrade successfully and those with upgrade issues are identical in both hardware design and software. We are not using an SD card in the hardware
A comparison of boot-failure device logs against successful boot logs reveals SDMMC-related entries during failures. Furthermore, even after a manual switch from partition A to partition B and subsequent reboot, the system remains unable to boot, exhibiting the same error logs. Is it possible that an issue with the QSPI firmware upgrade is causing the system to attempt SD card enumeration?
let me have confirmation,
since it’s the command to create OTA payload with ROOTFS_AB=1.
did you running OTA update with the targets which has ROOTFS_AB enabled?
besides.. is it Jetson AGX Orin developer kit? or, you’re working with customize board?
did you running OTA update with the targets which has ROOTFS_AB enabled?
Yes, the targets have ROOTFS_AB enabled.
In the log, we can see it’s trying to boot slot 1 first but failed at busy spin. Then after several seconds, it reboot automatically on slot 0 and boot successfully. besides.. is it Jetson AGX Orin developer kit? or, you’re working with customize board?
I’m working on a customize board.
Comparing the first boot after USB flashing and OTA upgrade, the boot log for USB flashing does not show “Secondary storage device: SDMMC_USER”, while the boot log after OTA does. Is this due to differences in MB2 Configuration?(The USB flashing environment and the OTA package creation environment are the same.)
Is SDMMC_USER related to SD card or eMMC? If our board does not use an SD card, can we disable SDMMC? Is the MB2 code open source, and how can we obtain it?
umm.. you should not create OTA payload with jetson-agx-orin-devkit board configuration.
it’s suggest to update l4t_generate_ota_package.sh and ota_board_specs.conf for customize board to create OTA payloads.
here’s an example,
please update CUSTOMIZE_JETSON, and customize-jetson to your board name accordingly.
please refer to below,
--
scripts/ota-scripts/l4t_generate_ota_package.sh
scripts/ota-scripts/ota_board_specs.conf
diff --git a/scripts/ota-scripts/l4t_generate_ota_package.sh b/scripts/ota-scripts/l4t_generate_ota_package.sh
index ff920550..e4819b29 100755
--- a/scripts/ota-scripts/l4t_generate_ota_package.sh
+++ b/scripts/ota-scripts/l4t_generate_ota_package.sh
@@ -217,6 +217,8 @@ function construct_board_spec_name()
'jetson-agx-orin-devkit:0005'
'jetson-orin-nano-devkit:0001'
'jetson-orin-nano-devkit:0003'
'jetson-orin-nano-devkit:0004'
'jetson-orin-nano-devkit:0005'
'jetson-agx-orin-devkit-industrial:0008'
+ 'customize-jetson:0001' <=== must be the correct board sku.
@@ -1097,6 +1099,7 @@ function construct_board_spec_entry()
# For other devices, only keep the board spec entry for
# internal device.
if [[ "${item}" =~ jetson-orin-nano-devkit ]] \
+ || [[ "${item}" =~ customize-jetson ]] \
|| [[ "${item}" =~ internal ]]; then
echo "'${item}'" >>"${tmp_board_spec_file}"
fi
diff --git a/scripts/ota-scripts/ota_board_specs.conf b/scripts/ota-scripts/ota_board_specs.conf
index e3226326..a8d370c0 100644
--- a/scripts/ota-scripts/ota_board_specs.conf
+++ b/scripts/ota-scripts/ota_board_specs.conf
@@ -79,6 +79,15 @@
# customize-jetson
+ customize_jetson_ota_emmc_r36_spec=(
+ # customize-jetson
+
'boardid=3767;fab=300;boardsku=0001;boardrev=;fuselevel_s=1;chiprev=;chipsku=00:00:00:D4;board=customize-jetson;rootdev=nvme0n1p1;bup_type=bl;signed_img_dir=images-R36-ToT'
+)
+ CUSTOMIZE_JETSON_R36_3_ALIAS="customize_jetson_ota_emmc_r36_spec"
+ CUSTOMIZE_JETSON_R36_4_ALIAS="customize_jetson_ota_emmc_r36_spec"
@@ -101,4 +110,5 @@ T23X_DEVICES=(
'JETSON_AGX_ORIN_DEVKIT'
'JETSON_AGX_ORIN_DEVKIT_INDUSTRIAL'
'JETSON_ORIN_NANO_DEVKIT'
+ 'CUSTOMIZE_JETSON'
)
--
Can we confirm that this issue is caused by using the jetson‑agx‑orin‑devkit board configuration?
Could you identify the specific component or configuration that triggers the problem and explain the underlying mechanism?
Why are some devices able to boot successfully, even though they display SDMMC log messages, while others fail to boot, despite all of them using the same OTA package?
I replaced mb2_t234.bin in R36.3 with the mb2_t234.bin from R36.4.4, and the issue no longer reproduces.
May I only replace mb2_t234.bin from R36.4.4 into R36.3?
Will there be any compatibility/matching issues, and are there any other potential risks?
there’re some mb2 bug fixes regarding to Jetson security, such as bootloader secureboot and disk encryption.
it’s suggest you’ve OTA update moving forward to the latest JP-6 if you’re going to work with Jetson security.
