Flash fails Orin AGX at 99%

hbhat1 · November 7, 2024, 9:01pm

Flashing fails at 99% showing write checkpoint and then stalls leading to failure. Here are the log files
SDKM_logs_2024-11-07_20-40-47.zip (424.1 KB)
Kindly help debug the issue.

DavidDDD · November 8, 2024, 2:49am

Hi hbhat1,
From log

Info: mount.nfs: access denied by server while mounting

Try:

Disable the firewall setting on the host pc.
And refer to the doc to check whether dev kit in the recovery mode.
Run sdk again.

If the issue persists, try to dump the serial console log from device with this
Jetson Nano & NX Style – Serial Debug Console
Thanks

hbhat1 · November 8, 2024, 5:01pm

Hi David,
Thanks for your reply. I have already disabled my firewall by running sudo ufw disable. For putting the jetson on Recovery mode I long pressed the middle button+power button, plug in USB-C power and then connected to my host ubuntu computer. Upon checking lsusb I could see the Nvidia Corp. stuff. Let me try reproducing the serial console logs.

hbhat1 · November 8, 2024, 6:37pm

I can confirm over here that it is running in recovery mode based on that document. Let me know if you need more logs

hbhat1 · November 8, 2024, 7:32pm

here is the output after checking the mount:

(base) hive@integration-assembly:~$ ls -a /home/hive/nvidia/nvidia_sdk/JetPack_5.1.3_Linux_JETSON_AGX_ORIN_TARGETS/Linux_for_Tegra/
.
..
apply_binaries.sh
bootloader
build_l4t_bup.sh
clara-agx-xavier-devkit.conf
e3900-0000+p2888-0004-b00.conf
flash.sh
generate_capsule
holoscan-devkit.conf
igx-orin-devkit.conf
jetson_additional_board_spec.cfg
jetson-agx-orin-devkit-as-jao-32gb.conf
jetson-agx-orin-devkit-as-nano4gb.conf
jetson-agx-orin-devkit-as-nano8gb.conf
jetson-agx-orin-devkit-as-nx-16gb.conf
jetson-agx-orin-devkit-as-nx-8gb.conf
jetson-agx-orin-devkit.conf
jetson-agx-orin-devkit-industrial.conf
jetson-agx-orin-devkit-industrial-maxn.conf
jetson-agx-orin-devkit-maxn.conf
jetson-agx-xavier-devkit.conf
jetson-agx-xavier-ind-noecc.conf
jetson-agx-xavier-industrial.conf
jetson-agx-xavier-industrial-mxn.conf
jetson_board_spec.cfg
jetson-orin-nano-devkit.conf
jetson-orin-nano-devkit-nvme.conf
jetson-xavier.conf
jetson-xavier-maxn.conf
jetson-xavier-nx-devkit.conf
jetson-xavier-nx-devkit-emmc.conf
jetson-xavier-nx-devkit-qspi.conf
jetson-xavier-slvs-ec.conf
kernel
l4t_generate_soc_bup.sh
l4t_sign_image.sh
l4t_uefi_sign_image.sh
nvautoflash.sh
nvmassfusegen.sh
nvsdkmanager_flash.sh
nv_tegra
nv_tools
odmfuse.func
odmfuseread.sh
odmfuse.sh
p2822-0000+p2888-0001.conf
p2822-0000+p2888-0004.conf
p2822-0000+p2888-0008.conf
p2822-0000+p2888-0008-maxn.conf
p2822-0000+p2888-0008-noecc.conf
p2972-0000.conf.common
p2972-0000-devkit-maxn.conf
p2972-0000-devkit-slvs-ec.conf
p2972.conf
p3449-0000+p3668-0000-qspi-sd.conf
p3449-0000+p3668-0001-qspi-emmc.conf
p3509-0000+p3668-0000-qspi.conf
p3509-0000+p3668-0000-qspi-sd.conf
p3509-0000+p3668-0001-qspi-emmc.conf
p3509-a02+p3767-0000.conf
p3509-a02+p3767-0000-maxn.conf
p3668.conf.common
p3701.conf.common
p3711-0000+p3701-0002.conf
p3737-0000+p3701-0000-as-p3701-0004.conf
p3737-0000+p3701-0000-as-p3767-0000.conf
p3737-0000+p3701-0000-as-p3767-0001.conf
p3737-0000+p3701-0000-as-p3767-0003.conf
p3737-0000+p3701-0000-as-p3767-0004.conf
p3737-0000+p3701-0000.conf
p3737-0000+p3701-0000-maxn.conf
p3737-0000+p3701-0008.conf
p3737-0000+p3701-0008-maxn.conf
p3740-0002-b00+p3701-0002.conf
p3740-0002-b01+p3701-0002.conf
p3740-0002-b01+p3701-0002-dgpu-efifb.conf
p3740-0002-c01+p3701-0008.conf
p3740-0002-c01+p3701-0008-dgpu-efifb.conf
p3767.conf.common
p3768-0000+p3767-0000.conf
p3768-0000+p3767-0000-maxn.conf
p3768-0000+p3767-0000-nvme.conf
README_Autoflash.txt
rootfs
source
source_sync.sh
Tegra_Software_License_Agreement-Tegra-Linux.txt
tools
(base) hive@integration-assembly:~$ showmount -e
Export list for integration-assembly:
/srv/nfs/jetson *
(base) hive@integration-assembly:~$ sudo systemctl status nfs-kernel-server
● nfs-server.service - NFS server and services
     Loaded: loaded (/lib/systemd/system/nfs-server.service; enabled; vendor pr>
     Active: active (exited) since Fri 2024-11-08 19:15:38 GMT; 13min ago
    Process: 44750 ExecStartPre=/usr/sbin/exportfs -r (code=exited, status=0/SU>
    Process: 44751 ExecStart=/usr/sbin/rpc.nfsd $RPCNFSDARGS (code=exited, stat>
   Main PID: 44751 (code=exited, status=0/SUCCESS)

Nov 08 19:15:37 integration-assembly systemd[1]: Starting NFS server and servic>
Nov 08 19:15:38 integration-assembly systemd[1]: Finished NFS server and servic>

(base) hive@integration-assembly:~$ sudo blkid
/dev/nvme0n1p2: UUID="2fa8fee2-a328-404b-b1a7-e6ddc187f632" TYPE="ext4" PARTUUID="2ddd39b9-fe77-4cb0-a8b3-909b6f4fae37"
/dev/loop0: TYPE="squashfs"
/dev/loop1: TYPE="squashfs"
/dev/loop2: TYPE="squashfs"
/dev/loop3: TYPE="squashfs"
/dev/loop4: TYPE="squashfs"
/dev/loop5: TYPE="squashfs"
/dev/loop6: TYPE="squashfs"
/dev/loop7: TYPE="squashfs"
/dev/nvme0n1p1: UUID="7695-CC9C" TYPE="vfat" PARTLABEL="EFI System Partition" PARTUUID="d3af6295-7d89-4ff4-9cf6-ccec1125fa21"
/dev/loop9: TYPE="squashfs"
/dev/loop8: TYPE="squashfs"
/dev/loop10: TYPE="squashfs"
/dev/loop13: TYPE="squashfs"
/dev/loop12: TYPE="squashfs"
/dev/loop11: TYPE="squashfs"
/dev/loop14: TYPE="squashfs"
/dev/loop16: TYPE="squashfs"
/dev/loop17: TYPE="squashfs"
/dev/loop23: TYPE="squashfs"
/dev/loop22: TYPE="squashfs"
/dev/loop20: TYPE="squashfs"
/dev/loop21: TYPE="squashfs"
/dev/loop19: TYPE="squashfs"
/dev/loop25: TYPE="squashfs"
/dev/loop24: TYPE="squashfs"
/dev/loop15: TYPE="squashfs"
/dev/loop18: TYPE="squashfs"
/dev/loop26: TYPE="squashfs"
/dev/loop27: TYPE="squashfs"
(base) hive@integration-assembly:~$ df -h
Filesystem      Size  Used Avail Use% Mounted on
udev             16G     0   16G   0% /dev
tmpfs           3.2G  2.3M  3.2G   1% /run
/dev/nvme0n1p2  468G  202G  243G  46% /
tmpfs            16G     0   16G   0% /dev/shm
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs            16G     0   16G   0% /sys/fs/cgroup
/dev/loop0      128K  128K     0 100% /snap/bare/5
/dev/loop1      105M  105M     0 100% /snap/core/17200
/dev/loop3       64M   64M     0 100% /snap/core20/2318
/dev/loop4       75M   75M     0 100% /snap/core22/1439
/dev/loop6       56M   56M     0 100% /snap/core18/2829
/dev/loop7      219M  219M     0 100% /snap/gnome-3-34-1804/93
/dev/loop8      576M  576M     0 100% /snap/obs-studio/1302
/dev/loop13     576M  576M     0 100% /snap/obs-studio/1299
/dev/loop10     131M  131M     0 100% /snap/docker/2915
/dev/loop12     183M  183M     0 100% /snap/postman/287
/dev/loop11     486M  486M     0 100% /snap/gnome-42-2204/126
/dev/loop14      41M   41M     0 100% /snap/snapd/20092
/dev/loop20      39M   39M     0 100% /snap/snapd/21759
/dev/loop21      92M   92M     0 100% /snap/gtk-common-themes/1535
/dev/loop23     132M  132M     0 100% /snap/docker/2932
/dev/loop25      13M   13M     0 100% /snap/snap-store/1113
/dev/loop16     174M  174M     0 100% /snap/postman/266
/dev/loop19     347M  347M     0 100% /snap/gnome-3-38-2004/119
/dev/loop17     506M  506M     0 100% /snap/gnome-42-2204/176
/dev/loop22     350M  350M     0 100% /snap/gnome-3-38-2004/143
/dev/loop15      82M   82M     0 100% /snap/gtk-common-themes/1534
/dev/loop24      13M   13M     0 100% /snap/snap-store/959
/dev/loop18     219M  219M     0 100% /snap/gnome-3-34-1804/77
/dev/nvme0n1p1  511M  5.3M  506M   2% /boot/efi
tmpfs           3.2G   88K  3.2G   1% /run/user/1002
/dev/loop26      56M   56M     0 100% /snap/core18/2846
/dev/loop27      64M   64M     0 100% /snap/core20/2434
/dev/loop9       74M   74M     0 100% /snap/core22/1663
/dev/loop2       67M   67M     0 100% /snap/core24/609
/dev/loop5       10M   10M     0 100% /snap/nmap/3582

hbhat1 · November 8, 2024, 7:39pm

SDKM_logs_JetPack_5.1.3_Linux_for_Jetson_AGX_Orin_[32GB_developer_kit_version]_2024-11-08_19-21-17.zip (320.4 KB)

DavidDDD · November 11, 2024, 10:45am

Hi hbhat1,
To gather more information, please answer the following questions:

Is your host machine a virtual machine or a physical device?
Is your device a dev kit or a custom carrier board?

Additionally, please capture and share the serial console log during the flashing process. Also, provide the SDK export log as well.

hbhat1 · November 12, 2024, 4:02pm

Hi David,

I’m using Ubuntu 20.04 physical host
The board is a devkit not a custom carrier board.
I will try to capture serial console logs. The above logs I provided are logs which it generates after flash fails. Do you have other process to export the SDK logs?

hbhat1 · November 12, 2024, 4:41pm

Cannot seem to recognize the minicom mentioned in the article for collecting logs.

hive@integration~$ minicom -D /dev/ttyACM0 -8 -b 115200
minicom: cannot open /dev/ttyACM0: No such file or directory
hive@integration:~$ sudo adduser $USER dialout
The user `hive' is already a member of `dialout'.

The SDK manager detects the AGX board but not seen by minicom. Could you help me with understanding why is this issue?

hbhat1 · November 12, 2024, 5:08pm

PFA, the SDK manager logs requested:
SDKM_logs_2024-11-12_17-07-12.zip (533.4 KB)
Minicom logs are not available as minicom cannot detect the usb connection

DavidDDD · November 13, 2024, 6:25am

Hi,
With the log provided,

16:57:06.974 - Info: [   0.3384 ] Sending bct_br
16:57:12.142 - Info: [   0.3857 ] ERROR: might be timeout in USB write.

Please try the following steps:

sudo -s
echo -1 > /sys/module/usbcore/parameters/autosuspend

And run the sdk manager again

Regarding Minicom, try running dmesg command after plugging in the USB to check for any related information. Alternatively, try using a different cable to connect.

Thanks

hbhat1 · November 16, 2024, 1:31am

Hi,
with dmesg, I can see nvidia detected but minicom cannot get any ttyUSB0 or ttyACM0. Is there any alternative to collecting serial console logs?

WayneWWW · November 16, 2024, 4:57pm

You have to connect the micro usb port on the Orin AGX devkit so that it will give you a ttyXXX on host PC to get the log.

Topic		Replies	Views
Flash Jetpack 5.1.2 stops at 99.5% Jetson Orin NX board-design , flash	17	341	April 30, 2024
Flashing linux on nvme ssd fails at mount of NFS server Jetson AGX Orin reflash	9	198	July 28, 2024
Jetson Orin AGX won't finish boot after upgrade to Jetpack 5.1.2 Jetson AGX Orin boot	21	899	October 5, 2023
Flashing of Nvidia jetson orin nx board Jetson Orin NX reflash	72	575	July 3, 2024
SDKManager Jetson Orin AGX 32GB flash failed Jetson AGX Orin cudnn	6	690	October 25, 2023
Cannot flash jetson orin nano with any method Jetson Orin Nano reflash	11	696	April 19, 2024
Jetson Orin Nano Developmen Kit - NVMe flash fails in every possible way Jetson Orin Nano boot , reflash , kernel , ubuntu	37	4081	July 27, 2023
Orin has black screen after jetpack upgrade, fan 100% and "Reading board information failed" Jetson AGX Orin boot , reflash	7	861	October 25, 2023
Flash Failed - New SSD Drive Jetson AGX Orin reflash	3	559	October 20, 2023
Sdkmanager flash error Jetson AGX Orin sdkm , reflash , nvbugs	29	6077	May 16, 2023

Flash fails Orin AGX at 99%

Related topics