Orin NX 16 GB flashing

[alex.imp]

Hello,

We are trying to flash Orin NX 16GB on a custom carrier board which is reference of Jetson Xavier NX reference carrier board (P3509-0000).

I have followed this guide: Quick Start — Jetson Linux Developer Guide documentation

Here carrier board has no-eMMC, the OS is to be flashed on SSD 2TB on PCIE0
So following this guide: Jetson Orin NX Series — Jetson Linux Developer Guide documentation
I have also modified the:
Linux_for_Tegra/bootloader/t186ref/BCT/tegra234-mb2-bct-misc-p3767-0000.dts to have
cvb_eeprom_read_size = <0x0>

$ sudo ./tools/kernel_flash/l4t_initrd_flash.sh --external-device nvme0n1p1
-c tools/kernel_flash/flash_l4t_external.xml -p “-c bootloader/t186ref/cfg/flash_t234_qspi.xml”
–showlogs --network usb0 p3509-a02+p3767-0000 internal

This flashing successfully completes and i am able to boot-up the SoM. But the dmesg keeps flooding with -

[ 1300.303279] nvme 0004:01:00.0: PCIe Bus Error: severity=Corrected, type=Physical Layer, (Receiver ID)
[ 1300.312829] nvme 0004:01:00.0: device [144d:a80a] error status/mask=00000001/0000e000
[ 1300.321143] nvme 0004:01:00.0: [ 0] RxErr
[ 1300.327143] pcieport 0004:00:00.0: AER: Corrected error received: 0004:01:00.0
[ 1300.346793] pcieport 0004:00:00.0: AER: Corrected error received: 0004:01:00.0
[ 1300.346822] nvme 0004:01:00.0: PCIe Bus Error: severity=Corrected, type=Physical Layer, (Receiver ID)
[ 1300.356340] nvme 0004:01:00.0: device [144d:a80a] error status/mask=00000001/0000e000
[ 1300.364622] nvme 0004:01:00.0: [ 0] RxErr
[ 1300.448810] pcieport 0004:00:00.0: AER: Corrected error received: 0004:01:00.0

For reference - These are the PCIE connection on the custom carrier board.

I have also tried the flashing with following changes but have same results:

$ sudo ./tools/kernel_flash/l4t_initrd_flash.sh --external-device nvme0n1p1
-c tools/kernel_flash/flash_l4t_nvme.xml -p “-c bootloader/t186ref/cfg/flash_t234_qspi.xml”
–showlogs --network usb0 p3509-a02+p3767-0000 internal

Any suggestion on this issue? Do i need to share more information on the issue please do let me know.

[WayneWWW]

Does the same nvme work when it is tested on p3509 carrier board?

[alex.imp]

I actually do not have the carrier board to validate that. But the SSD is working fine and I am able to ssh into the linux.

Here is some more information on the partitions created on SSD.

$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
loop0 7:0 0 16M 1 loop
zram0 251:0 0 1.8G 0 disk [SWAP]
zram1 251:1 0 1.8G 0 disk [SWAP]
zram2 251:2 0 1.8G 0 disk [SWAP]
zram3 251:3 0 1.8G 0 disk [SWAP]
nvme0n1 259:0 0 1.8T 0 disk
├─nvme0n1p1 259:1 0 1.8T 0 part /
├─nvme0n1p2 259:2 0 64M 0 part
├─nvme0n1p3 259:3 0 448K 0 part
├─nvme0n1p4 259:4 0 32M 0 part
├─nvme0n1p5 259:5 0 64M 0 part
├─nvme0n1p6 259:6 0 448K 0 part
├─nvme0n1p7 259:7 0 32M 0 part
├─nvme0n1p8 259:8 0 80M 0 part
├─nvme0n1p9 259:9 0 512K 0 part
├─nvme0n1p10 259:10 0 300M 0 part
├─nvme0n1p11 259:11 0 64M 0 part
├─nvme0n1p12 259:12 0 80M 0 part
├─nvme0n1p13 259:13 0 512K 0 part
└─nvme0n1p14 259:14 0 64M 0 part

To not that after a while the device gives following message and I loose the ssh connection for a 15-20 seconds.
client_loop: send disconnect: Connection reset

[WayneWWW]

Could you try other kind of SSD too?

[alex.imp]

I will be testing SSD mentioned in the NVIDAI documentation but it will be delivered after a day or two.

So I am seeking other area which can be explore for this issue.

I have tried to update the system, now the message dump has stopped it seems but still i think something is wrong with installation.

dmesg.txt (64.5 KB)

[WayneWWW]

The BSP seems old too. You can also try to upgrade to 35.4.1 first.

[alex.imp]

I tried the mentioned BSP - with the EMMC changes in the bootloader directory. Similar process mentioned above. The flashing completed successfully, L4T-README opens up at first but I am not getting access to the SSH on /ttyACM0. Post this the L4T-README goes away.

On the same note according to the documentation 35.4.1 has support for Orin NX 8GB

And it is specifically highlighted BSP for for Orin NX16GB is 35.2.1, this was the reason for the use of old BSP -

So to work with 35.4.1 do i need to perform some addition for the support for 16GB variant?

[WayneWWW]

Hi,

There is no “highlight” for this BSP. That was just because this is the first BSP start to support Orin NX 16GB…
It has nothing special and it is still a old BSP from latest one. Old BSP always has some bugs that already resolved in later version…

[WayneWWW]

@alex.imp

I tried the mentioned BSP - with the EMMC changes in the bootloader directory. Similar process mentioned above. The flashing completed successfully, L4T-README opens up at first but I am not getting access to the SSH on /ttyACM0. Post this the L4T-README goes away.

Also, Please try to learn how to use uart serial console to check log.

Jetson Nano & NX Style - Serial Debug Console - JetsonHacks

[alex.imp]

@WayneWWW
I have new SSD to work with now and have tried some more solutions since yesterday.

Since we have no eMMC and are working with only NVMe SSD i also followed a issue and modified the boot order mentioned in this guide.
https://docs.nvidia.com/jetson/archives/r35.3.1/DeveloperGuide/text/SD/Bootloader/UEFI.html#customizing-the-default-boot-order-in-l4tconfiguration-dtbo-in-the-bsp-directory

gNVIDIATokenSpaceGuid {
DefaultBootPriority {
data = “nvme”;
/* data = “Replace above data with your default boot order strings”; */
locked;
};
};

Towards end of the process the flashing failed - (logs attached)
flasing_logs_1.txt (243.6 KB)
minicom_logs_1.txt (100.3 KB)

We re tried to flash the board with

sudo ./tools/kernel_flash/l4t_initrd_flash.sh --flash-only --external-device nvme0n1p1
-c tools/kernel_flash/flash_l4t_external.xml -p “-c bootloader/t186ref/cfg/flash_t234_qspi.xml”
–showlogs --network usb0 p3509-a02+p3767-0000 internal

Flash was successful this time but still no access to the OS and initial configuration.
minicom_logs_2-flashonly.txt (414.8 KB)
flasing_logs_2-flashonly.txt (41.8 KB)

[WayneWWW]

Hi,

Several mistakes in your logs…

1. Each line of your minicom log got trunactaed after specific length… if you dont’ know what I mean, open your log and read it by yourself. Resize your terminal and dump log again.

2. Your minicom_logs_2-flashonly.txt hit HDMI issue which has nothing do with nvme… please do not connect HDMI first. I don’t really want to take care about HDMI issue for now.

[alex.imp]

@WayneWWW

@WayneWWW

trunactaed after specific length

1. I have tried to flash the board and collected the logs. I tried to compare the logs for the truncated length but i found it to similar in every attempt.
   7FEB_Flashing_logs.txt (242.6 KB)
   7FEB_Minicom_logs.txt (100.3 KB)

@WayneWWW

Your minicom_logs_2-flashonly.txt hit HDMI issue which has nothing do with nvme… please do not connect HDMI first. I don’t really want to take care about HDMI issue for now.

2. I have nothing connected to the carrier board other than SoM, NVMe SSD, Debug USB, Ethernet and Power supply.
   HDMI is actually is not required for my application and carrier board does not have the hardware for the same.

[WayneWWW]

Then please do not use p3509-a02+p3767-0000 board config for your flash because this one would enable HDMI by default.

[alex.imp]

Any suggestion on the board config files that I should use?

Also do i need to reformat the NVMe SSD for flashing as it has the partitions from previous flashing attempts?

[kayccc]

Is this still an issue to support? Any result can be shared?

[alex.imp]

We reverted back to Linux version 35.2.1 and are currently using it as 35.4.1 keeps failing with similar changes.
As of now the Linux version 35.2.1 are meeting our functional needs.