OrinNX 8GB flash support for CustomBoard in 35.3.1

shinichiro.adachi · April 25, 2023, 6:04am

We are using custom conf based on p3509-a02+p3767-0000.conf
The modification for custom config is only PINMUX_CONFIG, PMC_CONFIG, DTB_FILE, ODMDATA.

We have already confirmed that it is possible to write to OrinNX 16GB + Custom Board with the following command.

	$ sudo ./tools/kernel_flash/l4t_initrd_flash.sh \
	--external-device nvme0n1p1 \
	-c tools/kernel_flash/flash_l4t_external.xml \
	-p "-c bootloader/t186ref/cfg/flash_t234_qspi.xml" \
	--network usb0 \
	--showlogs \
	dx-u2200+p3767-0000 \
	internal

However, when we use same command to write to OrinNX 8GB + Custom Board, the following error occurs at startup.

E> I2C_DEV: Could not read 256 registers of size 1 from slave 0xae at 0x00000000 via instance 0.
E> eeprom: Failed to read I2C slave device
C> Task 0x0 failed (err: 0x1f1e050d)
E> Top caller module: I2C_DEV, error module: I2C, reason: 0x0d, aux_info: 0x05
I> Busy Spin

Custom Board is exactly the same.
Do we need to make any modifications to the OrinNX 8GB write?
Is it possible to share the same massflash blob for OrinNX 8GB/16GB?

KevinFFF · April 25, 2023, 6:59am

Hi shinichiro.adachi,

Please refer to the BOARDSKU difference between Orin NX 16GB and 8GB.

Jetson Orin NX 16GB (P3767-0000)
Jetson Orin NX 8GB (P3767-0001)
Do you distinguish them in your board config (dx-u2200+3767-0000.conf)?

You could refer to the following line in p3768-0000+p3767-0000.conf.

        if [ "${board_sku}" = "0000" ] || [ "${board_sku}" = "0002" ]; then
                # use default DTB and CFG files except for TS1 & EB1 revisions
                if [ "${board_FAB}" = "TS1" ] || [ "${board_FAB}" = "EB1" ]; then
                        PINMUX_CONFIG="tegra234-mb1-bct-pinmux-p3767-dp-a01.dtsi";
                        PMC_CONFIG="tegra234-mb1-bct-padvoltage-p3767-dp-a01.dtsi";
                        BPFDTB_FILE="tegra234-bpmp-3767-0000-a00-3509-a02.dtb";
                fi
        elif [ "${board_sku}" = "0001" ]; then
                BPFDTB_FILE="tegra234-bpmp-3767-0001-3509-a02.dtb";
                DTB_FILE="tegra234-p3767-0001-p3768-0000-a0.dtb";
        elif [ "${board_sku}" = "0003" ] || [ "${board_sku}" = "0005" ]; then
                BPFDTB_FILE="tegra234-bpmp-3767-0003-3509-a02.dtb";
                DTB_FILE="tegra234-p3767-0003-p3768-0000-a0.dtb";
        elif [ "${board_sku}" = "0004" ]; then
                BPFDTB_FILE="tegra234-bpmp-3767-0004-3509-a02.dtb";
                DTB_FILE="tegra234-p3767-0004-p3768-0000-a0.dtb";
        fi

shinichiro.adachi · April 25, 2023, 9:14am

Hi,@KevinFFF,

We refer to the following URL and modify update_flash_args_common() as follows.

update_flash_args_common()
{
...
	elif [ "${board_sku}" = "0001" ]; then
		#DTB_FILE="tegra234-p3767-0000-p3509-a02.dtb";
		DTB_FILE="tegra234-p3767-0001-p3509-a02.dtb";
		BPFDTB_FILE="tegra234-bpmp-3767-0001-3509-a02.dtb";
	elif [ "${board_sku}" = "0003" ] || [ "${board_sku}" = "0005" ]; then

And, we flashed with the following command as same as above.

	$ sudo ./tools/kernel_flash/l4t_initrd_flash.sh \
	--external-device nvme0n1p1 \
	-c tools/kernel_flash/flash_l4t_external.xml \
	-p "-c bootloader/t186ref/cfg/flash_t234_qspi.xml" \
	--network usb0 \
	--showlogs \
	dx-u2200+p3767-0000 \
	internal

SKU is confirmed to be set at 0001 during flash log.

Board ID(3767) version(300) sku(0001) revision(M.3)
Chip SKU(00:00:00:D4) ramcode(00:00:00:02) fuselevel(fuselevel_production) board_FAB(300)
emc_opt_disable_fuse:(0)

As a result, log output stopped after UEFI booting and nothing was displayed on HDMI as shown below.

I/TC: This is only for TZ-SE testing and should NOT be used for a shipping product!
I/TC: Primary CPU switching to normal world boot

 疂etson UEFI firmware (version 3.1-32827747 built on 2023-03-19T14:56:32+00:00)

Do we need to change all BPFDTB_FILE/DTB_FILE in the conf file to p3767-0001’s file,not just update_flash_args_common()?

dx-u2200+p3767-0000.conf (3.6 KB)

WayneWWW · April 25, 2023, 9:26am

please follow the document and disable the cvb eeprom read.

https://docs.nvidia.com/jetson/archives/r35.3.1/DeveloperGuide/text/HR/JetsonModuleAdaptationAndBringUp/JetsonOrinNxNanoSeries.html?highlight=cvb#eeprom-modifications

shinichiro.adachi · April 25, 2023, 9:33am

Hi, @WayneWWW,

please follow the document and disable the cvb eeprom read.

We already did it.

Linux_for_Tegra$ cat bootloader/t186ref/BCT/tegra234-mb2-bct-misc-p3767-0000.dts
/dts-v1/;

#include "tegra234-mb2-bct-common.dtsi"

/ {
	mb2-misc {
		eeprom {
			cvm_eeprom_i2c_instance = <0>;
			cvm_eeprom_i2c_slave_address = <0xa0>;
			cvm_eeprom_read_size = <0x0>;
			cvb_eeprom_i2c_instance = <0x0>;
			cvb_eeprom_i2c_slave_address = <0xae>;
			cvb_eeprom_read_size = <0x0>;
		};
	};
};

Linux_for_Tegra/bootloader$ sudo find ./ -name "*.dts" -print |xargs grep -n "cvb_eeprom_read_size"
./tegra234-mb2-bct-misc-p3767-0000.dts:13:			cvb_eeprom_read_size = <0x0>;
./t186ref/BCT/tegra234-mb2-bct-misc-p3767-0000.dts:13:			cvb_eeprom_read_size = <0x0>;
./t186ref/BCT/tegra234-mb2-bct-misc-p3701-0002-p3711-0000.dts:12:            cvb_eeprom_read_size = <0x0>;
./t186ref/BCT/tegra234-mb2-bct-misc-p3701-0002-p3740-0002.dts:41:            cvb_eeprom_read_size = <0x0>;

Linux_for_Tegra/bootloader$ sudo find ./ -name "*.dtsi" -print |xargs grep -n "cvb_eeprom_read_size"
./tegra234-mb2-bct-common.dtsi:40:            cvb_eeprom_read_size = <0x0>;

WayneWWW · April 25, 2023, 9:36am

Please enable UEFI debug log and share full log agin.

For every log you shared on this forum in future, please always share full log. Start from the moment you power on the board.

shinichiro.adachi · April 26, 2023, 1:09am

Please enable UEFI debug log and share full log agin.

After updating UEFI build environment to R35.3.1,
we are encounting following error at UEFI build.

$ edk2_docker edk2-nvidia/Platform/NVIDIA/Jetson/build.sh
...
nvidia: found command: python3.
nvidia: found command: virtualenv.
nvidia: found command: mono.
nvidia: found command: aarch64-linux-gnu-gcc.
nvidia: Activating Python virtual environment.
nvidia: Building from tarball
nvidia: Updating build environment (edk2-nvidia/Platform/NVIDIA/Jetson/PlatformBuild.py).
edk2-nvidia/Silicon/NVIDIA/scripts/prepare_stuart.sh: line 49: stuart_update: command not found

Could you tell me how to resolve the error?
or could you give me R35.3.1 UEFI debug binary?

shinichiro.adachi · April 26, 2023, 2:57am

Redone "edk2_docker edkrepo clone ~ " and finally UEFI for R35.3.1 build succeeded.
Replace bootloader/uefi_jetson.bin to uefi_Jetson_DEBUG.bin.
And, we flashed with the command as same as previous.

[hostpc_log]

***************************************
*                                     *
*  Step 3: Start the flashing process *
*                                     *
***************************************
Waiting for target to boot-up...
Waiting for target to boot-up...
Waiting for target to boot-up...
Waiting for target to boot-up...

[console_log]

TegraI2cStartRequest: No ACK received
Failed to read eeprom (No Response)
ValidateEepromData: CRC mismatch, expected 35 got FF
Eeprom data validation failed(Device Error)
TegraI2cStartRequest: No ACK received
Failed to read eeprom (No Response)
ValidateEepromData: CRC mismatch, expected 35 got FF
Eeprom data validation failed(Device Error)
PROGRESS CODE: V02010000 I0
PROGRESS CODE: V02010004 I0
PROGRESS CODE: V02010004 I0
PROGRESS CODE: V02010004 I0
PROGRESS CODE: V02010000 I0
PROGRESS CODE: V02010004 I0
PROGRESS CODE: V02010004 I0
PROGRESS CODE: V02010004 I0
MnpSyncSendPacket: No network cable detected.
MnpSyncSendPacket: No network cable detected.

It seems that EEPROM CRC errors are occurring in UEFI during flashing from the host.
20230426_1150_OrinNX8GB_CustomBoard_flash_host_log.txt (261.7 KB)
20230426_1150_OrinNX8GB_CustomBoard_flash_console_log.txt (105.0 KB)

WayneWWW · April 26, 2023, 3:02am

Hi,

Is this a fresh module? Sounds like eeprom value is already corrupted.

Do you have nv devkit to validate?

shinichiro.adachi · April 26, 2023, 5:41am

Do you have nv devkit to validate?

I haven’t nv devkit, so confirmed by XavierNX Devkit instead.

When I use OrinNX 8GB + XavierNX EVK, flash succeeded with no EEPROM error.

20230426_1436_OrinNX8GB_XavierNXEVK_flash_console_log.txt (164.6 KB)
20230426_1436_OrinNX8GB_XavierNXEVK_flash_host_log.txt (2.5 KB)

WayneWWW · April 26, 2023, 5:45am

Sounds weird. It feels like there are some i2c hardware issue on your custom board which leads you not even able to read cvm eeprom?

Could you enable UEFI log on devkit cases too and see if it still prints eeprom error?

shinichiro.adachi · April 26, 2023, 6:48am

Could you enable UEFI log on devkit cases too and see if it still prints eeprom error?

Enable UEFI debug log and flash again for XavierNX EVK.

PROGRESS CODE: V02010004 I0
DeviceDiscoveryBindingStart, failed to enable clocks Device Error
TegraI2cStartRequest: No ACK received
TegraI2cStartRequest: No ACK received
TegraI2cStartRequest: No ACK received
TegraI2cStartRequest: No ACK received
TegraI2cStartRequest: No ACK received
TegraI2cStartRequest: No ACK received
Failed to read eeprom (No Response)
TegraI2cStartRequest: No ACK received
Failed to read eeprom (No Response)

It seems No ACK received in XavierNX EVK case.

Our Custom Board has one I2C-EEPROM on c240000 bus, i2c addr is 0x50. Would this cause a problem? The content is blank.

This is command log in R35.2.1.

root@test-desktop:~# i2cdetect -y -a -r 0
     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
50: 50 -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
70: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --"

root@test-desktop:~# i2cdump -f -y 0 0x50
No size specified (using byte-data access)
     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f    0123456789abcdef
00: 02 00 fe 00 00 00 00 00 00 00 00 ff 00 00 00 00    ?.?.............
10: 00 01 00 01 36 39 39 2d 31 33 37 36 37 2d 30 30    .?.?699-13767-00
20: 30 30 2d 33 30 30 20 47 2e 33 00 00 00 00 00 00    00-300 G.3......
...

root@test-desktop:~# i2cdetect -y -a -r 1
     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
20: -- -- -- -- -- -- -- -- -- -- -- -- 2c -- -- --
30: -- -- UU -- -- -- -- -- -- -- -- -- -- -- -- --
40: UU -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
50: UU -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
70: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --

root@test-desktop:~#  hexdump -C /sys/devices/platform/c240000.i2c/i2c-1/1-0050/eeprom
00000000  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|
*
00001000

20230426_1529_OrinNX8GB_XavierNXEVK_flash_console_log_with_UEFI_debug.txt (322.2 KB)
20230426_1529_OrinNX8GB_XavierNXEVK_flash_host_log_with_UEFI_debug.txt (296.1 KB)

WayneWWW · April 26, 2023, 7:35am

The devkit does not have the CRC mismatch error, so the error is different.

If you think your extra i2c eeprom is causing error, please disable it in the device tree first and see if you can bypass it.

shinichiro.adachi · April 28, 2023, 3:50am

Finally, I just made the following changes in dx-u2200+p3767-0000.conf

--- dx-u2200+p3767-0000.conf
+++ dx-u2200+p3767-0000.conf
@@ -55,7 +55,7 @@ update_flash_args_common()
                        BPFDTB_FILE="tegra234-bpmp-3767-0000-a00-3509-a02.dtb";
                fi
        elif [ "${board_sku}" = "0001" ]; then
-               DTB_FILE="tegra234-p3767-0000-p3509-a02.dtb";
+               DTB_FILE="tegra234-p3767-0000-dx-u2200.dtb";
                BPFDTB_FILE="tegra234-bpmp-3767-0001-3509-a02.dtb";
        elif [ "${board_sku}" = "0003" ] || [ "${board_sku}" = "0005" ]; then

We confirmed that execute ./tools/kernel_flash/l4t_initrd_flash.sh directly can be written successfully.

20230427_1553_OrinNX8GB_CustomBoard_flash_console_log_succeed_with_nodebug.txt (184.1 KB)
20230427_1553_OrinNX8GB_CustomBoard_flash_host_log_succeed_with_nodebug.txt (295.6 KB)

shinichiro.adachi · April 28, 2023, 3:51am

After that, creating massflash blob, and following error occur when writing from the massflash blob.

[0144.977] E> BL_CARVEOUT: Failed to allocate memory of size 0x8000000 for CO:31.
[0144.984] C> Task 0x0 failed (err: 0x49490003)
[0144.988] E> Top caller module: BL_CARVEOUT, error module: BL_CARVEOUT, reason: 0x03, aux_info: 0x00
[0144.997] C> Boot Info Table status dump :
1
[0145.001] I> Busy Spin

Is there any workaround for this?

20230428_1148_OrinNX8GB_CustomBoard_massflash_console_log_failed.txt (4.2 KB)
20230428_1148_OrinNX8GB_CustomBoard_massflash_host_log_failed.txt.txt (1.1 KB)

WayneWWW · April 28, 2023, 4:21am

Are you talking about this error only happens with massflash but not initrd flash with single module?

shinichiro.adachi · April 28, 2023, 4:56am

Yes, direct flash command is below.
This command suceeded with no error.

sudo ./tools/kernel_flash/l4t_initrd_flash.sh \
    --external-device nvme0n1p1 \
    -c tools/kernel_flash/flash_l4t_external.xml \
    -p "-c bootloader/t186ref/cfg/flash_t234_qspi.xml" \
    --network usb0 \
    --showlogs \
    dx-u2200+p3767-0000 \
    internal

And, massflash blob create command is below.

TARGET_CONF=dx-u2200+p3767-0000
ROOTFS_SIZE=$((18*1024*1024*1024))
echo "run l4t_initrd_flash.sh"

sudo ./tools/kernel_flash/l4t_initrd_flash.sh --no-flash \
        --external-device nvme0n1p1 \
        -S ${ROOTFS_SIZE} \
        -c tools/kernel_flash/flash_l4t_external.xml \
        -p "-c bootloader/t186ref/cfg/flash_t234_qspi.xml" \
        --network usb0 \
        --massflash 5 \
        --showlogs \
        ${TARGET_CONF} \
        internal

And, massflash command is below.
It causes Busy Spin error.

sudo ./tools/kernel_flash/l4t_initrd_flash.sh --flash-only --massflash 5

WayneWWW · April 28, 2023, 4:58am

Just to clarify

1.does this massflash issue happen to multiple modules or only one module?

Is your issue able to reproduce on NV devkit? I mean same module with same flash command on devkit. Will it reproduce issue?

shinichiro.adachi · April 28, 2023, 5:48am

does this massflash issue happen to multiple modules or only one module?

Multiple modules.

Is your issue able to reproduce on NV devkit? I mean same module with same flash command on devkit. Will it reproduce issue?

Yes. I swapped OrinNX 8GB CPU Module from CustomBoard to XavierNX EVK, and flash with same command, reproduced Busy Spin in EVK.

20230428_1429_OrinNX8GB_XavierNXEVK_flash_BusySpin_console_log.txt (4.2 KB)
20230428_1429_OrinNX8GB_XavierNXEVK_flash_BusySpin_host_log.txt (876 Bytes)

WayneWWW · April 28, 2023, 5:51am

Hi,

There is a bug in the flash config. It seems you are using the problematic dtb to flash orin nx 8GB.

Could you try to use the correct dtb first?

Topic		Replies	Views
Issue with "Waiting for target to boot-up..." Error while Flashing Jetson Orin NX Jetson Orin NX kernel , flash	39	1530	July 17, 2024
Orin AGX no boot, no UART, after flashing secure boot and encrypted internal disk Jetson AGX Orin boot , security , jetson , uefi	7	110	December 17, 2024
Orin NX - Flashing Problems Jetson Orin NX reflash	25	2103	July 23, 2023
MB2 still trying to read CVB EEPROM after changing misc dts Jetson Orin NX board-design , eeprom	18	233	October 8, 2024
Jetson Orin NX - board can't flash after partial fuse burn Jetson Orin NX security , nvbugs	53	313	September 24, 2024
Orin Nano and tc358743 capture issue Jetson Orin Nano camera	101	5098	March 18, 2024
Orin NX JetPack 5.1 flashing fails Jetson Orin NX flash	5	579	December 5, 2023
[MASS FLASH] Flashing Multiple Jetson Orin NX 8GB(QSPI+NVME) Devices Always Failed Jetson Orin NX hw	31	1598	October 27, 2023
Unrecognized module SKU error while flashing Jetson Orion NX 16 GB NVme module using initrdsh Jetson Orin NX flash	39	985	July 2, 2024
Need to understand error from tegrarcm_v2 Jetson Orin NX reflash	32	1969	July 12, 2023

OrinNX 8GB flash support for CustomBoard in 35.3.1

Related topics