NVME external drive not recognized on PCIe C0

Yes, you noticed where went wrong.

1 Like

So I updated the tegra234-bpmp-3701-0005-3737-0000.dtb.txt file in the path /Linux_for_Tegra/bootloader/t186ref/ with:

    uphy {
            status = "okay";
            hsio-uphy-config = <0x10>;
            hsstp-lane-map = <0x3>;
            nvhs-uphy-config = <0x1>;
            gbe-uphy-config = <0x0>;
    };

Converted it to a tegra234-bpmp-3701-0005-3737-0000.dtb using the dtc tool:

dtc --in-format dts tegra234-bpmp-3701-0005-3737-0000.dtb.txt --out-format dtb --out tegra234-bpmp-3701-0005-3737-0000.dtb

Reflashed and get the same error when booting:
��E/TC:?? 00 get_rpc_alloc_res:645 RPC allocation failed. Non-secure world result: ret=00
E/LD: init_elf:486 sys_open_ta_bin(bc50d971-d4c9-42c4-82cb-343fb7f37896)
E/TC:?? 00 ldelf_init_with_ldelf:131 ldelf failed with res: 0xffff000c
��

3h

��Unhandled Exception in EL3.
x30 = 0x0000000050000d00
x0 = 0x0000000000000000
x1 = 0x00000000be000011
x2 = 0x0000000000000000
x3 = 0x0000000000000011
x4 = 0x0000000000100000
x5 = 0x000000102d1fe5b8
x6 = 0x0000000401000000
x7 = 0x0000000401000000
x8 = 0x0000000000000000
x9 = 0x000000005001c380
x10 = 0x55aaa055071dbd35
x11 = 0x55aa8255ce1abfe1
x12 = 0x0000000000000001
x13 = 0x0000000000000003
x14 = 0x0000000000000001
x15 = 0x0000000000000003
x16 = 0x000000102902803c
x17 = 0x00000000307cf10e
x18 = 0x0000001028f3b2f0
x19 = 0x000000005001cec0
x20 = 0x0000000000000000
x21 = 0x0000000000000000
x22 = 0x0000000000000000
x23 = 0x0000000000000000
x24 = 0x0000000000000000
x25 = 0x0000000000000000
x26 = 0x0000000000000000
x27 = 0x0000000000000000
x28 = 0x0000000000000000
x29 = 0x0000000000000000
scr_el3 = 0x000000000003073d
sctlr_el3 = 0x00000000b0cd183f
cptr_el3 = 0x0000000000000000
tcr_el3 = 0x0000000080823518
daif = 0x00000000000002c0
mair_el3 = 0x00000000004404ff
spsr_el3 = 0x00000000600003c9
elr_el3 = 0x0000001028f35280
ttbr0_el3 = 0x0000000050026ac1
esr_el3 = 0x00000000be000011
far_el3 = 0x0000000000000000
spsr_el1 = 0x0000000000000000
elr_el1 = 0x0000000000000000
spsr_abt = 0x0000000000000000
spsr_und = 0x0000000000000000
spsr_irq = 0x0000000000000000
spsr_fiq = 0x0000000000000000
sctlr_el1 = 0x0000000030d00800
actlr_el1 = 0x0000000000000000
cpacr_el1 = 0x0000000000300000
csselr_el1 = 0x0000000000000000
sp_el1 = 0x0000000000000000
esr_el1 = 0x0000000000000000
ttbr0_el1 = 0x0000000000000000
ttbr1_el1 = 0x0000000000000000
mair_el1 = 0x0000000000000000
amair_el1 = 0x0000000000000000
tcr_el1 = 0x0000000000000000
tpidr_el1 = 0x0000000000000000
tpidr_el0 = 0x0000000080000000
tpidrro_el0 = 0x0000000000000000
par_el1 = 0x0000000000000800
mpidr_el1 = 0x0000000081000000
afsr0_el1 = 0x0000000000000000
afsr1_el1 = 0x0000000000000000
contextidr_el1 = 0x0000000000000000
vbar_el1 = 0x0000000000000000
cntp_ctl_el0 = 0x0000000000000005
cntp_cval_el0 = 0x00000000115f1307
cntv_ctl_el0 = 0x0000000000000000
cntv_cval_el0 = 0x0000000000000000
cntkctl_el1 = 0x0000000000000000
sp_el0 = 0x0000001028f3b2f0
isr_el1 = 0x0000000000000040
cpuectlr_el1 = 0xa000000b40543000
gicd_ispendr regs (Offsets 0x200 - 0x278)
Offset: value
0000000000000200: 0x0000000000000000
0000000000000204: 0x0000000000000000
0000000000000208: 0x0000000000000000
000000000000020c: 0x0000000000000000
0000000000000210: 0x0000000000000000
0000000000000214: 0x0000000000000000
0000000000000218: 0x0000000000010000
000000000000021c: 0x0000000000020000
0000000000000220: 0x0000000000000000
0000000000000224: 0x0000000000000000
0000000000000228: 0x0000000000000000
000000000000022c: 0x0000000000000000
0000000000000230: 0x0000000000000000
0000000000000234: 0x0000000000000000
0000000000000238: 0x0000000000000000
000000000000023c: 0x0000000000000000
0000000000000240: 0x0000000000000000
0000000000000244: 0x0000000000000000
0000000000000248: 0x0000000000000000
000000000000024c: 0x0000000000000000
0000000000000250: 0x0000000000000000
0000000000000254: 0x0000000000000000
0000000000000258: 0x0000000000000000
000000000000025c: 0x0000000000000000
0000000000000260: 0x0000000000000000
0000000000000264: 0x0000000000000000
0000000000000268: 0x0000000000000000
000000000000026c: 0x0000000000000000
0000000000000270: 0x0000000000000000
0000000000000274: 0x0000000000000000
0000000000000278: 0x0000000000000000
000000000000027c: 0x0000000000000000

Please make sure all the device tree node has to match your ODMDATA.

I mean not only C0, C7. Your USB and MGBE needs to match this too. If something is not in use, disable it.

Also, you have to do full flash but not only modify /boot/dtb.

We are doing a full flash every time. We had the USB interfaces and ethernet working, just not the NVME. I’m not sure what else to change.

This process is much harder than the Xavier development. Are we going to run into these same issues when we start Orin NX & Orin Nano development?

Probably. If you want to play with such ODMATA setting.

Ok, we will try and see if the ODMDATA values are correct and maybe start changing those one at a time (we dont have GBE and are trying to put the SoMs in Endpoint Mode).

Well, I changed the ODMDATA for our desired UPHY (Config #2) but it doesn’t get passed to the tegra234-bpmp-3701-0005-3737-0000.dts file. I feel like I’m missing something, maybe in kernel/device tree compiling.

It sounds like you still didn’t get the situation. Let me clarify the whole concept again.

Hope you and your colleagues can understand.

  1. I am not sure why the ODMDATA was not working in the beginning. Actually there is an alternative to check. Dump this node from your jetson

ubuntu@jetson:/$ sudo cat /sys/kernel/debug/bpmp/debug/uphy/config

And this document has the table to tell you how to interpret the result.
https://docs.nvidia.com/jetson/archives/r35.5.0/DeveloperGuide/HR/JetsonModuleAdaptationAndBringUp/JetsonAgxOrinSeries.html?highlight=odmdata#uphy-lane-configuration

  1. The “Unhandled Exception in EL3.” in UEFI stage means your device tree does not match to the ODMDATA. It is not just “my usb interfaces are fine, so my device tree should be ok” .
    For example, you replace USB3.2 P0 to PCIe C0. Then did you disable usb3-0 in your device tree or not?
    Also, you told us you are not using MGBE. Then did you disable MGBE node in your device tree?
    That is what I called “you have to match the device tree to ODMDATA”. But not just “I already added C0 and C7”.

ok that makes a little more sense, thank you. We will work on it this upcoming week. How do we know which device tree files match, say the USB3-0 interface? Is there a document section that states this info? Usually we just do a text search in files on source directory.

Just do text search again or trace the code. I don’t really do any “code memorization” for a device tree. Especially when there are 20 ~30 device trees for the whole Jetson platform family… Just learn the basics method and you can find the file for any kind of device trees…

As for the usb related, adaptation guide has it too.

https://docs.nvidia.com/jetson/archives/r35.4.1/DeveloperGuide/text/HR/JetsonModuleAdaptationAndBringUp/JetsonAgxOrinSeries.html#porting-the-universal-serial-bus

1 Like

Checking this register from a version that boots (without the C0/C7 changes or the tegra234-bpmp-3701-0005-3737-0000.dtb manual changes) gives us:

0x40800000

I’m not sure how to interpret the result from this chart:

The following table shows the layout of the ODMDATA register inside the T234:

31:26 25:23 22:18 17 16 15 14 13:0
HSIO UPHY Config HSIO UPHY Config GBE UPHY Config GBE3 Mode GBE2 Mode GBE1 Mode GBE0 Mode Reserved
Config Number in NVHS UPHY Lane Mapping Options Config Number in NVHS UPHY Lane Mapping Options Config Number in GBE UPHY Lane Mapping Options 0:5G, 1:10G 0:5G, 1:10G 0:5G, 1:10G 0:5G, 1:10G TBD
BPMP-FW DTB /uphy/hsio-uphy-config BPMP-FW DTB /uphy/nvhs-uphy-config BPMP-FW DTB /uphy/gbe-uphy-config BPMP-FW DTB /uphy/gbe3-enable-10g BPMP-FW DTB /uphy/gbe2-enable-10g BPMP-FW DTB /uphy/gbe1-enable-10g BPMP-FW DTB /uphy/gbe0-enable-10g TBD

Is my ODMDATA being interpreted correctly as 'gbe-uphy-config-0,nvhs-uphy-config-1,hsio-uphy-config-16,hsstp-lane-map-3?

I am not sure what is the difficulty in interpreting this…

Open a calculator in engineer mode. Put 0x40800000 to it and it will give you the value of each bit…
And the table already told you the meaning of each bit…