Jetson AGX Orin board not able to enumerate FPGA PCIe card (Latest kernel 35.1.0)

Hi NV support team,

I have one FPGA board with PCIe x 8 Gen3 design. I have tested the FPGA board PCIe design on Gigabyte and Supermicro motherboard. Both motherboard able to enumerate PCIe x8 Gen3 speed and work normally.

But when I plug the FPGA board to Jetson AGX Orin PCIe x16 connector, Jetson is not able to detect it. I found that the +12V supply on the connector is not available after booting up. After going thru the forum search, I try to follow this link SOLUTION/TUTORIAL: Jetson ORIN Enabling PCIE power to change the device tree settings.

  • p3737_vdd_3v3_pcie: regulator@105 - Remove the flag “regulator-boot-on” and change the pin to TEGRA234_MAIN_GPIO(H, 4)
  • p3737_vdd_12v_pcie: regulator@114 - Remove the flag “regulator-boot-on”
    2.2) Find the file tegra234-p3737-pcie.dtsi, it is in ./Linux_for_Tegra/source/public/hardware/nvidia/platform/t23x/concord/kernel-dts/cvb

pcie_ep@141a0000 - Replace the nvidia,refclk-select-gpios = <&tegra_aon_gpio TEGRA234_AON_GPIO(AA, 4) GPIO_ACTIVE_HIGH>; with the nvidia,refclk-select-gpios = <&tegra_main_gpio TEGRA234_MAIN_GPIO(Q, 4) GPIO_ACTIVE_HIGH>;

I rebuilt the latest linux kernel and DTB with the above changes successfully. I reflash the Orin board using commandline and succeeded in flashing as well.

This time when I power up the Orin board, the PCIe 12V and 3.3V supply is available after booting up. Both supply did not go away, it means the device tree changes works. But Orin board still not able to enumerate the FPGA board. Looking at the dmesg msg, I found this error “Failed to get slot regulators” and “Phy link never came up”. May I know how to solve the problem? The FPGA PCIe interface is working on both motherboard. I have uploaded the whole dmesg log.

[ 7.511955] tegra194-pcie 14100000.pcie: Adding to iommu group 10
[ 7.524022] tegra194-pcie 14100000.pcie: Using GICv2m MSI allocator
[ 7.531128] tegra194-pcie 14160000.pcie: Adding to iommu group 11
[ 7.542520] tegra194-pcie 14160000.pcie: Using GICv2m MSI allocator
[ 7.549447] tegra194-pcie 141a0000.pcie: Adding to iommu group 12
[ 7.560563] tegra194-pcie 141a0000.pcie: Using GICv2m MSI allocator
[ 7.567056] tegra194-pcie 141a0000.pcie: Failed to get slot regulators: -517
[ 9.837596] tegra194-pcie 14100000.pcie: Using GICv2m MSI allocator
[ 9.847532] tegra194-pcie 14100000.pcie: host bridge /pcie@14100000 ranges:
[ 9.859430] tegra194-pcie 14100000.pcie: IO 0x0030100000…0x00301fffff → 0x0030100000
[ 9.871813] tegra194-pcie 14100000.pcie: MEM 0x20a8000000…0x20afffffff → 0x0040000000
[ 9.880510] tegra194-pcie 14100000.pcie: MEM 0x2080000000…0x20a7ffffff → 0x2080000000
[ 9.998401] tegra194-pcie 14100000.pcie: Link up
[ 10.004171] tegra194-pcie 14100000.pcie: PCI host bridge to bus 0001:00
[ 10.186777] pcieport 0001:00:00.0: Adding to iommu group 10
[ 10.221738] pcieport 0001:00:00.0: PME: Signaling with IRQ 51
[ 10.227959] pcieport 0001:00:00.0: AER: enabled with IRQ 51
[ 10.234560] tegra194-pcie 14160000.pcie: Using GICv2m MSI allocator
[ 10.246503] tegra194-pcie 14160000.pcie: host bridge /pcie@14160000 ranges:
[ 10.253686] tegra194-pcie 14160000.pcie: IO 0x0036100000…0x00361fffff → 0x0036100000
[ 10.262396] tegra194-pcie 14160000.pcie: MEM 0x2428000000…0x242fffffff → 0x0040000000
[ 10.271093] tegra194-pcie 14160000.pcie: MEM 0x2140000000…0x2427ffffff → 0x2140000000
[ 11.386882] tegra194-pcie 14160000.pcie: Phy link never came up
[ 11.393046] tegra194-pcie 14160000.pcie: PCI host bridge to bus 0004:00
[ 11.468275] pcieport 0004:00:00.0: Adding to iommu group 11
[ 11.474264] pcieport 0004:00:00.0: PME: Signaling with IRQ 53
[ 11.480457] pcieport 0004:00:00.0: AER: enabled with IRQ 53
[ 11.514349] tegra194-pcie 141a0000.pcie: Using GICv2m MSI allocator
[ 11.525231] tegra194-pcie 141a0000.pcie: Failed to get slot regulators: -517
[ 11.536253] vdd-3v3-pcie: supplied by vdd-3v3-sys
[ 11.613938] tegra194-pcie 141a0000.pcie: Using GICv2m MSI allocator
[ 11.735417] tegra194-pcie 141a0000.pcie: host bridge /pcie@141a0000 ranges:
[ 11.742613] tegra194-pcie 141a0000.pcie: IO 0x003a100000…0x003a1fffff → 0x003a100000
[ 11.751309] tegra194-pcie 141a0000.pcie: MEM 0x2b28000000…0x2b2fffffff → 0x0040000000
[ 11.760006] tegra194-pcie 141a0000.pcie: MEM 0x2740000000…0x2b27ffffff → 0x2740000000
[ 12.870643] tegra194-pcie 141a0000.pcie: Phy link never came up
[ 12.879230] tegra194-pcie 141a0000.pcie: PCI host bridge to bus 0005:00
[ 12.954554] pcieport 0005:00:00.0: Adding to iommu group 12
[ 12.960529] pcieport 0005:00:00.0: PME: Signaling with IRQ 55
[ 12.966692] pcieport 0005:00:00.0: AER: enabled with IRQ 55

Regards
YE
jetson_pcie_problem (79.9 KB)

So is this issue is on Orin or AGX Xavier? You filed the topic is AGX forum.

Hi WayneWWW,

Sorry the issue is on Orin board.

I just change another FPGA board with Gen3 x4 lane design, Jetson is able to enumerate the FPGA board successfully. One difference is this FPGA PCIe signal PCIE_PRSNT_B_X4 (B31) is tie to ground. Jetson is able to detect Gen3 x4.

But the other FPGA design is Gen3 x 8, Jetson not able to enumerate. So I tie Jetson PCIe connector pin B48 PRSNT2n_X8 to ground to tell Jetson PCIe controller to train to x8 lane. But Jetson still not able to enumerate (Phy link never came up). May I know whether the existing kernel code for C5 PCIe controller can train external device to x8 lane? Thanks.

Regards
YE

Hi,

  1. If you are using jetpack5.0.2, then you shouldn’t need to add any patch by yourself. The BSP shall already contain the fix for 12v.

  2. Yes, the C5 shall be able to train x8 line.

Hi,

Okay. Yes, I am using Jetpack5.0.2.

Need to mention that the FPGA board with Gen3 x8 design, I am using a 50cm PCIe cable between Jetson and FPGA board. Maybe the cable is too long, that’s why Jetson cannot enumerate due to signal lane quality.

When I put a video I/O card with x8 lane PCIe finger direct to Jetson connector, the controller is able to detect Gen3 x8 speed.

Can the C5 controller able to boost the lane signal quality in order to compensate for the 50cm PCIe cable that I am using? Can the kernel code be modified due to this? The FPGA board with x8 lane design do not have a PCIe gold finger to be inserted to the connector, I have to use PCIe 50cm cable.

If C5 controller cannot do anything, then I have to change the FPGA speed to Gen2 x8 and try again on Jetson.

Regards
YE

Pease find the Jetson AGX Orin Series Tuning and Compliance Guide Application Note from Jetson Download Center | NVIDIA Developer.

Thanks

Hi kaycc,

I will try on shorter PCIe cable and try again. Looks like the C5 controller cannot be tuned for signal strength and pre-emphasis. Thank you.

Regards
YE

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.