Jetson TK1 mini PCIe port not detecting PCIe switch

Autonomous Machines Jetson & Embedded Systems Jetson TK1
1

We are working with a custom design based on Jetson TK1 board with another custom designed NXP’s single board computer. Both the custom boards had to communicate over PCIe through a PCIe switch(89HPES12NT3ZBBCGI) where Tegra Tk1 will be root complex and NXP’s processor will be endpoint. I am always failing to successfully establish a link between them.

I tried to communicate custom NXP’s board with NXP’s reference board and it detected both:PCIe switch and the endpoint processor. Then I replaced NXP’s reference board with Jetson TK1 but it again failed to establish a link.

In Uboot Jetson shows link up on port 0;that is connected to mini PCIe port; but when the board boots up, it does not show any device when I run lspci command. Please see the following logs on jetson TK1 kit

Uboot Logs

tegra-pcie: PCI regions:
tegra-pcie:   I/O: 0x12000000-0x12010000
tegra-pcie:   non-prefetchable memory: 0x13000000-0x20000000
tegra-pcie:   prefetchable memory: 0x20000000-0x40000000
tegra-pcie: 2x1, 1x1 configuration
tegra-pcie: probing port 0, using 2 lanes
tegra-pcie: probing port 1, using 1 lanes

Lspci command

ubuntu@tegra-ubuntu:~$ lspci
00:00.0 PCI bridge: NVIDIA Corporation Device 0e13 (rev a1)
01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0c)

Dmesg Command

ubuntu@tegra-ubuntu:~$ dmesg | grep pci
[    0.000000] Kernel command line: console=ttyS0,115200n8 console=tty1 no_console_suspend=1 lp0_vec=2064@0xf46ff000 video=tegrafb mem=1862M@2048M memtype=255 ddr_die=2048M@2048M section=256M pmuboard=0x0177:0x0000:0x02:0x43:0x00 vpr=151M@3945M tsec=32M@3913M otf_key=c75e5bb91eb3bd947560357b64422f85 usbcore.old_scheme_first=1 core_edp_mv=1150 core_edp_ma=4000 tegraid=40.1.1.0.0 debug_uartport=lsport,3 power_supply=Adapter audio_codec=rt5640 modem_id=0 android.kerneltype=normal usb_port_owner_info=0 fbcon=map:1 commchip_id=0 usb_port_owner_info=0 lane_owner_info=6 emc_max_dvfs=0 touch_id=0@0 tegra_fbmem=32899072@0xad012000 board_info=0x0177:0x0000:0x02:0x43:0x00 root=/dev/mmcblk0p1 rw rootwait tegraboot=sdmmc gpt pci=noaer
[    1.382249] pci_bus 0000:00: root bus resource [mem 0x32100000-0x3fffffff]
[    1.382278] pci_bus 0000:00: root bus resource [mem 0x12100000-0x320fffff pref]
[    1.382306] pci_bus 0000:00: root bus resource [io  0x1000-0xffff]
[    1.382331] pci_bus 0000:00: No busn resource found for root bus, will use [bus 00-ff]
[    1.382392] pci 0000:00:00.0: [10de:0e13] type 01 class 0x060400
[    1.382488] pci 0000:00:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    1.382744] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[    1.382942] pci 0000:01:00.0: [10ec:8168] type 00 class 0x020000
[    1.382976] pci 0000:01:00.0: reg 10: [io  0x0000-0x00ff]
[    1.383023] pci 0000:01:00.0: reg 18: [mem 0x00000000-0x00000fff 64bit]
[    1.383055] pci 0000:01:00.0: reg 20: [mem 0x00000000-0x00003fff 64bit pref]
[    1.383158] pci 0000:01:00.0: supports D1 D2
[    1.383169] pci 0000:01:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    1.385316] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01
[    1.385332] pci_bus 0000:00: busn_res: [bus 00-ff] end is updated to 01
[    1.385624] pcieport 0000:00:00.0: Signaling PME through PCIe PME interrupt
[    1.385652] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt
[    1.385679] pcie_pme 0000:00:00.0:pcie01: service driver pcie_pme loaded
[    1.385773] pcieport 0000:00:00.0: BAR 8: assigned [mem 0x32100000-0x321fffff]
[    1.385803] pcieport 0000:00:00.0: BAR 9: assigned [mem 0x12100000-0x121fffff 64bit pref]
[    1.385833] pcieport 0000:00:00.0: BAR 7: assigned [io  0x1000-0x1fff]
[    1.385863] pci 0000:01:00.0: BAR 4: assigned [mem 0x12100000-0x12103fff 64bit pref]
[    1.385908] pci 0000:01:00.0: BAR 2: assigned [mem 0x32100000-0x32100fff 64bit]
[    1.385949] pci 0000:01:00.0: BAR 0: assigned [io  0x1000-0x10ff]
[    1.385977] pcieport 0000:00:00.0: PCI bridge to [bus 01]
[    1.386000] pcieport 0000:00:00.0:   bridge window [io  0x1000-0x1fff]
[    1.386025] pcieport 0000:00:00.0:   bridge window [mem 0x32100000-0x321fffff]
[    1.386054] pcieport 0000:00:00.0:   bridge window [mem 0x12100000-0x121fffff 64bit pref]
[    4.816718] ehci-pci: EHCI PCI platform driver
[   13.980787] vgaarb: this pci device is not a vga device
[   14.254782] vgaarb: this pci device is not a vga device
[   19.815777] vgaarb: this pci device is not a vga device
[   19.825720] vgaarb: this pci device is not a vga device
[  620.495872] vgaarb: this pci device is not a vga device

Can someone comment what might be the possible reason of this issue?

2

Can someone help me.

3

With the forum switch to new software I’m thinking some posts will have been missed (sorry). I cannot answer your specific question, but many custom PCIe setups involving PCI devices which are not instantly available, and instead require time to boot, cause errors until PCI enumeration is delayed from the Jetson side. For example, there have been several FPGA devices which take too long to boot, and enumeration completes prior to the FPGA being able to respond.

Normally PCI would be able to detect such that late enumeration is automatic, but Jetsons are configured for energy savings and do not devices in need of late enumeration. There have been some posts on the forum of how to delay enumeration, and this takes care of the issue much of the time. In other cases people just ensure that the PCI device is fully booted prior to booting the Jetson. Perhaps you have a timing issue for boot versus enumeration.

The posts I know of on delayed enumeration are for the 64-bit platforms, but if you think delayed enumeration might be the issue, then someone here can probably provide details to delay enumeration on a TK1.