PCIe Hot-Plug not working

Please provide the following info (tick the boxes after creating this topic):
Software Version
DRIVE OS 6.0.4 SDK
other

Target Operating System
Linux
QNX
other

Hardware Platform
DRIVE AGX Orin Developer Kit (940-63710-0010-D00)
DRIVE AGX Orin Developer Kit (940-63710-0010-C00)
DRIVE AGX Orin Developer Kit (not sure its number)
other

SDK Manager Version
1.8.2.10409
other

Host Machine Version
native Ubuntu Linux 20.04 Host installed with SDK Manager
native Ubuntu Linux 20.04 Host installed with DRIVE OS Docker Containers
native Ubuntu Linux 18.04 Host installed with DRIVE OS Docker Containers
other

I want to use a miniSAS cable to communicate between two NVIDIA DRIVE AGX Orin Devkits(Hereafter, it is abbreviated as DDPO).
So I connected two DDPO with miniSAS cables and operated them according to the instructions in the URL below.

Unfortunately, DDPO configured as PCIe Root Port did not recognize the other DDPO configured as PCIe Endpoint as a PCIe device (DDPO configured as PCIe Root Port does not show Device ID for DDPO configured as PCIe Endpoint after entering lspci command (Linux command)).

Please answer the following questions.

Question1
Has there been a track record of communication between two DDPO using miniSAS cables?

Question2
Do I need to update PCIe Retimer FW?

Question3
Do I need to change the Device Tree?
Also, do I need to update AURIX FW?

Question4
If there is any additional work that needs to be done, could you tell me specifically?

Dear @shibata-a,
We will check internally and get back to you on this.

Question1
Has there been a track record of communication between two DDPO using miniSAS cables?
[NV] You should be able to see the related messages in kernel log while you execute the following hotplug commands:
echo 0 > controllers/141c0000.pcie_ep/start
echo 1 > controllers/141c0000.pcie_ep/start

**Question2**
Do I need to update PCIe Retimer FW?
**[NV]** Could you please help to provide your current FW version?

**Question3**
Do I need to change the Device Tree?
Also, do I need to update AURIX FW?
**[NV]** No, no change needed in device trees.
Moreover, I don't think this related to Aurix, but please paste your Aurix version here. Thanks.

**Question4**
If there is any additional work that needs to be done, could you tell me specifically
[NV] Development guide provides all the steps you need. There shouldn't be any other extra steps.
Could you please list all your steps/commands in both EP and RP?
We can review if there was anything missing.

Dear @BruceYangNV ,

Question1

[NV]
You should be able to see the related messages in kernel log while you execute the following hotplug commands:
echo 0 > controllers/141c0000.pcie_ep/start
echo 1 > controllers/141c0000.pcie_ep/start

Reply =>
Unfortunately, neither EP nor RP produced any relevant Linux kernel logs (dmesg).
(no change in dmesg)

Question2

[NV]
Could you please help to provide your current FW version?

=> Unfortunately, I couldn’t confirm it. How do I check the PCIe Retimer FW version currently in use at DDPO?

Question3

[NV]
please paste your Aurix version here. Thanks.

Reply =>
1.46.15

Question4

[NV]
Development guide provides all the steps you need. There shouldn’t be any other extra steps.
Could you please list all your steps/commands in both EP and RP?
We can review if there was anything missing.

Reply =>
I followed the instructions in the following URL.

■RP
$ sudo modprobe nvscic2c-pcie-epc

■EP
$ sudo modprobe nvscic2c-pcie-epf
$ sudo -s
$ cd /sys/kernel/config/pci_ep/
$ mkdir functions/nvscic2c_epf_22CC/func
$ echo 0x10DE > functions/nvscic2c_epf_22CC/func/vendorid
$ echo 0x22CC > functions/nvscic2c_epf_22CC/func/deviceid
$ ln -s functions/nvscic2c_epf_22CC/func controllers/141c0000.pcie_ep
$ echo 0 > controllers/141c0000.pcie_ep/start
$ echo 1 > controllers/141c0000.pcie_ep/start

Finally, I typed the lspci command (a Linux command) on the RP’s console, but the EP’s DeviceID did not appear in the PCIe device list.

Thanks,

Q1. You should get these kind of information after echo 0 & echo 1 to controllers/141c0000.pcie_ep/start
------dmesg----
[ 421.843994] tegra194-pcie 141a0000.pcie: host bridge /pcie@141a0000 ranges:
[ 421.844023] tegra194-pcie 141a0000.pcie: IO 0x003a100000…0x003a1fffff → 0x003a100000
[ 421.844030] tegra194-pcie 141a0000.pcie: MEM 0x2b28000000…0x2b2fffffff → 0x0040000000
[ 421.844033] tegra194-pcie 141a0000.pcie: MEM 0x2740000000…0x2b27ffffff → 0x2740000000
[ 421.951432] tegra194-pcie 141a0000.pcie: Link up
[ 421.951550] tegra194-pcie 141a0000.pcie: PCI host bridge to bus 0005:00

Q2. Actually, we couldn’t know the current version of FW.
But you can follow the page PCIe Retimer
to flash it to the latest version.

Q4. It looks correct. Did you hit any error messages when you executed these commands?