PCIe Replay Timer Timeout

Autonomous Machines Jetson & Embedded Systems Jetson TX1
#1

Hi

We have a custom carrier board with a Renesas uPD720202 USB3.0 controller connected to pcie0l0, the device is detected and does function be we get a lot of warnings reported:

[  230.833890] pcieport 0000:00:01.0: AER: Corrected error received: id=0010
[  230.840702] pcieport 0000:00:01.0: PCIe Bus Error: severity=Corrected, type=Data Link Layer, id=0008(Transmitter ID)
[  230.851228] pcieport 0000:00:01.0:   device [10de:0fae] error status/mask=00001000/00002000
[  230.859641] pcieport 0000:00:01.0:    [12] Replay Timer Timeout  
[  230.865775] pcieport 0000:00:01.0: AER: Corrected error received: id=0010
[  230.873476] pcieport 0000:00:01.0: AER: Corrected error received: id=0010
[  230.880298] pcieport 0000:00:01.0: PCIe Bus Error: severity=Corrected, type=Data Link Layer, id=0008(Transmitter ID)
[  230.908159] pcieport 0000:00:01.0:   device [10de:0fae] error status/mask=00001000/00002000
[  230.925932] pcieport 0000:00:01.0:    [12] Replay Timer Timeout

I initially thought theses errors were due to the PCIe routing/connections but because they are reported at the Data Link Layer I am less inclined to think that, is that correct?

Thanks

K.

#2

I think these are because of issues with USB3.0 controller’s ASPM working. Please disable ASPM completely. This can be done either by adding ‘pcie_aspm=off’ to kernel command line or by disabling ASPM config.

#3

Thanks,

ASPM is not enabled in the kernel:

# CONFIG_PCIEASPM is not set

but I don’t know if it has been enabled by u-boot, I’ll check.

#4

Hi kyler,

Have you clarified whether this case by ASPM?
Any update?

Thanks

#5

Hi,

ASPM is disabled in our kernel, so I don’t think that can be the cause.

I am going to purchase a PCIe card with a uPD720202 USB controller to try in the Jetson TX1 dev kit, I have tried a card with a uPD720200 controller and see no problems.

I am also going to build another set of out hardware to see if the problem is related to this specific board.

Any other suggestions greatly received.

Thanks

Kyle.

#6

We have verified with this particular USB3.0 controller (without ASPM enabled) and didn’t see any issue. To me, it looks mostly likely the carrier board issue. It is worth trying to use Nvidia’s base board once to confirm.

#7

I too have tested this controller with the TX1 dev kit and see no issues.
I also changed the pcie speed to Gen1 on our hardware (cat /sys/kernel/debug/pcie/apply_link_speed) and see no issues, so it’s probably a layout issue with our hardware.

Thanks for your help.

#8

I was seeing the same problem on our custom board and it turned out to be missing DC blocking caps. If you are going to attach a PCIe device directly to the Jetson TX1/TX2, you need to remember to put 0.1uF DC blocking caps on BOTH the TX and RX signal pairs. We put the ones on the TX pairs coming from the Jetson (as shown in the OEM Product Design Guide) but forgot about the other pair being driven by our PCIe device. It’s easy to forget the the caps are normally placed on the add-in card TX lines but when you put the device directly on the carrier board, it can missed.