Credit flow control value for PCIe

Hi,
We have an FPGA system connected with Jetson Xavier NX on M.2 PCIe port. For some reason FPGA is not able to initiate the transaction with Jetson. It is suspected that the value of flow control credit could be the reason.
I have following questions:

  1. How can I verify that the flow control credit is enabled/disabled?
  2. How to know the flow control credit’s current value?
  3. How to set the value of flow control credit if possible? Which all register are responsible for that?

I tried to read PCIE_X4_EP_PF0_PORT_LOGIC_LANE_SKEW_OFF_0 register which has FLOW_CTRL_DISABLE field and PCIE_X4_RC_PF0_PORT_LOGIC_TX_P_FC_CREDIT_STATUS_OFF_0 which has TX_P_HEADER_FC_CREDIT and TX_P_DATA_FC_CREDIT fields using devmem2 utility but the board reboots on reading them.

These registers I tried reading were for the block PCIE_C4_CTL block which has memory mapped at 0x14160000.

Regards,
Meet

I did some research and found TX_P_HEADER_FC_CREDIT and TX_P_DATA_FC_CREDIT fields but it says its what receiver on the other end of the link advertises. I cant find anything related to what Jetson advertises. Would like to use some help here.

Thanks and regards,
Meet

Hi,

It is not mapped to 0x14160000, use setpci to read PCIe config space.

Thanks,
Manikanta

Hi Manikanta,
According to the manual it says PCIE_C4_CTL (where the Wifi chip is installed) is at

but anyway, the pcie slot that we are using for FPGA is PCIE_C5_CTL

Tried reading the values of all the said registers on that location too and got the same results, i.e., Jetson reboots on reading with devmem2.

Anyway, my main problem is to know the FC credits that Jetson advertises to FPGA.
So I decided to read with the pcie driver in nvidia/drivers/pci/dwc/pcie-tegra.c, I modified the function tegra_pcie_dw_host_init() to read the register values at the end of the function and got the following values for respective registers:

PORT_LOGIC_LANE_SKEW: 0x38000000
PORT_LOGIC_TX_P_FC_CREDIT_STATUS: 0x32166
PORT_LOGIC_TX_NP_FC_CREDIT_STATUS: 0x38fff
PORT_LOGIC_TX_CPL_FC_CREDIT_STATUS: 0xfffff
PORT_LOGIC_VC0_P_RX_Q_CTRL: 0x46258260
PORT_LOGIC_VC0_NP_RX_Q_CTRL: 0x625824e
PORT_LOGIC_VC0_CPL_RX_Q_CTRL: 0x6200000
RAS_DES_CAP_SD_STATUS_L2_REG: 0x0
RAS_DES_CAP_SD_STATUS_L3FC_REG_0: 0x0

Does the PORT_LOGIC_VC0_P_RX_Q_CTRL corresponds to the FC credits that Jetson advertises to FPGA? If not, which other register could potentially be related to it?

Thanks and regards,
Meet

Hi,

0x141a0000 is PCIe application register space, not configuration space.
setpci tool doesn’t need base address, you can read configuration space using BDF and offset.

PORT_LOGIC_TX_P_FC_CREDIT_STATUS: 0x32166 → Posted Request header and data credits advertised by Tegra
PORT_LOGIC_TX_NP_FC_CREDIT_STATUS: 0x38fff → Non posted Request header and data credits advertised by Tegra
PORT_LOGIC_TX_CPL_FC_CREDIT_STATUS: 0xfffff → Completion header and data credits advertised by Tegra

Thanks,
Manikanta

Hi,
Yes, 0x141a0000 is not the config space. What i meant was reading the register values of the PCIe application register space. Sorry, for not being clear.

Thank you for the clarification on credits advertised by Jetson.

Regards,
Meet

Hi,
Sorry, marking it as not solved yet.

I checked the values of these registers with another device (other than FPGA) and found that the values are different than what they are when FPGA is connected.

What our FPGA engineer told me is that the value advertised by the Jetson will be same for different devices connected to that PCIe slot.

I am confused now as to whether those registers (PORT_LOGIC_TX_P_FC_CREDIT_STATUS, PORT_LOGIC_TX_NP_FC_CREDIT_STATUS, PORT_LOGIC_TX_CPL_FC_CREDIT_STATUS) dictates to the Jetson advertising those credits to FPGA or its the other way around (FPGA advertising to Jetson).

The description of TX_P_DATA_FC_CREDIT is given as follows:
Transmit Posted Data FC Credits. The posted Data credits advertised by the receiver at the other end of the link, updated with each UpdateFC DLLP

Wouldn’t it mean the receiver on the other end of the link is FPGA?

Thanks and Regards,
Meet

Hi,

Following registers have the credit info advertised by Tegra PCIe RP.
PF0_PORT_LOGIC_VC0_P_RX_Q_CTRL_OFF_0
PF0_PORT_LOGIC_VC0_NP_RX_Q_CTRL_OFF_0
PF0_PORT_LOGIC_VC0_CPL_RX_Q_CTRL_OFF_0

Thanks,
Manikanta

Hi,
Thanks for the response.
So, in your last reply those registers you mentioned (PORT_LOGIC_TX_P_FC_CREDIT_STATUS, PORT_LOGIC_TX_NP_FC_CREDIT_STATUS, PORT_LOGIC_TX_CPL_FC_CREDIT_STATUS) are the credits that FPGA advertised to Jetson, right?
Can you please confirm this?

Thanks,
Meet

Correct

Hi,
Thank you!

Regards,
Meet

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.