I am using an TX2 NX module plugged into a standard NX developer carrier board. I have an FPGA-based PCIe device (our own design) connected to the 2-lane PCIe M.2 Key M slot on the carrier. This device works well with the same carrier and a Xavier NX module and also with a Xavier AGX system (both with 4 lane PCIe). However, with the TX2 NX we are unable to stream data properly.
It appears as if the TX2 NX provides very few buffers for flow control compared with the Xavier NX or AGX. The initial flow control packets captured with our analyser (using PCIe Gen1, because we don’t have a Gen 2 analyser) show that the AGX advertises:
127 credits for posted write headers and 608 credits for data.
For the TX2 NX these numbers are much smaller:
8 credits for posted write headers and 48 credits for data.
Until now our device refuses to write data to CPU memory if the advertised credits are too small, which explains why we cannot stream data on the TX2 NX. Of course, we can reduce the required credit level in our device and it works again.
So my questions are: why are these numbers so much smaller on the TX2 NX than the AGX and is there anyway to reconfigure the PCIe bus / root complex / bridge to provide more buffers and advertise a higher initial credit level?