We are planning to connect PCIe 2.0 based USB card (x1) to the AGX Xavier’s PCIe port. We could note that the maximum bandwidth supported for PCIe up to Gen3 is 5Gbps as per the OEM DG.
Could you please let us know if AGX Xavier can support PCIe Gen 2 speeds up to 5Gbps when operating with PCIe 2.0 cards?
According to Wikipedia, PCIe 3.0 could support speeds up to 8GT/s (8Gbps) which is higher than AGX Xavier’s specification. Is there an internal limitation in AGX Xavier or am I missing something here?
Please help us with this query.
Thanks
Xavier AGX should fully support gen. 3 (technically maybe even gen. 4, but I’ve never seen it obtain this spec). The most common issue is that signal quality must be good, and if not, then the bus will throttle back to a slower generation spec. Aside from this, whatever produces and consumes the data could bottleneck and make it appear the bus is underperforming.
Also, keep in mind that those bandwidths are not the data rates you will see. Those are the theoretical maximums not including overhead. Overhead itself can be significant, e.g., slower generations have an 8b/10b encoding scheme, and faster generations have a 128b/130b encoding scheme. That would mean needing to send 10 bits to receive 8 bits, or 130 bits to receive 128 bits, and that is just encoding overhead (there could be other overhead in addition to this).
Thank you for the detailed reply & the contributing factors @linuxdev
However, my prime concern here is whether AGX Xavier is capable of 5Gbps raw bandwidth(ignoring protocol, encoding overhead and bottlenecks) in PCIe 2.0 mode? There seems to be no explicit mention of the PCIe 2.0 case in OEM DG.
As mentioned in the original post, AGX Xavier supports only 5GT/s for PCIe Gen 3 which is lower than the actual PCIe 3.0 spec (8Gbps). My concern here is if there is any similar difference for PCIe 2.0 mode?
One does not need to specifically handle PCIe v2 unless you want to forcibly reduce the speed when higher speeds are available. PCIe will “link train” and pick the fastest protocol the chain of devices can handle. I would expect that if conditions are correct:
- A PCIe v4 link would run at PCIe v4.
- A PCIe v3 link would run at PCIe v3.
- A PCIe v2 link would run at PCIe v2.
- A faster link, such as PCIe v3, if signal quality is not sufficient, would run at v2 or v1 speed (depending on what the signal quality supports), or to go completely missing if v1 speed cannot be achieved due to signal quality issues. The speed is not programmed, it is chosen by training.
The link clocks and speeds are part of the speed being run at, e.g., PCIe v2 link clocks and timings will run where expected for that given link speed. Whether data throughput actually achieves this is quite a different story, but the actual clocks and raw bit rates should be exactly what the spec claims. Something which might throw this off:
- The CPU servicing the driver interrupt is busy at times when PCIe wants to talk;
- The CPU servicing the driver is working well, and nearly always available, but the software consuming or producing the data has a bottleneck (e.g., perhaps it is storing or retrieving from a hard drive which is much slower than the link).
- If something is inefficient in data transfer, e.g., as a contrived example, if data alignment must be a certain way for DMA, but the data is not aligned, and thus some conversion might need to be performed (which consumes time) prior to the DMA taking place. This is a wide open book of things which might stall out the PCIe even if it is working correctly.
You basically have to try the hardware to know what it will do. Identifying what is slowing it down is not necessarily an easy task, and even if you do know what the slow-down is (if that occurs), it may not be easy to fix it. On the other hand, if you don’t have the ability to test for your specific hardware, then there is absolutely no way to truly answer.
@n.balaji
5Gbs
you may have over LAN [ethernet wire]
on custom carrier boards
AV
I was going through the technical reference manual of Xavier SoC and I was able to notice the below mentioned in the document. Can NVIDIA support let us know the difference in speeds in x4/x8 controllers used as x1 in comparison to directly using x1 controllers?
We may have x1 PCIe Gen 2.0 to USB 3.0 chips connected to both of the below PCIe port (refer image) and we would like to understand the impact due to this.
