Dual port 100G on Pci3.0 bus - bandwidth

I’m confused about the speed of the ports of some Mellanox cards and the PCIe slot bandwidth they are installed on.

As I understand it a single 100G port is full duplex right ? Meaning you could have 100 + 100 G of traffic going on both directions.

How can you have a ConnectX-5 100Gbit dual port card sitting on PCIe 3.0 x16 slots ? PCIe 3.0 x16 can give you ~128Gbit of bandwidth so how is it possible to use the full bandwidth of this card on such slots ? As bidirectionally it could push 200 + 200 G right ?

I’m probably missing something obvious so asking for help.


or the MCX414A-GCAT for that matter, dual port 50Gbit on x8 PCIe 3.0 which should have up to ~64Gbit of bandwidth…

@dropbrick: While looking for information on a similar topic, I found the following page: Specifications - ConnectX-5 InfiniBand/VPI - NVIDIA Networking Docs
Quoting from there:

Note: PCIe 3.0 x16 bus can supply a maximum bandwidth of 128Gb/s only (=16 * 8GT/s, including overhead), and therefore cannot support 200Gb/s when both network ports of MCX556A-ECAT run at 100Gb/s.