Data transfer between two GPUs on Tesla K80

Hi All,

Does anyone know the data transfer speed between two 12 GB memory on Tesla K80?

I am wondering whether it is faster than the data transfer speed between two K40s on the same motherboard? I am considering which card I should buy.

Many thanks for your suggestions and answers!

Transfers between the 2 GPUs on a K80 should run at Gen3 speeds.

Two K40’s will depend on what kind of motherboard and what kind of slot(s) they are plugged into.

Hi txbob,

Many thanks for your response. I really appreciate it.

My question actually is: whether K80 could get extra data transfer rate bonus by integrating two GPUs on the same card?

I read an introduction by Jan Smith from Microway. The author claims that the using of PCIe switch boosts the data transfer speed to 16 times. I am not sure whether that also means the data transfer between two GK210 GPUs (K80) is faster than that of two K40 GPUs in the ideal case.

Because of the special computation work I am doing, I care much about the computing speed and also the data transfer between two GPUs. Since GK110B GPU on K40 is better in performance comparing with a single GK210 GPU on K80, if we can’t get a data transfer rate bonus from K80, whether just buying two K40 would be a better choice?

http://www.microway.com/hpc-tech-tips/introducing-nvidia-tesla-k80-gpu-accelerator-kepler-gk210/

Nowhere in the article you linked does the author claim this: " The author claims that the using of PCIe switch boosts the data transfer speed to 16 times. " That is not stated in the text anywhere.

Yes, there is a diagram with “x16” on it. You are misinterpreting that. That is is referring to the fact that each PCIE link from each GK210 GPU to the on-board PCIE switch is a “x16” link, which is common parlance to indicate that it has 16 lanes for transfer. This is typical, and not unique to K80, or GPUs that exist in a 2-GPU per board configuration.

The PCIE switch on that board is a Gen3 switch, which can support Gen3 transfers between the two on-board GK210 GPUs, regardless of the type of system or slot it is plugged into. There is no “bonus” beyond a Gen3 transfer rate.

Again, the same statement cannot be made about two K40s. The transfer rate between the two K40s could be at PCIE Gen3 transfer rate, if the two K40s are plugged into a qualified system that supports PCIE Gen3 slots. But it will depend on the system they are plugged into. The transfer rate could be lower, for example if they are plugged into PCIE Gen2 slots.

You may want to read the wikipedia article on PCI-Express for understanding of various terms, like “lane”, “x16”, “Gen2”, and “Gen3” (Gen2 == PCI Express 2.0 or v2)

http://en.wikipedia.org/wiki/PCI_Express

Thank you, txbob.

I really appreciate your time and introduction. That makes me clear. Thanks again! ^_^