From NIC to GPU.

[Alexey_Skidanov]

Peter, so far option (B) is most practical. That is, allocate pinned, mapped memory using cuMemHostAlloc() and pass it down into your kernel driver. There, you can retrieve the physical address (using find_vma() on linux) and initialize DMA as neccessary.

I have been using this method for FPGA → GPU pipelining, where the FPGA is actually an advanced 10GbE NIC that does preliminary image processing on incoming video streams. The performance gains have been very good (greatly reduced latency and CPU usage), but it is fairly unstable and crashes the system often. I think the NV driver changes the mapping or something while the FPGA is still writing to it.

The very top ‘Release Highlight’ of Cuda 3.1 is ‘GPUDirect™ gives 3rd party devices direct access to CUDA Memory’. However, there is absolutely no reference to this in the toolkit itself. Anyone care to clarify?

I’ve tried thid method. It works but unstable: sometimes I get a computer freeze.

[kevleyski]

Wondering if this got implemented

Have a good use case for NIC into GPU compute shader and back out to the NIC

[Robert_Crovella]

@kevleyski

Wondering if this got implemented

I’m not sure what “this” refers to. However this thing that already got mentioned:

@sudoer

the recent announcement of an Mellanox Infiniband card that can DMA to a Fermi GPU directly.

is a real thing. It is referred to as GPUDirect RDMA. It can be done with either Infiniband or (RoCE) Ethernet.