# GPU Direct 1 and 2

**URL:** <https://forums.developer.nvidia.com/t/gpu-direct-1-and-2/22496>\
**Category:** CUDA Programming and Performance\
**Created:** [May 18, 2011, 6:19am UTC](https://forums.developer.nvidia.com/t/gpu-direct-1-and-2/22496 "2011-05-18T06:19:30Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![ikidntu](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@ikidntu](https://forums.developer.nvidia.com/u/ikidntu)\
**Post date:** [May 18, 2011, 6:19am UTC](https://forums.developer.nvidia.com/t/gpu-direct-1-and-2/22496/1 "2011-05-18T06:19:30Z")

</div>

In NVIDIA GPU direct section:

> **[GPUDirect](https://developer.nvidia.com/gpudirect)**

> [@](#):
>
> GPUDirect v2.0 adds support for peer-to-peer (P2P) communication between GPUs over PCIe in the same system, and lays the foundation for P2P communication between GPUs and other devices in a future release.

It appears to me that GPU Direct 2.0 include GPU Direct 1.0 features ( it said “adds support” ).

Is this true ?

- If yes, How could I test my MPI to see whether the feature of GPU Direct 1.0 is there ? I couldn’t figure out anything from the example code i.e mpi\_pinned.c in the package.

- If no, Is it possible to install the driver in the website together with CUDA 4.0 driver? Because the file name “nvidia-gpudirect-3.2-1.tar.gz” suggests driver 3.2.

Thank you

---

<div class="post-metadata">

**Author:** ![ikidntu](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@ikidntu](https://forums.developer.nvidia.com/u/ikidntu)\
**Post date:** [May 19, 2011, 1:25am UTC](https://forums.developer.nvidia.com/t/gpu-direct-1-and-2/22496/2 "2011-05-19T01:25:29Z")

</div>

Could anyone enlighten me on this issue ? Thanks a lot

---

<div class="post-metadata">

**Author:** ![mfatica](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@mfatica](https://forums.developer.nvidia.com/u/mfatica)\
**Post date:** [May 19, 2011, 4:21am UTC](https://forums.developer.nvidia.com/t/gpu-direct-1-and-2/22496/3 "2011-05-19T04:21:26Z")

</div>

P2P has nothing to do with the old GPU direct patch ( needed only to use RDMA with the Infiniband driver on Linux).

With 4.0 final, the GPU direct patch will be obsolete, just wait a little bit.

---

<div class="post-metadata">

**Author:** ![tmurray](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@tmurray](https://forums.developer.nvidia.com/u/tmurray)\
**Post date:** [May 19, 2011, 5:04am UTC](https://forums.developer.nvidia.com/t/gpu-direct-1-and-2/22496/4 "2011-05-19T05:04:11Z")

</div>

> [@](#):
>
> With 4.0 final, the GPU direct patch will be obsolete, just wait a little bit.

you always spoil my surprises :(

---

<div class="post-metadata">

**Author:** ![avidday](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/avidday/32/59219_2.png) [@avidday](https://forums.developer.nvidia.com/u/avidday)\
**Post date:** [May 19, 2011, 5:07am UTC](https://forums.developer.nvidia.com/t/gpu-direct-1-and-2/22496/5 "2011-05-19T05:07:53Z")

</div>

Yeah that is twice he has done that :)

---

<div class="post-metadata">

**Author:** ![ikidntu](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@ikidntu](https://forums.developer.nvidia.com/u/ikidntu)\
**Post date:** [May 19, 2011, 5:35am UTC](https://forums.developer.nvidia.com/t/gpu-direct-1-and-2/22496/6 "2011-05-19T05:35:21Z")

</div>

Yeah, Thanks guys! I will wait for the final release.

Hopefully It also supports “enhanced mode” of MPI i.e MPI\_Send and MPI\_Recv directly on cuda devices

Cheers !

---

<div class="post-metadata">

**Author:** ![ikidntu](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@ikidntu](https://forums.developer.nvidia.com/u/ikidntu)\
**Post date:** [May 27, 2011, 1:51am UTC](https://forums.developer.nvidia.com/t/gpu-direct-1-and-2/22496/7 "2011-05-27T01:51:49Z")

</div>

CUDA 4.0 Final release is out however I couldn’t find anywhere mentioned GPU-Direct 1.0. Is the claim previously still correct ?

---

<div class="post-metadata">

**Author:** ![mfatica](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@mfatica](https://forums.developer.nvidia.com/u/mfatica)\
**Post date:** [May 27, 2011, 2:36am UTC](https://forums.developer.nvidia.com/t/gpu-direct-1-and-2/22496/8 "2011-05-27T02:36:58Z")

</div>

With the 4.0 driver, just set this variable:

export CUDA\_NIC\_INTEROP=1

This will enable an alternate path for cudaMallocHost that is compatible with the IB drivers.  
No need to change your code to use cudaHostRegister or to install the GPU direct v1 patch.

---

<div class="post-metadata">

**Author:** ![ikidntu](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@ikidntu](https://forums.developer.nvidia.com/u/ikidntu)\
**Post date:** [June 15, 2011, 4:06am UTC](https://forums.developer.nvidia.com/t/gpu-direct-1-and-2/22496/9 "2011-06-15T04:06:54Z")

</div>

Thank mfatica, It helps a lot.  
Performance over MPI improves about 30%

---

<div class="post-metadata">

**Author:** ![ikidntu](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@ikidntu](https://forums.developer.nvidia.com/u/ikidntu)\
**Post date:** [June 15, 2011, 4:06am UTC](https://forums.developer.nvidia.com/t/gpu-direct-1-and-2/22496/10 "2011-06-15T04:06:54Z")

</div>

Thank mfatica, It helps a lot.  
Performance over MPI improves about 30%
