# multiple gpu and unified memory

**URL:** <https://forums.developer.nvidia.com/t/multiple-gpu-and-unified-memory/47545>\
**Category:** CUDA Programming and Performance\
**Created:** [January 26, 2017, 10:22pm UTC](https://forums.developer.nvidia.com/t/multiple-gpu-and-unified-memory/47545 "2017-01-26T22:22:54Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![grynet](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@grynet](https://forums.developer.nvidia.com/u/grynet)\
**Post date:** [January 26, 2017, 10:22pm UTC](https://forums.developer.nvidia.com/t/multiple-gpu-and-unified-memory/47545/1 "2017-01-26T22:22:54Z")

</div>

Hello,

I am wondering how unified memory works with multiple gpu systems on nvlink and non-nvlink based systems. I provided a simple code. In that case, I have 4 gpus and I am binding a thread with gpu using OpenMP here. I guess, openmp makes kernel submission concurrent to different gpus. I am doing some calculating with different address of same array “a” and “b”.

My questions are that:

- Is that code valid for both systems ? 
- Will cuda runtime does P2P GPU transfer for array "a", even if there is only read here ? 
- Will cuda runtime does P2P GPU transfer for array "b" ? 

Thanks in advance

```auto
__global__ void kernel(int *a, int *b, int n) {
  int tid = ...
  b[tid] = a[tid];
} 

int main(int argc, char const *argv[]){
  int *a,*b, n=65536;
  cudaMallocManaged(a, n*sizeof(int));
  cudaMallocManaged(b, n*sizeof(int));

  int num_of_gpu = 4;
  
  #pragma omp parallel for
  for (int i = 0; i < num_of_gpu; ++i) {
    cudaSetDevice(i);
    kernel<<< 1, 1024 >>>(&a[(n/num_of_gpu)*i], &b[(n/num_of_gpu)*i], n/num_of_gpu);
  }
  return 0;
}

```

---

<div class="post-metadata">

**Author:** ![Robert\_Crovella](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/robert_crovella/32/14043_2.png) [@Robert\_Crovella](https://forums.developer.nvidia.com/u/Robert_Crovella)\
**Post date:** [February 2, 2017, 3:41am UTC](https://forums.developer.nvidia.com/t/multiple-gpu-and-unified-memory/47545/2 "2017-02-02T03:41:48Z")

</div>

You may want to read the documentation:

[url][Programming Guide :: CUDA Toolkit Documentation](http://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#um-unified-memory-programming-hd%5B/url%5D)

[url][http://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#um-advanced-topics-hd[/url]](http://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#um-advanced-topics-hd%5B/url%5D)

There’s not enough information in your question to answer it. nvlink doesn’t make code that was previously valid become “invalid”.

---

<div class="post-metadata">

**Author:** ![Daniel\_Wong](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/daniel_wong/32/60786_2.png) [@Daniel\_Wong](https://forums.developer.nvidia.com/u/Daniel_Wong)\
**Post date:** [March 29, 2022, 9:35pm UTC](https://forums.developer.nvidia.com/t/multiple-gpu-and-unified-memory/47545/3 "2022-03-29T21:35:57Z")

</div>

**How to allocate unified memory that can use all available GPU memory?**  
I currently found that unified memory will only consider one GPU when calling `cudaMallocManaged` even we have multiple GPUs on the system.

---

<div class="post-metadata">

**Author:** ![Robert\_Crovella](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/robert_crovella/32/14043_2.png) [@Robert\_Crovella](https://forums.developer.nvidia.com/u/Robert_Crovella)\
**Post date:** [March 29, 2022, 9:44pm UTC](https://forums.developer.nvidia.com/t/multiple-gpu-and-unified-memory/47545/4 "2022-03-29T21:44:07Z")

</div>

It can’t be done. There is no facility in CUDA to have a single pointer allocation, part of which is associated with one GPU and part of it is associated with another GPU.

Having said that, you can allocate unified memory that is larger than the memory available on a single GPU. That requires a oversubscription ready system (pascal or newer GPU, and linux) with sufficient host memory, and in a multi-GPU setup there are additional issues to consider. These issues depend on system configuration (whether the GPUs are on the same fabric, or not). You can get insight by reading the managed memory section of the programming guide. I’ve already provided the links above.
