Detecting out of memory with OpenMP target

Accelerated Computing HPC Compilers nvc, nvc++ and nvfortran
1

Hello all,

Is there a way to detect OOM when allocating memory with:

#pragma omp target enter data map(alloc:ptr[0:size])

I didn’t succeed with a try/catch, it seems that it exits without throwing an exception…

Out of memory allocating 1977963936 bytes of device memory
Failing in Thread:1
total/free CUDA memory: 85007794176/1577975808
Present table dump for device[1]: NVIDIA Tesla GPU 0, compute capability 8.0, threadid=1
...empty...
call to cuMemAlloc returned error 2: Out of memory
Out of memory allocating 1978018512 bytes of device memory

I ask this, cause in my MPI code, only one process has a OOM, and the code hangs unhappily.

Thanks,

Pierre

2

Hi Pierre,

Not with a data directive. You’d need to switch to using “omp_target_alloc” and then check that the return code is not NULL.

Note that we had an issue where we weren’t returning NULL, aborting instead, but was corrected in our 23.7 release.

-Mat

1 Like
3

Thanks Mat for the info :-)

I am trying now to detect OOM with cudaMemGetInfo/hipMemGetInfo before allocation.