Stream management and interoperability between OpenACC and CUDA libraries

Accelerated Computing HPC Compilers nvc, nvc++ and nvfortran
1

Hello,

For a code ported with OpenACC and a CUDA library, I need to insure that the default OpenACC async queue and the default CUDA library stream are the same.

One way is to use acc_get_cuda_stream(acc_async_sync).

I’m wondering if the second advice found in this post (linking with “-cuda”) is still valid.

Is is still the case that the developer should manage explicitly the synchronization between OpenACC and CUDA libraries ?

2

Hi Olga and welcome!

Presuming you need the OpenACC and CUDA Library to use the same stream, then yes, you’ll want to use “acc_get_cuda_stream”. Example can be found at:

https://docs.nvidia.com/hpc-sdk/compilers/fortran-cuda-interfaces/index.html#using-cuda-libraries-from-openacc-host-code

-Mat

3

Thank you for reply Mat!

Is there a similar mechanism for OpenMP Target/CUDA libraries interoperability?

4

OpenMP is deficient here and doesn’t have a standard way to do this. Though, we did add an non-standard extension which you can try, “ompx_get_cuda_stream”.

For examples of it’s use, you can find these under the “<NVHPC_install_dir>/examples/CUDA-Libraries” directory. From there, look for the CUDA library you’re interested in and then under that, the “omp” directory. For example: “/opt/nvhpc/CUDA-Libraries/cuFFT/test_fft_omp_ftn/tcufft2dompf1.f90”

Hope this helps,
Mat