Do concurrent + openacc pragmas parallelism behavior

Accelerated Computing HPC Compilers nvc, nvc++ and nvfortran
1

Hello,

For a program in which I am using do concurrent to accelerate my code on the GPUs I wanted to add OpenACC pragmas to add asynchronous behaviour. To do this, I resorted to simply:

!$acc parallel loop async(1) 
do concurrent(j=1:nj, i=1:ni)

end do

However, this resulted in the do concurrent only getting parallelized over vectors instead of gangs and vectors. This made the code really slow.

Here is a sample reproducible (using nvhpc 26.1)



  !$acc parallel loop collapse(2)
  do j =1, dims ; do i = 1,dims
    c(i,j) = alpha * a(i,j) + b(i,j)
  end do ; end do

  !$acc parallel loop
  do concurrent (j=1:dims, i=1:dims)
    c(i,j) = alpha * a(i,j) + c(i,j)
  end do

When compiled, you will see:

     31, Generating NVIDIA GPU code
         32,   ! blockidx%x threadidx%x collapsed
             !$acc loop gang, vector(128) collapse(2) ! blockidx%x threadidx%x
     36, Generating NVIDIA GPU code
         37, !$acc loop vector(128) ! threadidx%x
             !$acc loop seq

Is there a way to retain the do concurrent and use openacc directives ?

Cheers

2

Hi jg4,

Use “kernels” instead of “parallel”. “parallel” only applies to explicit loops, while with “kernels” the compiler can discover parallelism hence can apply parallelism across both do concurrent dimensions.

% cat test.F90
subroutine test(A,B,C,dims,alpha)

  real(8) :: A(:,:), B(:,:), C(:,:)
  integer :: i, j, dims
  real(8) :: alpha

  !$acc data present(A,B,C)
  !$acc parallel loop collapse(2)
  do j =1, dims ; do i = 1,dims
    c(i,j) = alpha * a(i,j) + b(i,j)
  end do ; end do

  !$acc kernels loop collapse(2)
  do concurrent (j=1:dims, i=1:dims)
    c(i,j) = alpha * a(i,j) + c(i,j)
  end do
  !$acc end data

end subroutine test
% nvfortran -c test.F90 -acc -Minfo=accel
test:
      7, Generating present(a(:,:),c(:,:),b(:,:))
      8, Generating implicit firstprivate(dims)
         Generating NVIDIA GPU code
          9,   ! blockidx%x threadidx%x collapsed
             !$acc loop gang, vector(128) collapse(2) ! blockidx%x threadidx%x
      9, Generating implicit firstprivate(alpha)
     14, Loop is parallelizable
         Generating NVIDIA GPU code
         14,   ! blockidx%x threadidx%x collapsed
             !$acc loop gang, vector(128) collapse(2) ! blockidx%x threadidx%x

Hope this helps,
Mat