Simple assignment not parallelizing in 18.7 - worked in 18.4

caplanr · August 23, 2018, 6:28pm

Hello,

I just installed the new 18.7 compiler and am testing my OpenACC Fortran code on it.

The first problem is that the following code (which parallelized in every compiler 18.4 and before) no longer does:

      allocate (v_shear_t(ntm,np))
      allocate (v_shear_p(nt,npm))
c
!$acc enter data create(v_shear_t,v_shear_p)
!$acc kernels default(present)
      v_shear_t=0.
      v_shear_p=0.
!$acc end kernels

The compiler now spits out:

  10237, Generating enter data create(v_shear_t(:,:),v_shear_p(:,:))
  10238, Generating implicit present(v_shear_t(:,:),v_shear_p(:,:))
  10239, Loop carried dependence due to exposed use of v_shear_t(:,:) prevents parallelization
         Parallelization would require privatization of array v_shear_t(:,:)
         Accelerator serial kernel generated
         Accelerator kernel generated
         Generating Tesla code
      10239, !$acc loop seq

I REALLY do not want to have to expand all these statements into explicit loops as there are MANY of them.
Can you please submit this to the engineers for the next update?
Thanks,
Ron

MatColgrove · August 24, 2018, 2:17am

Hi Ron,

I tried to reproduce your issue here but it works fine for me. Can you please post a reproducing example or modify the one below to better show the error?

Thanks,
Mat

% cat test2.f90

function foo(nt,ntm,np,npm)

     integer :: nt,ntm,np,npm,i
     real(8) :: rc, foo
     double precision, dimension(:,:),allocatable :: v_shear_t, v_shear_p
     allocate (v_shear_t(ntm,np))
     allocate (v_shear_p(ntm,np))

!$acc enter data create(v_shear_t,v_shear_p)
!$acc kernels default(present)
      v_shear_t=0.
      v_shear_p=0.
!$acc end kernels

!$acc kernels loop
      do i=1,ntm
        do j=1,np
          rc = rc + v_shear_t(i,j)*v_shear_p(i,j)
        enddo
      enddo
      foo=rc
end function foo


sky4:/local/home/colgrove% pgf90 -c test2.f90 -ta=tesla:cc70 -Minfo=accel -V18.7
foo:
     10, Generating enter data create(v_shear_t(:,:),v_shear_p(:,:))
     11, Generating implicit present(v_shear_t(:,:),v_shear_p(:,:))
     12, Loop is parallelizable
         Accelerator kernel generated
         Generating Tesla code
         12, !$acc loop gang, vector(4) ! blockidx%y threadidx%y
             !$acc loop gang, vector(32) ! blockidx%x threadidx%x
     13, Loop is parallelizable
         Accelerator kernel generated
         Generating Tesla code
         13, !$acc loop gang, vector(4) ! blockidx%y threadidx%y
             !$acc loop gang, vector(32) ! blockidx%x threadidx%x
     16, Generating implicit copyin(v_shear_t(1:ntm,1:np),v_shear_p(1:ntm,1:np))
     17, Loop is parallelizable
     18, Loop is parallelizable
         Accelerator kernel generated
         Generating Tesla code
         17, !$acc loop gang, vector(128) ! blockidx%x threadidx%x
         18, !$acc loop gang ! blockidx%y
         19, Generating implicit reduction(+:rc)

caplanr · August 24, 2018, 5:10pm

Hi,
Using your test code I have figured out that the problem only happens when the arrays are from a module as follows:

PGI-2018: ~/Desktop/bugtest1 $ cat test3.f 
      module shear_profile
      real(8),dimension(:,:),allocatable :: v_shear_t, v_shear_p
      end module

      function foo(nt,ntm,np,npm)
      use shear_profile

      integer :: nt,ntm,np,npm,i
      real(8) :: rc, foo
      allocate (v_shear_t(ntm,np))
      allocate (v_shear_p(ntm,np))

!$acc enter data create(v_shear_t,v_shear_p)
!$acc kernels default(present)
      v_shear_t=0.
      v_shear_p=0.
!$acc end kernels

!$acc kernels loop default(present)
      do i=1,ntm
        do j=1,np
          rc = rc + v_shear_t(i,j)*v_shear_p(i,j)
        enddo
      enddo
      foo=rc

      end function foo 
PGI-2018: ~/Desktop/bugtest1 $ pgf90 -O3 -c test3.f -ta=tesla:cc60 -Minfo=accel
foo:
      0, Accelerator kernel generated
         Generating Tesla code
     15, Generating enter data create(v_shear_t(:,:),v_shear_p(:,:))
     16, Generating implicit present(v_shear_t(:,:),v_shear_p(:,:))
     17, Loop carried dependence due to exposed use of v_shear_t(:,:) prevents parallelization
         Parallelization would require privatization of array v_shear_t(:,:)
         Accelerator serial kernel generated
         Accelerator kernel generated
         Generating Tesla code
         17, !$acc loop seq
     18, Loop carried dependence due to exposed use of v_shear_p(:,:) prevents parallelization
         Parallelization would require privatization of array v_shear_p(:,:)
         Accelerator serial kernel generated
         Accelerator kernel generated
         Generating Tesla code
         18, !$acc loop seq
     21, Generating implicit present(v_shear_t(1:ntm,1:np),v_shear_p(1:ntm,1:np))
     22, Loop is parallelizable
     23, Loop is parallelizable
         Accelerator kernel generated
         Generating Tesla code
         22, !$acc loop gang, vector(128) ! blockidx%x threadidx%x
         23, !$acc loop gang ! blockidx%y
         24, Generating implicit reduction(+:rc)
PGI-2018: ~/Desktop/bugtest1 $

As a thought - have the rules of when it is necessary to use “declare” changed in OpenACC 3.0? As far as I understand, it is only needed if the array is being used in a function within a compute region. Is this correct?

caplanr · October 10, 2018, 7:05pm

Hi,

Any progress on this issue?

The lack of parallelization in this case is killing my code’s performance. Running my code using 18.7 is 25% slower than running it with 18.4.

Thanks!

Ron

MatColgrove · October 11, 2018, 3:35pm

Hi Ron,

Apologies that I missed you’re follow-up post. I was flying back from China that day.

I just added a problem report (TPR#26335) and sent it off to engineering.

Unfortunately I don’t have a good work around for you other than to make these explicit loops instead of using array syntax. If there are only a couple of arrays, it should be easy to make the switch, but if there are many, it will be a pain.

-Mat

% cat ron.F90
      module shear_profile
      real(8),dimension(:,:),allocatable :: v_shear_t, v_shear_p
      end module

      function foo(nt,ntm,np,npm)
      use shear_profile

      integer :: nt,ntm,np,npm,i
      real(8) :: rc, foo

      allocate (v_shear_t(ntm,np))
      allocate (v_shear_p(ntm,np))

!$acc enter data create(v_shear_t,v_shear_p)
!$acc kernels default(present)
      do i=1,ntm
        do j=1,np
          v_shear_t(i,j)=0.
          v_shear_p(i,j)=0.
        enddo
      enddo
!$acc end kernels

!$acc kernels loop default(present)
      do i=1,ntm
        do j=1,np
          rc = rc + v_shear_t(i,j)*v_shear_p(i,j)
        enddo
      enddo
      foo=rc

      end function foo

% pgf90 -ta=tesla:cc70 -Minfo=accel -c ron.F90 -V18.7
foo:
     14, Generating enter data create(v_shear_t(:,:),v_shear_p(:,:))
     15, Generating implicit present(v_shear_p(1:ntm,1:np),v_shear_t(1:ntm,1:np))
     16, Loop is parallelizable
     17, Loop is parallelizable
         Accelerator kernel generated
         Generating Tesla code
         16, !$acc loop gang, vector(128) ! blockidx%x threadidx%x
         17, !$acc loop gang ! blockidx%y
     24, Generating implicit present(v_shear_t(1:ntm,1:np),v_shear_p(1:ntm,1:np))
     25, Loop is parallelizable
     26, Loop is parallelizable
         Accelerator kernel generated
         Generating Tesla code
         25, !$acc loop gang, vector(128) ! blockidx%x threadidx%x
         26, !$acc loop gang ! blockidx%y
         27, Generating implicit reduction(+:rc)

MatColgrove · January 3, 2019, 10:46pm

Hi Ron,

I got an update for engineering:

Although the allocatable array assignment within an accelerator region is mostly reverting to Fortran 95 behavior in device code, we are still seeing a performance impact (compiler fails to parallelize the code) due to the compiler created allocate, reallocate, deallocate, or conformability calls. We are actively working on the fix for the future releases.

For more information about Fortran 2003 Allocatable Arrays please see:

-Mat

caplanr · January 4, 2019, 2:48am

Thanks for the update.

Great blog post!

I have changed all my “a=b” statements to a(:,:,:)=b(:,:,:) and it looks like the code is parallelizing all statements.

aglobus1 · July 2, 2019, 4:16pm

FYI for others coming to this thread, the issue should be fixed in versions 19.5 and above.

Topic		Replies	Views
Complex loop that worked in 18.4 not accel in 18.7 Legacy PGI Compilers (archived)	2	1703	August 24, 2018
Vector array assignments within a $acc parallel region Legacy PGI Compilers (archived)	13	11166	November 27, 2013
Array assignment in OpenACC using dummy indices Legacy PGI Compilers (archived)	2	2860	August 21, 2015
Privatization of array Legacy PGI Compilers (archived)	9	17800	July 14, 2010
Dealing with allocatable arrays with OpenACC Legacy PGI Compilers (archived)	8	2203	November 30, 2020
Parallelizing a loop Legacy PGI Compilers (archived)	9	5648	March 1, 2016
The result changes, after adding the openacc statement Legacy PGI Compilers (archived)	11	4077	July 21, 2018
accelerator parallization issues Legacy PGI Compilers (archived)	18	26990	April 12, 2010
Code execution depends strangely on irrelevant parameters Legacy PGI Compilers (archived)	8	8275	October 22, 2013
understanding problems with acc directives. Legacy PGI Compilers (archived)	7	12848	May 3, 2010

Simple assignment not parallelizing in 18.7 - worked in 18.4

Related topics