explain output

Hi all!

could you please explain what those two lines mean:

Generating present_or_copyin(r_g$ac(:))
Generating present_or_copyin(r_r$ac(m))

In particular I’m interesting in parts “$ac(:)” and “$ac(m)”.

I have r_r and r_g vars declared as

REAL, DIMENSION(ntb_r), PARAMETER, PRIVATE:: &
r_r = (/1.e-6,2.e-6,3.e-6,4.e-6,5.e-6,6.e-6,7.e-6,8.e-6,9.e-6, &

REAL, DIMENSION(ntb_g), PARAMETER, PRIVATE:: &
r_g = (/1.e-5,2.e-5,3.e-5,4.e-5,5.e-5,6.e-5,7.e-5,8.e-5,9.e-5, &

Inside the loop I read an appropriate constants “r_g(i)”.

Hi Alexey,

This happens when the program has initialized data, such as this array parameter. To do this, the compiler creates and initializes a local array to the device.

  • Mat

Hi Mat!

Thank you.

Do you have any plans to put this data in constant memory instead of local?

Alexey

Do you have any plans to put this data in constant memory instead of local?

Not yet, but we’ll look into it. The problem is that constant memory is only 64k, so we’d need to have a whole program view of the total amount of data put into constant memory.

  • Mat

Hi Mat!

Is it true PGI copy data to GPU from FORTRAN DATA section each time at kernel launch? This behavior seems very strange. For example,

2163 !$acc data copyin(vr,vg,Dr,Dg,crg,cre,cgg,cge,dtr,dtg,N0r_exp,N0g_exp,r_r,r_g) &
2164 !$acc copyout(tcg_racg,tmr_racg,tcr_gacr,tmg_gacr,tnr_racg, &
2165 !$acc tnr_gacr) create(N_r)
2166       do km = km_s, km_e
2167          m = km / ntb_r1 + 1
2168          k = mod( km , ntb_r1 ) + 1
2169          lam_exp = (N0r_exp(k)*am_r*crg(1)/r_r(m))**ore1
2170          lamr = lam_exp * (crg(3)*org2*org1)**obmr
2171          N0_r = N0r_exp(k)/(crg(2)*lam_exp) * lamr**cre(2)
2172 !$acc kernels
2173          do n2 = 1, nbr
2174             N_r(n2) = N0_r*Dr(n2)**mu_r *DEXP(-lamr*Dr(n2))*dtr(n2)
2175          enddo

PGI reports

   2163, Generating create(n_r(:))
         Generating copyout(tnr_gacr(:,:,:,:))
         Generating copyout(tnr_racg(:,:,:,:))
         Generating copyout(tmg_gacr(:,:,:,:))
         Generating copyout(tcr_gacr(:,:,:,:))
         Generating copyout(tmr_racg(:,:,:,:))
         Generating copyout(tcg_racg(:,:,:,:))
         Generating copyin(dtg(:))
         Generating copyin(dtr(:))
         Generating copyin(cge(:))
         Generating copyin(cgg(:))
         Generating copyin(cre(:))
         Generating copyin(crg(:))
         Generating copyin(dg(:))
         Generating copyin(dr(:))
         Generating copyin(vg(:))
         Generating copyin(vr(:))
   2172, Generating present_or_copyin(dg(:))
         Generating present_or_copyin(dtg(:))
         Generating present_or_copyin(cgg(:))
         Generating present_or_copyin(n0g_exp$ac(:))
         Generating present_or_copyin(r_g$ac(:))
         Generating present_or_copyin(cge(:))
         Generating present_or_copyin(vr(:))
         Generating present_or_copyin(vg(:))
         Generating present_or_copyout(tcg_racg(:,:,:,:))
         Generating present_or_copyin(r_r$ac(m))
         Generating present_or_copyout(tmr_racg(:,:,:,:))
         Generating present_or_copyout(tcr_gacr(:,:,:,:))
         Generating present_or_copyout(tmg_gacr(:,:,:,:))
         Generating present_or_copyout(tnr_racg(:,:,:,:))
         Generating present_or_copyout(tnr_gacr(:,:,:,:))
         Generating present_or_create(n_r(:))
         Generating present_or_copyin(dtr(:))
         Generating present_or_copyin(dr(:))

there is no copyin record for r_r, r_g, n0g_exp vars on line 2163. As the result I have three memory transfer operations before kernel launch.

        2163: data copyin reached 10 times
             device time(us): total=51 max=6 min=5 avg=5
        2172: data copyin reached 4107 times
             device time(us): total=23,710 max=60 min=4 avg=5
        2173: kernel launched 1369 times

Any idea?

Thanks.

Hi Alexey,

Parameter arrays get initialized on the device so don’t need be copied. You should have seen a warning during compilation:

PGF90-W-0155-Constant or Parameter used in data clause -

I think what you might be seeing is the section descriptions for the arrays being copied over. The arrays themselves should already be on the device.

It’s a a lot of output, but you can set the environment variable “PGI_ACC_DEBUG=1” to see every device call. It might shed some light what variables are being copied.

  • Mat

Hi Mat,

I don’t see this warning. I use

$ pgfortran -V

pgfortran 13.2-0 32-bit target on x86 Linux -tp nehalem
Copyright 1989-2000, The Portland Group, Inc.  All Rights Reserved.
Copyright 2000-2013, STMicroelectronics, Inc.  All Rights Reserved.

What option should I add? I use “-Minfo”

Do you have the “-w” flag on your compile line? “-w” disables warning messages so could be why you don’t see it. Otherwise, I’m not sure and will need a reproducing example to better understand what’s going on.

  • Mat

Hi Mat!

You are right, I have “-w” option. Thank you.