"error: this kind of pragma may not be used here" Why?

Accelerated Computing HPC Compilers Legacy PGI Compilers
#1

Hello. I have a code, which worked fine with pgc++ 19.10. But now nvc++21.9-0 does not compile it and throws an error:

/data/70-gaa/Desktop/CURRENT_WORK/tpt3_current_work/tpt/include/T3DataHolder.h", line 1597: error: this kind of pragma may not be used here
  #pragma acc parallel loop gang vector copy(count01,count23,init,fin) present(particles,ind23)
                                                                                               ^
          detected during instantiation of "void DataHolder<Floating>::Compress() [with Floating=t3::FloatingType]" at line 346 of "/data/70-gaa/Desktop/CURRENT_WORK/tpt3_current_work/main.cpp"

Use the compilation line:

cmake . -DCMAKE_CXX_COMPILER=nvc++ -DCMAKE_CXX_FLAGS="-acc=gpu -Minfo=acc -tp=haswell -Minline -cuda -gpu=cc70"

Compile on Intel KNL with GPU Titan V installed in it.
The piece of code looks like:

constexpr unsigned int GL=10000000;
constexpr unsigned int Nbin = 8;
constexpr unsigned int BLt = GL/Nbin;
int ind01[Nbin][BLt];
int ind23[Nbin][BLt];
int count01[Nbin];
int count23[Nbin];
int init[Nbin];
int fin[Nbin];
...
//this is the line #1597:
#pragma acc parallel loop gang vector copy(count01,count23,init,fin) present(particles,ind23)
    {
     for(int b=0; b<Nbin; ++b)
     {
       for(int i=init[b]; i<fin[b]; ++i)
       {
         if(particles[i].ir<2) ind23[b][count01[b]--]=i;
         else                  ind23[b][count23[b]++]=i;
       }
    }
  }

particles - array of structures.
Please, be so kind to help me fix this issue.

#2

Hi _and,

I’m not sure why it worked before, but the “loop” directive needs to be placed just before a for loop. Hence you’ll want to remove the extra “{ }” or move the “loop” directive inside of the “{”.

Either:

#pragma acc parallel loop gang vector copy(count01,count23,init,fin) present(particles,ind23)
     for(int b=0; b<Nbin; ++b)
     {
       for(int i=init[b]; i<fin[b]; ++i)
       {
         if(particles[i].ir<2) ind23[b][count01[b]--]=i;
         else                  ind23[b][count23[b]++]=i;
       }
  }

or

#pragma acc parallel copy(count01,count23,init,fin) present(particles,ind23)
    {
#pragma loop gang vector
     for(int b=0; b<Nbin; ++b)
     {
       for(int i=init[b]; i<fin[b]; ++i)
       {
         if(particles[i].ir<2) ind23[b][count01[b]--]=i;
         else                  ind23[b][count23[b]++]=i;
       }
    }
  }

Hope this helps,
Mat

#3

Thank You very much for the answer.
I also have a code snippet:

#include "T3Inelasticdd_DB.hh"

struct Particle{
  double m[16];
};

constexpr unsigned int GL=5000000;
constexpr unsigned int Nbin = 8;
constexpr unsigned int BLt = GL/Nbin;

Particle particles[GL];
int ind01[Nbin][BLt];
int ind23[Nbin][BLt];
Particle arr1[GL];
Particle arr2[GL];
Particle arr3[GL];
double csBorderDataFS[GL];
long int MAX_ELEMENT;
unsigned int SHIFT;
int Ntop=0;
long int INJECTED_PARTICLES=0;
int LIFE=0;  
constexpr int SKA_BINS_NUMBER=1000;
double SKA_Tls[SKA_BINS_NUMBER];

int main(int argc, char **argv)
{
#ifdef OPENACC
#pragma acc data create(ind01,ind23,arr1,arr2,arr3,csBorderDataFS) \
  copyin(particles) \
  copy(SKA_Tls[0:SKA_BINS_NUMBER])
  {
#endif
  }
}

Use nvc++ and the compilaion flags:

-DCMAKE_CXX_FLAGS="-acc=gpu -Minfo=acc -tp=haswell -Minline -cuda -gpu=cc70"

The compilation error is:

[7%] Linking CXX executable Test
CMakeFiles/Test.dir/main.cpp.o: in function `__sti___51__data_70_gaa_Desktop_CURRENT_WORK_MinRepEx_main_cpp_Ntop':
nvc++3wfeL30c9dld.ll:(.text+0x913): relocation truncated to fit: R_X86_64_PC32 against symbol `__I___51__data_70_gaa_Desktop_CURRENT_WORK_MinRepEx_main_cpp_Ntop' defined in .bss section in CMakeFiles/Test.dir/main.cpp.o
nvc++3wfeL30c9dld.ll:(.text+0x91e): relocation truncated to fit: R_X86_64_PC32 against symbol `__I___51__data_70_gaa_Desktop_CURRENT_WORK_MinRepEx_main_cpp_Ntop' defined in .bss section in CMakeFiles/Test.dir/main.cpp.o
pgacclnk: child process exit status 1: /usr/bin/ld
make[2]: *** [CMakeFiles/Test.dir/build.make:513: Test] Error 2
make[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/Test.dir/all] Error 2
make: *** [Makefile:91: all] Error 2

You see, I don’t use T3Inelasticdd_DB.hh in this code snippet. If to comment #include, the code compiles. The code also compiles if only to set:

constexpr unsigned int GL=1000000;

I understand I don’t provide You the code of T3Inelasticdd_DB.hh, but, can’t You guess, please, what may cause such an error?
Is there a pggdecode in nvc++ to decode __I___51__data_70_gaa_Desktop_CURRENT_WORK_MinRepEx_main_cpp_Ntop?

#4

I don’t know the size of “Particle” but assume these arrays are going over the 2GB limit set by the Small memory model (default). Try adding the flag “-mcmodel=medium” which switches to the Medium memory model allowing for static objects greater than 2GB.

#5

You were right. Thank You very much for the answer!