OpenMP: Variable in data clause is partially present on the device

Accelerated Computing HPC Compilers nvc, nvc++ and nvfortran
1

I am trying to offload an MPI-parallelized Lattice Boltzmann solver on a pair of Nvidia A100s. When I run my code, I get an error claiming that I have data that is partially present on the device. Here’s a small snippet:

      !$omp target data map(tofrom:lb_dom%fIn(1:19,1:32,1:32,1:32),lb_dom%fOut(1:19,1:32,1:32,1:32),lb_dom%lx)
      !$omp target teams distribute parallel do collapse(3) thread_limit(256)
      do k=1,lb_dom%lx(3)
         do j=1,lb_dom%lx(2)
            do i=1,lb_dom%lx(1)
               !computations including fIn and fOut
            end do
          end do
       end do
       !$omp end target data

and the error

Present table errors:
lb_dom lives at 0x65f8e0 size 4016 partially present in
host:0x65fc90 device:0x152f232fa000 size:272 presentcount:0+1 line:1785 name:descriptor file:/xx/xx/lb.F90
host:0x65fdb8 device:0x152f232fa200 size:272 presentcount:0+1 line:1785 name:descriptor file:/xx/xx/lb.F90
FATAL ERROR: variable in data clause is partially present on the device: name=lb_dom

This is the only mapping clause wherein I move data to/from the device. What could the issue be?

2

My guess is that you need to copy “lb_dom” itself first, i.e. something like “map(to:lbm_dom) map(tofrom:lb_dom%fIn(…”.

This way the members are attached to the parent’s device structure. Otherwise it’s getting implicitly copied three times.

If this doesn’t help, if you can provide a minimal reproducing example, I can take a look.

3

No luck so far- going to work on an MRE