OpenMP error: use of undefined value

Accelerated Computing HPC Compilers nvc, nvc++ and nvfortran
1

Hello,
I am currently developing a Fortran library to compute integrals of a user-defined function over a given manifold. These integrals involve a sampling over the points of the manifold which I have parallelized using OpenMP. My goal is to implement GPU offloading by using nvfortran, but right now I am only trying to reproduce previous results in multicore CPUs. The sampling process is done as follows,

!$ OMP PARALLEL DEFAULT(SHARED) PRIVATE(k)
  !$ OMP DO COLLAPSE(3)
  do ik1 = 1, task%samples(1)
    do ik2 = 1, task%samples(2)
      do ik3 = 1, task%samples(3)
        k = ...
        data_k(ik1, ik2, ik3, :) = task%calculator(task, system, k, error)
      enddo
    enddo
  enddo
  !$ OMP END DO
!$ OMP END PARALLEL

Where task is a Fortran derived type containing information on the sampling and a procedure pointer task%calculator, which refers to the previously mentioned user-defined function.

When compiling with ifort/ifx/gfortran with the -qopenmp/-fopenmp flag, the library compiles and works properly. However, when doing so in nvfortran with -fopenmp/-mp=multicore flags leads to the error

/opt/nvidia/hpc_sdk/Linux_x86_64/23.7/compilers/share/llvm/bin/llc: error: /opt/nvidia/hpc_sdk/Linux_x86_64/23.7/compilers/share/llvm/bin/llc: /tmp/nvfortran8xJm0Wzu_b0o.ll:3085:12: error: use of undefined value '%u'
    store ptr %u, ptr  %536, align 8, !dbg !339

Some related posts [1], [2] have addressed variations of this issue in the past, but have failed to provide a general solution. I have noticed that disabling OpenMP leads to a successful compilation and that the compiler error originates in the assignment data_k(ik1, ik2, ik3, :) = task%calculator(task, system, k, error). However, if instead of sampling the procedure pointer task%calculator one samples a “regular” procedure test with the same interface as task%calculator, even when using OpenMP, the compilation is successful.

Is there any way to circumvent this issue while using procedure pointers?

Thanks for your help,
Álvaro

A remark:
The interface of task%calculator is:

   abstract interface
     function abs_calculator(task, system, k, error) result(u)
       import :: global_k_data, sys, dp

       class(global_k_data), intent(in) :: task
       type(sys), intent(in)            :: system
       real(kind=dp), intent(in)        :: k(3)
       logical, intent(inout)           :: error

      complex(kind=dp) :: u(product(task%integer_indices), product(task%continuous_indices))
    end function abs_calculator
   end interface
2

Hi Álvaro,

This looks like a compiler code generation issue. Can you provide a minimal reproducing example so I can report it to our team? Hopefully I can find a work around for you as well.

-Mat

3

Hi Mat, and thank you for the early response.

I am attaching a minimal reproducing example within this post.
procpoint_nvf.F90 (1.8 KB)

I am compiling with:

  1. nvfortran procpoint_nvf.F90: no compiler error and expected behavior.

  2. nvfortran -fopenmp procpoint_nvf.F90: no compiler error but segmentation fault on execution.

As mentioned in the original post, when uncommenting ln 73 of the attached example and commenting ln 75 I recover the expected behavior in both compilation cases. While I do not get the same error referencing a undefined variable as in the original post, I believe that the error in both cases may be related.

Bests,

Álvaro

4

Thanks Álvaro,

I can’t explain the disconnect, but while I can reproduce the code gen error in 23.5, it compiles fine for me with 23.7. Also, I’m not able to reproduce the seg fault.

In case it’s relevant, can you let me know the OS and CPUs you’re using? Any other information that might help me reproduce the issue?

Here’s my output on a AMD EPYC 7513 dual socket (64 cores total) running Oracle Linux.

% nvfortran procpoint_nvf.F90 -fopenmp -V23.5
/proj/nv/Linux_x86_64/23.5/compilers/share/llvm/bin/llc: error: /proj/nv/Linux_x86_64/23.5/compilers/share/llvm/bin/llc: /tmp/nvfortraniZ_ueyWoodzWX.ll:279:12: error: use of undefined value '%u'
        store ptr %u, ptr  %51, align 8, !dbg !92
                  ^
% nvfortran procpoint_nvf.F90 -fopenmp -V23.7
% a.out
 (0.000000000000000,0.000000000000000)  (0.000000000000000,0.000000000000000)
%

-Mat

5

Thank you Mat,

I will also try to compile with version 23.7 and see if it solves the problem.

I am running the code on a Intel i7 10700 (8 core) and Ubuntu 22.04.

Álvaro

6

FYI, from the error output, it does look like you’re already using 23.7 and hence the disconnect between what you show what I’m able to reproduce.

I don’t have access to this particular type of CPU, but tried on several other Intel based CPUs (Skylake and Icelake) with the same results that I see on the AMD system.

7

Excuse my misunderstanding, I thought I was using V23.5 instead of 23.7. In relation to the disconnect, I don’t have any clue either on where the difference may be, since as you pointed out it does not seem a CPU issue. I will also check my compilator installation and test again if needed. By the way, the expected result from running

should be (48.00000000000000,0.000000000000000) (96.00000000000000,0.000000000000000).

However, in relation to the original compilation-time error, I have realized that the problem may be on the interface block declaration of f starting on ln 11 in the example above. By removing the abstract keyword in the interface block I was able to compile and run the code without issues and with the expected result with and without OpenMP. Of course, this does not explain the disconnect between our tests in relation to the different compiler versions, but may work as a temporary workaround for my original issue.

Álvaro

8

I missed that it was getting wrong answers.

I’ve submitted a problem report to engineering, TPR #34227.

The wrong answers seem to occur at -O1, so another work around is to use “-O0 -mp”, but performance could suffer. Without OpenMP, it will still get wrong answers, but just at -O2 or above.

I suspect the compiler ICE is still there, but it’s just luck that I’m not seeing it. Best guess is that the two are related with whatever is causing the “associated” check to be false, is then later causing the ICE when compiled with OpenMP enabled. Then again the ICE in 23.5 seems to still occur with “-O0 -mp”, so maybe not.