Problem with NVFORTRAN and R

So using our program just as a regular program works fine. It’s just when I try to attach the subroutine to R that things get bad.

I have a question about the DLLmain.cpp subroutine, please. Does it have an .h file to accompany it, too, please?

Also, I think that xilink is an Intel compiler thing. Is that true, please?

Thanks,
Erin

I was able to produce a DLL file from DLLMain.cpp via g++

Any suggestions to link the pgfortran to the DLL, please?

I don’t have the Intel compiler.

Thanks,
Erin

I didn’t think g++ was available for native Windows so not clear on how you created a DLL using it? Are you using the WSL2 install? Granted, I don’t use WSL, but thought it was native Linux so wouldn’t use DLLs.

There are two distinct issues in this thread. When built on WSL, it appears that the GOMP library’s OpenACC interfaces are getting used. The second is on native Windows where the OpenACC runtime isn’t getting initialized, in which case we need to include a DLLmain to initialize the runtime upon load of the DLL.

Sorry that I’m confused, but I not sure which of the two we’re trying to solve. Seems like you’re mixing the two. Though, this may just be my misunderstanding.

This is absolutely confusing!

  1. I’m putting a temporary hold on the WSL project. WSL is being wonky at the moment.

  2. Windows with Fortran/C++. I am making a little bit of progress. Here is a Fortran subroutine, a C++ subroutine that compiles nicely with gfortran/g++ and works on R.

subroutine fpsub1(a,b)
      real a,b
      a=1.0
      b=2.0


      call cppfunction(a,b)



    end subroutine fpsub1

extern “C” {
void cppfunction_(float *a, float *b);
}

void cppfunction_(float *a, float *b) {
  *a=5.0;
  *b=6.0;
}

C:\updateR>gfortran -c fpsub1.f90 -o fpsub1.o -fPIC

C:\updateR>g++ -c cppfunction.cpp -o cppfunction.o

C:\updateR>gfortran -shared -o fpsub1.dll fpsub1.o cppfunction.o -lstdc++

Now run in R:

dyn.load(“fpsub1.dll”)
.Fortran(“fpsub1”,a=as.single(0.0),b=as.single(0.0))
$a
[1] 5
attr(,“Csingle”)
[1] TRUE

$b
[1] 6
attr(,“Csingle”)
[1] TRUE


So far so good.

Now I put a little OpenACC in

subroutine fnvsub1(a,b,x,n)
!DEC$ ATTRIBUTES DLLEXPORT :: fnvsub1
implicit none
real :: a,b,x(n)
integer :: n, i
a=1.0
b=2.0

  call cppfunction(a,b)
  !$acc parallel
  do i=1,n
     x(i) = 2*i + 1
  enddo
  !$acc end parallel
  

end subroutine fnvsub1
I used the same cppfunction.
But the pgfortran is balking in the linking:

C:\updateR>pgfortran -c -acc -Minfo=accel fnvsub1.90 -ta=tesla:nordc,cuda10.1 -o fnvsub1.obj

C:\updateR>g++ -c cppfunction.cpp -o cppfunction.o

C:\updateR>pgfortran -o fnvsub1.dll fnvsub1.obj cppfunction.o -acc -Minfo=accel -ta=tesla:nordc,cuda10.1
Creating library fnvsub1.lib and object fnvsub1.exp
f90main.obj : error LNK2019: unresolved external symbol MAIN_ referenced in function main
fnvsub1.dll : fatal error LNK1120: 1 unresolved externals
./fnvsub1.exf: error STP001: cannot open file

C:\updateR>pgfortran -o fnvsub1.dll fnvsub1.obj cppfunction.o -acc -Minfo=accel -ta=tesla:nordc,cuda10.1 -lstdc++
LINK : fatal error LNK1104: cannot open file ‘libstdc++.lib’
./fnvsub1.exf: error STP001: cannot open file

I have a feeling that I'm really close...

Thanks.

What GNU are you using? As far as I’m aware, I didn’t think GNU had a native Windows compiler and was only available with Cygwin or MinGw. Since NV/PGI are native Windows compilers, we aren’t object compatible with GNU on MinGw or Cygwin.

We actually use gcc 8.3.

We get that via Rtools, which has some stuff from MinGw.

I do have the free version of Visual Studio. Would that work better, please?

Most likely, though I’ve not tried myself so don’t know what issues you may encounter.

I’m back at it using cl.exe.
Here is my latest question, please: where would I find __pgi_acc_preinit ,please? Or how would I go about finding where those would be, please?

Thanks!

It’s in the OpenACC runtime libraries which you’ll need to link in similar to the command that I showed earlier.

  1. Starting to give up hope on Windows. I am stuck on using cl.exe with the DLLMain.cpp, even with all of the libraries.

  2. Updated the WSL 2 again. Still having the libgmp problem.

Thanks.

Actually, giving up hope on both concepts.

I thought of something to try with WSL2. Add “-Bstatic_pgi” when you create the shared library. This will bring in the compiler’s static runtime libraries into the shared object itself so the OpenACC runtime calls aren’t resolved by an external library (i.e. libgomp). The SO will be rather larger but might work around the problem.

Still no luck.

~$ nvfortran -c pi1subnv.f90 -o pi1subnv.obj -acc -Minfo=accel -ta=tesla:cuda11.0 -Bstatic_pgi -fPIC                                                                                                                      pi1subnv:                                                                                                                    18, Generating Tesla code                                                                                                   19, !$acc loop gang, vector(128) ! blockidx%x threadidx%x                                                                   Generating reduction(+:sum1)                                                                                    18, Generating implicit copy(sum1) [if not already present]                                                        
erinh@DESKTOP-1OKQBGD:~$ nvfortran -acc -Minfo=accel -Mlarge_arrays -Bstatic_pgi -ta=tesla:cuda11.0 pi1subnv.obj -o pi1subnv1.so -laccapi -laccg -laccg2 -lcudadevice -shared -L/opt/pgi/linux86-64-llvm/19.10/lib 

Then in R:

> dyn.load("pi1subnv.so")                                                                                               > is.loaded("pi1subnv")                                                                                                 [1] TRUE                                                                                                                > .Fortran("pi1subnv",as.double(0.0),as.single(0.0))                                                                                                                                                                                            libgomp: TODO   

Thanks,
Erin

Now I’m getting a different error message

> dyn.load("pi1subnv.so")                                                                                            
 Error in dyn.load("pi1subnv.so") :                                                                                        
unable to load shared object '/home/erinh/pi1subnv.so':                                                                 libaccapi.so: cannot open shared object file: No such file or directory  

At least it’s different!

Odd, not sure what changed. but let’s try not including the libraries on the link like since you’re bring them in separately so the compiler wont guard them with the linker’s static flag. Also adding the verbose (-v) flag so we can see the linker’s command line

nvfortran -acc -Minfo=accel -Mlarge_arrays -Bstatic_pgi -ta=tesla:cuda11.0 pi1subnv.obj -o pi1subnv1.so -shared -v

Here we go. Same TODO error

unlink /tmp/pgcudafatOgl4yuQMk9m.o                                                                                      unlink /tmp/pgcudaOgl4guRPu8V.cubin                                                                                     unlink /tmp/pgcudaOgl490zP5ER.cubin                                                                                     unlink /tmp/pgcudaOgl4SCCPgwJ.cubin                                                                                     unlink /tmp/pgcudaOgl4tcuP7g6.cubin                                                                                     
unlink /tmp/pgcudaOgl4YuOPy_M.cubin                                                                                     unlink /tmp/pgcudaOgl4j7_PDXA.cubin                   

And from R

> dyn.load("pi1subnv1.so")                                                                                              
> is.loaded("pi1subnv")                                                                                                 
[1] TRUE                                                                                                                > .Fortran("pi1subnv",as.single(0.0),as.single(0.0))                                                                                                                                                                                            libgomp: TODO   

I’m running out of ideas. Though, you might try using gfortran instead (OpenACC enabled via the -fopenacc flag.) Though, you may need to upgrade to a newer version of GNU since I think OpenACC support in 8.3 was limited. It’s much better in GNU 10.1 and 11.

I actually tried that over the weekend. It compiles/links, but never goes to the GPU.

Rats.

Thanks