OpenACC kernels in Windows DLLs

I have some subroutines that I can compile as a DLL and link against an executable without any OpenACC directives. Adding directives yields linker errors around libaccg.lib. I understand that I don’t seem to be linking to that library but I’m unsure how to proceed, the compiler manual and user guide haven’t helped me.

My build process is below:

pgf90 -Bdynamic -m32 -Minfo=all -acc -ta=nvidia -c MCLIB_GPU.F90 && pgf90 -Mmakedll -Bdynamic -acc -ta=nvidia -Minfo=all -o MCLIB_GPU.dll MCLIB_GPU.obj

Hi mmawson57490,

We don’t currently support adding OpenACC to Window’s DLLs. It should be possible to support but we’ve had so few requests for it, it’s been a very low priority item.

Is this a critical item for you?

Thanks,
Mat

Hi Mat,

If it’s possible to compile it as a static library instead that would work. The project currently links several static compiled libraries together using a different compiler (Silverfrost) as that is what is used to compiled the bulk of the libraries.

Using a DLL (without OpenACC directives) I could link with the other static library files easily enough using the existing system. I didn’t really try compiling/linking it as a .lib as I’m not entirely sure what would be required.

Cheers,

Mark

Hi Mark,

I’ve put OpenACC into a static Windows library before but just needed to compile with “-ta=tesla:nordc”. CUDA relocatable device code (RDC) requires linking using the Nvidia device linker, so would need to be disabled. This fine but you wont be able to call OpenACC device routines except when included in the same module nor be able to access module device data from external modules.

What I haven’t done is try to link such a library using Silverfrost. I’m guess it will work since you can do it without OpenACC enabled, but I’m not sure. You will need to add the PGI OpenACC runtime libraries, along with the other PGI Fortran libraries, to your link.

libaccapi.lib
libaccg.lib
libaccn.lib
libaccg2.lib

Let us know how it goes,
Mat

Hi Mat,

Potentially stupid question, but which are PGI Fortran libraries I will need? I assume it’s the libraries linked by using -pgf90libs, but I can’t seem to find out which libraries those are.

*edit:Just to say that linking using the 4 libraries you mentioned doesn’t seem to work, I assume I’m missing others."

Cheers,

Mark

Hi Mark,

My apologies. Since you had previously linked before, I assumed you already had added the PGI Fortran run time libraries so only list the OpenACC libraries.

What I usually do is run the link with the PGI flag “-dryrun”. This will show what driver commands will be executed, including which libraries are linked, but not actually execute the command. You can also use the “–keeplnk” flag which will produce a file called “pgi.lnk” which lists the link options.

For example:

PGI$ pgfortran -acc -ta=tesla:nordc -dryrun --keeplnk x.obj
... cut ..
PGI$ cat pgi.lnk
C:\PROGRA~2\PGI/win32/15.10/lib\pgimainmx.obj
C:\PROGRA~2\PGI/win32/15.10/lib\f90main.obj
"-libpath:C:/Program Files/PGI/Microsoft Open Tools 12/lib"
"-libpath:C:/Program Files (x86)/Windows Kits/8.1/Lib/winv6.3/um/x86"
-incremental:no
-debug
-nologo
-stack:2097152,2097152
-merge:__nv_module_id=.data
-merge:.nvFatBinSegment=.data
-libpath:C:\PROGRA~2\PGI/win32/15.10/lib
x.obj
-out:x.exe
-defaultlib:libaccapi
-defaultlib:libaccg
-defaultlib:libaccn
-defaultlib:libaccg2
-defaultlib:libcudadevice
-defaultlib:ws2_32.lib
/DYNAMICBASE:NO
-defaultlib:libpgmp
-defaultlib:libpgf90rtl
-defaultlib:libpgf90
-defaultlib:libpgf90_rpm1
-defaultlib:libpgf902
-defaultlib:libpgf90rtl
-defaultlib:libpgftnrtl
-nodefaultlib:msvcrt
-defaultlib:libcmt
-defaultlib:libpgc
-defaultlib:libnspgc
-defaultlib:oldnames

Hope this helps,
Mat

Hi Mat,

Thanks for your help. Am I correct in thinking that the -nodefaultlib:msvcrt is used to avoid multiple definitions? At the moment I can’t find an option in the Silverfrost linker to exclude MSVCRT, and I’m getting an error about multiple definitions of a bad exception symbol. I’ve contacted their support for help, but I want to know if I’m on the right track.

Cheers,

Mark

Hi Mark,

Try taking libcmt off. msvcrt is the dynamic (DLL) C runtime library while libcmt is the the static C runtime library. You would get multiple defined reference errors if both are on the link.

  • Mat

Hi Mat,

Removing libcmt gets rid of the multiple defintions, but leaves some symbols missing:

WARNING the following symbols are missing:
_environ                                 C:\Program Files (x86)\PGI\win32\15.10\lib\libpgf90.lib (initpar.obj/   )
_fseeki64                                C:\Program Files (x86)\PGI\win32\15.10\lib\libpgf90.lib (error.obj/     )
_ftelli64                                C:\Program Files (x86)\PGI\win32\15.10\lib\libpgf90.lib (utils.obj/     )
_creat                                   C:\Program Files (x86)\PGI\win32\15.10\lib\libpgf902.lib (buffer.obj/    )
_errno                                   C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (pgstdinit.obj/ )
__iob_func                               C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (pgstdinit.obj/ )
_filbuf                                  C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (pgstdinit.obj/ )
_ungetc_nolock                           C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (pgstdinit.obj/ )
_flsbuf                                  C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (pgstdinit.obj/ )
isatty                                   C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (pgstdinit.obj/ )
setmode                                  C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (pgstdinit.obj/ )
_exit                                    C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (trace.obj/     )
_getpid                                  C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (trace.obj/     )
_tempnam                                 C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (pgtempnam.obj/ )
_imp__LoadLibraryA                       C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (cpuinfo.obj/   )
*** Imported common not satisfied: LoadLibraryA@4
_imp__GetProcAddress                     C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (cpuinfo.obj/   )
*** Imported common not satisfied: GetProcAddress@8
_imp__GetSystemInfo                      C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (cpuinfo.obj/   )
*** Imported common not satisfied: GetSystemInfo@4
_sleep                                   C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (init_nt.obj/   )
_imp__GetCurrentThreadId                 C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (init_nt.obj/   )
*** Imported common not satisfied: GetCurrentThreadId@0
_imp__ExitThread                         C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (init_nt.obj/   )
*** Imported common not satisfied: ExitThread@4
_imp__CloseHandle                        C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (init_nt.obj/   )
*** Imported common not satisfied: CloseHandle@4
_imp__CreateThread                       C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (init_nt.obj/   )
*** Imported common not satisfied: CreateThread@24
_imp__GetCurrentThread                   C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (setaff.obj/    )
*** Imported common not satisfied: GetCurrentThread@0
_imp__SetThreadAffinityMask              C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (setaff.obj/    )
*** Imported common not satisfied: SetThreadAffinityMask@8
_wassert                                 C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (hash.obj/      )
*** Imported common references not satisfied or Imported data missing

Linking against msvcrt.dll removes some of the missing symbols:

WARNING the following symbols are missing:
_ftelli64                                C:\Program Files (x86)\PGI\win32\15.10\lib\libpgf90.lib (utils.obj/     )
_errno                                   C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (pgstdinit.obj/ )
__iob_func                               C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (pgstdinit.obj/ )
_filbuf                                  C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (pgstdinit.obj/ )
_ungetc_nolock                           C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (pgstdinit.obj/ )
_flsbuf                                  C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (pgstdinit.obj/ )
isatty                                   C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (pgstdinit.obj/ )
setmode                                  C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (pgstdinit.obj/ )
_exit                                    C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (trace.obj/     )
_getpid                                  C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (trace.obj/     )
_tempnam                                 C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (pgtempnam.obj/ )
_imp__LoadLibraryA                       C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (cpuinfo.obj/   )
*** Imported common not satisfied: LoadLibraryA@4
_imp__GetProcAddress                     C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (cpuinfo.obj/   )
*** Imported common not satisfied: GetProcAddress@8
_imp__GetSystemInfo                      C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (cpuinfo.obj/   )
*** Imported common not satisfied: GetSystemInfo@4
_sleep                                   C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (init_nt.obj/   )
_imp__GetCurrentThreadId                 C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (init_nt.obj/   )
*** Imported common not satisfied: GetCurrentThreadId@0
_imp__ExitThread                         C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (init_nt.obj/   )
*** Imported common not satisfied: ExitThread@4
_imp__CloseHandle                        C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (init_nt.obj/   )
*** Imported common not satisfied: CloseHandle@4
_imp__CreateThread                       C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (init_nt.obj/   )
*** Imported common not satisfied: CreateThread@24
_imp__GetCurrentThread                   C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (setaff.obj/    )
*** Imported common not satisfied: GetCurrentThread@0
_imp__SetThreadAffinityMask              C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (setaff.obj/    )
*** Imported common not satisfied: SetThreadAffinityMask@8
_wassert                                 C:\Program Files (x86)\PGI\win32\15.10\lib\libpgc.lib (hash.obj/      )
*** Imported common references not satisfied or Imported data missing

I guess these missing symbols are to be found in libcmt?

All the symbols starting with “_imp” are from the “Kernels32.lib” library so you should be able to just add this to the link to grab these symbols.

While I’m not positive, for the rest of the symbols I think it is a mismatch between the static and dynamic MS C runtime lib. I’ve forgotten if it’s these symbols, but I know there are a few that have differing names between the two MS C runtime libraries.

Is there a way to get Silverfrost to link statically? or can you try using PGI to link adding the Silverfrost libs?

  • Mat

Hi Matt,

Happy new year.

I’m attempting to use PGI to do my linking now. I’m having a couple of errors; the first is duplicate definitions of various functions in libcmt.lib and one of the Silverfrost libraries (which I’m hoping to get around using Kernels32.lib as per your last post) and the second is that I’m getting a duplicate error for main. The main function exists in a .obj file that I pass into PGI to link, the linker spits out: error LNK2005: _main already defined in f90main.obj.

Do you have any ideas?

Cheers,

Mark

Hi Mark,

I’m having a couple of errors; the first is duplicate definitions of various functions in libcmt.lib and one of the Silverfrost libraries (which I’m hoping to get around using Kernels32.lib as per your last post)

You might try adding the flag “-Wl,-force:multiple” which tells the linker to ignore multiple definitions. This can cause issue of if the two definitions are distinct and the linker chooses the wrong one. but it’s worth a try.

The main function exists in a .obj file that I pass into PGI to link, the linker spits out: error LNK2005: _main already defined in f90main.obj.

Add the flag “-Mnomain” when linking a program that includes it’s own main routine.

Hope this helps,
Mat

Thanks Matt,

This does get everything to link and generate an executable. When I run I get an access violation (0xC0000005: Access violation reading location 0x00000000) from one of the Silverfrost DLLs I have to link against.

Cheers,

Mark

Well, shoot. Unfortunately have no experience with Silverfrost to know what’s wrong.

Ah well. Thanks for your help, I wouldn’t have got this far without it.