-Mcuda Option and ACC_DEVICE_NUM-Environment

Hi

I use Fortran PGI 14.10 with OpenACC.

If I use the compiler-option “-cuda” the environment-variable “ACC_DEVICE_NUM” will not be evaluated from my program.

I use -Mcuda to create data only on the device (https://forums.developer.nvidia.com/t/how-to-create-data-on-device/134350/1)

I have a computer with 4 GPU’s and I’d like to be able to select the device.

I tried to call acc_set_device_num “by hand”. But the I get an errormessage

line 41: cudaLaunch returned status 33: invalid resource handle

Any hint?
Benedikt[/code]

Hi Benedikt,

This has to do with the CUDA runtime environment that’s used. Linking with -Mcuda changes the runtime and effectively disables the environment variable. Instead, you can try calling the CUDA Fortran “cudaSetDevice” routine or use the “CUDA_VISIBLE_DEVICES” environment variable. http://devblogs.nvidia.com/parallelforall/cuda-pro-tip-control-gpu-visibility-cuda_visible_devices/.

I added a feature request (TPR#21222) to see if we can get “ACC_DEVICE_NUM” to work when an OpenACC program is linked with -Mcuda.

As for the “cudaLaunch” error, I wasn’t able to reproduce this (I tried using your previous test program). Though, some possible reasons could be that you didn’t use the openacc module so the routine and variables are implicitly typed, or you set the device after you already allocated the memory. If so, then the memory may be over on a different device.

  • Mat

Hi Mat

“cudaSetDevice” seems to work. Thanks. As a workaround I use a function hich evaluates the ACC_DEVICE_NUM-Variable…

      SUBROUTINE SETUP_ACCDEVICE
#ifdef _OPENACC 
      use cudafor
      CHARACTER(20) env
      INTEGER L,dn,dc,envStatus,cr
      L=0
      envSTATUS=0
      CALL GET_ENVIRONMENT_VARIABLE("ACC_DEVICE_NUM", env, L, 
     &                              envSTATUS) 
      if (envSTATUS.EQ.0) THEN
        READ(env,'(I4)') dn
        cr = cudaSetDevice(dn)
      END IF
      cr = cudaGetDevice(dn)
      cr = cudaGetDeviceCount(dc)
      WRITE(*,'(A8,I3,A4,I3)') 'Use GPU ',dn, ' of ', dc
#endif      
      END SUBROUTINE

Bye
Benedikt[/code]

TPR 21222 - OpenACC: ACC_DEVICE_NUM does not set device number if program is linked with -Mcuda
is fixed in the 15.5 release.

thanks,
dave