Program breaks when running on K80 and mps control daemon runs

Accelerated Computing CUDA CUDA Setup and Installation
1

Hi,

I have Tesla K80 and RTX 3060 Ti cards in my machine. I have a simple example program that uses OpenACC. It compiles and runs on both cards without a problem when mps control daemon does not run. However, when I firstly run the daemon (nvidia-cuda-mps-control -d) the program still runs OK on the RTX card but breaks/exits on K80.

I compiled the program with
nvc++ -acc=gpu -gpu=ccall,cuda11.2,fma -Minfo=all -O3 test.cpp

When I run it on K80 I get the following, where at line 31 is the first OpenACC directive in the file.

Current file: /home/popsi/test/cuda/openacc/test.cpp
function: main
line: 31
This file was compiled: -ta=tesla:cc35,cc50,cc60,cc70,cc75,cc80

To repeat, the program runs without issues on the RTX card and braks on K80.

As far as I know K80 supports CUDA Hyper-Q/mps.

Additional info:
CUDA Toolkit version is 11.2, Nvidia Driver is 460.39, Linux OS with kernel 5.9.16.

How to solve this problem?

Igor

2

Soon after I started this topis I fixed the problem.
Environment variables CUDA_MPS_PIPE_DIRECTORY and CUDA_MPS_LOG_DIRECTORY must point to folders that are accessible to user that runs the gpu-accelerated program. So I made them point to a user’s home subfolder. Still it is weird that program could run on the RTX GPU even with unset the env. vars, when defaults, /tmp/nvidia/mps was used.