GTX 970, 980, 980 Ti on CUDA 7.0/7.5, the following command changes
the mode from DEFAULT to EXCLUSIVE.
nvidia-smi --id=0 --compute-mode=EXCLUSIVE_PROCESS
The same command makes the strange behaviour on GTX 1080 (CUDA 8.0 on SUSE leap 42.1)
It changes the mode from DEFAULT to EXLUSIVE (this is verified by
nvidia-smi that it shows E.Process) but no gpu application can’t
get the access of GPUs. (I did this through Theano and I haven’t done
in other ways.)
Same theano applications run fine on DEFAULT mode.
What is wrong in here? Does the spec/procedure changed in CUDA 8.0? Is this
related to the hardware defect? Is this due to Theano?
python
Python 2.7.12 |Anaconda custom (64-bit)| (default, Jul 2 2016, 17:42:40)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
Type “help”, “copyright”, “credits” or “license” for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.orgimport theano
ERROR (theano.sandbox.cuda): ERROR: Not using GPU. Initialisation of device gpu failed:
Not able to select available GPU from 4 cards (all CUDA-capable devices are busy or unavailable).
ERROR (theano.sandbox.cuda): ERROR: Not using GPU. Initialisation of device gpu failed:
Not able to select available GPU from 4 cards (all CUDA-capable devices are busy or unavailable).
ERR!\nERR!\nTraceback (most recent call last):
File “”, line 1, in
File “/opt/anaconda2/lib/python2.7/site-packages/theano/init.py”, line 111, in
theano.sandbox.cuda.tests.test_driver.test_nvidia_driver1()
File “/opt/anaconda2/lib/python2.7/site-packages/theano/sandbox/cuda/tests/test_driver.py”, line 29, in test_nvidia_driver1
A = cuda.shared_constructor(a)
File “/opt/anaconda2/lib/python2.7/site-packages/theano/sandbox/cuda/var.py”, line 218, in float32_shared_constructor
enable_cuda=False)
File “/opt/anaconda2/lib/python2.7/site-packages/theano/sandbox/cuda/init.py”, line 554, in use
cuda_ndarray.cuda_ndarray.select_a_gpu()
RuntimeError: (‘Not able to select available GPU from 4 cards (all CUDA-capable devices are busy or unavailable).’, ‘You asked to force this device and it failed. No fallback to the cpu or other gpu device.’)
Thank you very much in advance.