Exclusive mode in GTX 1080

GTX 970, 980, 980 Ti on CUDA 7.0/7.5, the following command changes
the mode from DEFAULT to EXCLUSIVE.
nvidia-smi --id=0 --compute-mode=EXCLUSIVE_PROCESS

The same command makes the strange behaviour on GTX 1080 (CUDA 8.0 on SUSE leap 42.1)
It changes the mode from DEFAULT to EXLUSIVE (this is verified by
nvidia-smi that it shows E.Process) but no gpu application can’t
get the access of GPUs. (I did this through Theano and I haven’t done
in other ways.)

Same theano applications run fine on DEFAULT mode.

What is wrong in here? Does the spec/procedure changed in CUDA 8.0? Is this
related to the hardware defect? Is this due to Theano?

python
Python 2.7.12 |Anaconda custom (64-bit)| (default, Jul 2 2016, 17:42:40)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
Type “help”, “copyright”, “credits” or “license” for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org

import theano
ERROR (theano.sandbox.cuda): ERROR: Not using GPU. Initialisation of device gpu failed:
Not able to select available GPU from 4 cards (all CUDA-capable devices are busy or unavailable).
ERROR (theano.sandbox.cuda): ERROR: Not using GPU. Initialisation of device gpu failed:
Not able to select available GPU from 4 cards (all CUDA-capable devices are busy or unavailable).
ERR!\nERR!\nTraceback (most recent call last):
File “”, line 1, in
File “/opt/anaconda2/lib/python2.7/site-packages/theano/init.py”, line 111, in
theano.sandbox.cuda.tests.test_driver.test_nvidia_driver1()
File “/opt/anaconda2/lib/python2.7/site-packages/theano/sandbox/cuda/tests/test_driver.py”, line 29, in test_nvidia_driver1
A = cuda.shared_constructor(a)
File “/opt/anaconda2/lib/python2.7/site-packages/theano/sandbox/cuda/var.py”, line 218, in float32_shared_constructor
enable_cuda=False)
File “/opt/anaconda2/lib/python2.7/site-packages/theano/sandbox/cuda/init.py”, line 554, in use
cuda_ndarray.cuda_ndarray.select_a_gpu()
RuntimeError: (‘Not able to select available GPU from 4 cards (all CUDA-capable devices are busy or unavailable).’, ‘You asked to force this device and it failed. No fallback to the cpu or other gpu device.’)

Thank you very much in advance.

This is a known issue. Please update the GPU driver:

windows: use 368.69 or newer (368.69 was posted 7/ 6/2016)
linux: use 367.35 or newer (367.35 was posted 7/15/2016)