CUDA 7 not working properly on Ubuntu 14.04

Hi,

I have on nvidia gforce GTX 860M on my laptop, and I just installed CUDA 7 with new driver. After installation I run deviceQuerry and it passed the test, alongside with the bandwidth test. All the other tests in Nvidia Samples are failing.
Another thing is that when I return my laptop from sleep, I cant run even deviceQuerry, it says no capable cuda device found. After rebooting i can again run deviceQuerry, but all the other tests fail again.

this is result of deviceQuerry

CUDA Device Query (Runtime API) version (CUDART static linking)

Detected 1 CUDA Capable device(s)

Device 0: "GeForce GTX 860M"
  CUDA Driver Version / Runtime Version          7.0 / 7.0
  CUDA Capability Major/Minor version number:    5.0
  Total amount of global memory:                 2048 MBytes (2147352576 bytes)
  ( 5) Multiprocessors, (128) CUDA Cores/MP:     640 CUDA Cores
  GPU Max Clock rate:                            1020 MHz (1.02 GHz)
  Memory Clock rate:                             2505 Mhz
  Memory Bus Width:                              128-bit
  L2 Cache Size:                                 2097152 bytes
  Maximum Texture Dimension Size (x,y,z)         1D=(65536), 2D=(65536, 65536), 3D=(4096, 4096, 4096)
  Maximum Layered 1D Texture Size, (num) layers  1D=(16384), 2048 layers
  Maximum Layered 2D Texture Size, (num) layers  2D=(16384, 16384), 2048 layers
  Total amount of constant memory:               65536 bytes
  Total amount of shared memory per block:       49152 bytes
  Total number of registers available per block: 65536
  Warp size:                                     32
  Maximum number of threads per multiprocessor:  2048
  Maximum number of threads per block:           1024
  Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
  Max dimension size of a grid size    (x,y,z): (2147483647, 65535, 65535)
  Maximum memory pitch:                          2147483647 bytes
  Texture alignment:                             512 bytes
  Concurrent copy and kernel execution:          Yes with 1 copy engine(s)
  Run time limit on kernels:                     No
  Integrated GPU sharing Host Memory:            No
  Support host page-locked memory mapping:       Yes
  Alignment requirement for Surfaces:            Yes
  Device has ECC support:                        Disabled
  Device supports Unified Addressing (UVA):      Yes
  Device PCI Domain ID / Bus ID / location ID:   0 / 1 / 0
  Compute Mode:
     < Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >

deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 7.0, CUDA Runtime Version = 7.0, NumDevs = 1, Device0 = GeForce GTX 860M
Result = PASS

Even though it passes this test it is not able to run any of the sample codes.
after putting laptop to the sleep I get following message

./deviceQuery Starting...

 CUDA Device Query (Runtime API) version (CUDART static linking)

cudaGetDeviceCount returned 38
-> no CUDA-capable device is detected
Result = FAIL

these are some of the readings

lsmod | grep nvidia

nvidia_uvm             67139  0 
nvidia               8370760  1 nvidia_uvm
drm                   303102  8 i915,drm_kms_helper,nvidia

cat /proc/driver/nvidia/version

NVRM version: NVIDIA UNIX x86_64 Kernel Module  346.46  Tue Feb 17 17:56:08 PST 2015
GCC version:  gcc version 4.8.2 (Ubuntu 4.8.2-19ubuntu1) 

lspci | grep -i nvidia

01:00.0 3D controller: NVIDIA Corporation GM107M [GeForce GTX 860M] (rev ff)

ls -al /dev/nvidia*

crw-rw-rw- 1 root root 195,   0 Apr  8 21:59 /dev/nvidia0
crw-rw-rw- 1 root root 195, 255 Apr  8 21:59 /dev/nvidiactl
crw-rw-rw- 1 root root 250,   0 Apr  8 21:59 /dev/nvidia-uvm

as well I get message after running nvidia-settings :
You do not appear to be using the NVIDIA X driver. Please edit your X configuration file (just run nvidia-xconfig as root), and restart the X server.

I did previously run nvidia-xconfig as root

Couple of questions. First, how did you install the Nvidia driver? Second, do you have Optimus enabled? (i.e. Intel + Nvidia graphics)

I am running Ubuntu 14.04 LTS with CUDA 7 and the 346.47 driver. I disabled Optimus in the BIOS settings of my laptop so that only the discrete (Nvidia) graphics card is being used, as I had issues with switching. My laptop would choose the Intel HD sometimes and disable the CUDA driver. Also, I did not install the driver with the CUDA installation. I declined the driver installation then installed it separately.

I would try to clean out the system and remove CUDA and the driver (try sudo ./cuda_7.x.x_linux.run --uninstall), then reinstall CUDA first, then the driver second.

Thanks for reply,

I installed Nvidia driver alongside with CUDA, by downloading toolkit from nvida download page. I dont know how to check if I have Optimus enabled.

The thing is that it is working fine until I put laptop to sleep. Afterwards its not working at all.

Did you did you download driver separately or you use one which is provided in runfile installation?

I downloaded the driver separately.

If you want to check for Optimus, run ‘lspci’ in a terminal window and see if the Nvidia card and the Intel graphics are both present (like here: How to install and configure Nvidia Optimus driver on Ubuntu). I disabled Optimus in BIOS. There was an option to disable the integrated (i.e. Intel) graphics.

It’s strange that it only happens when you put it to sleep. Makes me think it’s a switching (i.e. Optimus) issue or a sleep state issue. I suspect that forcing the laptop to always use the discrete (i.e. Nvidia card) will fix the issue. This may slightly decrease your battery life (since the discrete uses more power) but I haven’t really noticed on my own system to be honest.

put following 2 lines in your /etc/rc.local

/usr/bin/nvidia-smi -pm ENABLED
/usr/bin/nvidia-smi -c EXCLUSIVE_PROCESS

https://chuanwen.wordpress.com/2015/07/19/run-cuda-on-ubuntu-14-04-2/

The trick with nvidia-smi seems interesting.

/usr/bin/nvidia-smi -pm ENABLED
/usr/bin/nvidia-smi -c EXCLUSIVE_PROCESS

I am seeing this behavior on Ubuntu 12.04 as well - so far I always had to reboot when needing to use CUDA after a suspend.

I will let you know if this helps.