Bandwidth Test fails using Cuda 9.1 and Visual Studio 2013

Accelerated Computing CUDA CUDA Setup and Installation
1

Dear all,

I have just started using GPU. I have two GPUs on my workstation, NVS 310 and Tesla K40c.
I have Windows 7 OS and use Visual Studio 2013.
I installed Cuda Toolkit 9.1 and followed the instruction given here:

[Nvidia Instruction][/http://developer.download.nvidia.com/compute/cuda/6_5/rel/docs/CUDA_Getting_Started_Windows.pdf]

When I run deviceQuery CUDA sample, the test is successful. It shows both GPUs and Result = Pass

Here is a copy of the test result:

deviceQuery.exe Starting…

CUDA Device Query (Runtime API) version (CUDART static linking)

Detected 2 CUDA Capable device(s)

Device 0: “Tesla K40c”
CUDA Driver Version / Runtime Version 9.1 / 9.1
CUDA Capability Major/Minor version number: 3.5
Total amount of global memory: 11422 MBytes (11977031680 bytes)
(15) Multiprocessors, (192) CUDA Cores/MP: 2880 CUDA Cores
GPU Max Clock rate: 745 MHz (0.75 GHz)
Memory Clock rate: 3004 Mhz
Memory Bus Width: 384-bit
L2 Cache Size: 1572864 bytes
Maximum Texture Dimension Size (x,y,z) 1D=(65536), 2D=(65536, 65536),3D=(4096, 4096, 4096)
Maximum Layered 1D Texture Size, (num) layers 1D=(16384), 2048 layers
Maximum Layered 2D Texture Size, (num) layers 2D=(16384, 16384), 2048 layers
Total amount of constant memory: 65536 bytes
Total amount of shared memory per block: 49152 bytes
Total number of registers available per block: 65536
Warp size: 32
Maximum number of threads per multiprocessor: 2048
Maximum number of threads per block: 1024
Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
Max dimension size of a grid size (x,y,z): (2147483647, 65535, 65535)
Maximum memory pitch: 2147483647 bytes
Texture alignment: 512 bytes
Concurrent copy and kernel execution: Yes with 2 copy engine(s)
Run time limit on kernels: No
Integrated GPU sharing Host Memory: No
Support host page-locked memory mapping: Yes
Alignment requirement for Surfaces: Yes
Device has ECC support: Enabled
CUDA Device Driver Mode (TCC or WDDM): TCC (Tesla Compute Cluster Driver)
Device supports Unified Addressing (UVA): Yes
Supports Cooperative Kernel Launch: No
Supports MultiDevice Co-op Kernel Launch: No
Device PCI Domain ID / Bus ID / location ID: 0 / 5 / 0
Compute Mode:
< Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >

Device 1: “NVS 310”
CUDA Driver Version / Runtime Version 9.1 / 9.1
CUDA Capability Major/Minor version number: 2.1
Total amount of global memory: 512 MBytes (536870912 bytes)
MapSMtoCores for SM 2.1 is undefined. Default to use 64 Cores/SM
MapSMtoCores for SM 2.1 is undefined. Default to use 64 Cores/SM
( 1) Multiprocessors, ( 64) CUDA Cores/MP: 64 CUDA Cores
GPU Max Clock rate: 1046 MHz (1.05 GHz)
Memory Clock rate: 875 Mhz
Memory Bus Width: 64-bit
L2 Cache Size: 65536 bytes
Maximum Texture Dimension Size (x,y,z) 1D=(65536), 2D=(65536, 65535),3D=(2048, 2048, 2048)
Maximum Layered 1D Texture Size, (num) layers 1D=(16384), 2048 layers
Maximum Layered 2D Texture Size, (num) layers 2D=(16384, 16384), 2048 layers
Total amount of constant memory: 65536 bytes
Total amount of shared memory per block: 49152 bytes
Total number of registers available per block: 32768
Warp size: 32
Maximum number of threads per multiprocessor: 1536
Maximum number of threads per block: 1024
Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
Max dimension size of a grid size (x,y,z): (65535, 65535, 65535)
Maximum memory pitch: 2147483647 bytes
Texture alignment: 512 bytes
Concurrent copy and kernel execution: Yes with 1 copy engine(s)
Run time limit on kernels: Yes
Integrated GPU sharing Host Memory: No
Support host page-locked memory mapping: Yes
Alignment requirement for Surfaces: Yes
Device has ECC support: Disabled
CUDA Device Driver Mode (TCC or WDDM): WDDM (Windows Display Driver Model)
Device supports Unified Addressing (UVA): Yes
Supports Cooperative Kernel Launch: No
Supports MultiDevice Co-op Kernel Launch: No
Device PCI Domain ID / Bus ID / location ID: 0 / 4 / 0
Compute Mode:
< Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >

deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 9.1, CUDA Runtime Versi
on = 9.1, NumDevs = 2
Result = PASS

However, when I run bandwidthTest, either my code crashes or give the following result:

[CUDA Bandwidth Test] - Starting…
Running on…

Device 0: Tesla K40c
Quick Mode

CUDA error at C:/ProgramData/NVIDIA Corporation/CUDA Samples/v9.1/1_Utilities/ba
ndwidthTest/bandwidthTest.cu:743 code=2(cudaErrorMemoryAllocation) “cudaEventCre
ate(&start)”

In cases that the bandwidthTest run crashes, when I run Visual Studio Debugger, I get the following message:

Unhandled exception at 0x000007FEDAEB0938 (nvcuda.dll) in bandwidthTest.exe: 0xC0000005: Access violation reading location 0x0000000000001170.

I looked at several forums to find out what is wrong, but I could not find any relevant info.

I will appreciate if anyone can help me out please.

Thanks,
Ali

2

CUDA 9.1 doesn’t support devices of compute capability 2.x - including your NVS 310. I’m not sure why it is behaving the way you describe, but I would guess that there is a connection.

If you wanted to try and resolve that issue, one option would be to remove that NVS310 from the system. Another option would be to downgrade to CUDA 8 - which should support that GPU. You could also try using an environment variable to restrict CUDA to only trying to use the K40c:

CUDA_VISIBLE_DEVICES=“0”

[url]http://www.dowdandassociates.com/blog/content/howto-set-an-environment-variable-in-windows-command-line-and-registry/[/url]

You may also be able to do something similar in the GPU driver control panel on windows.

3

Thanks,

I used the GPU driver control panel and set Tesla K40c to be used for computation only.
But, I still get same error.
It seems that it is not possible to allocate memory on the GPU. I do not understand why.