CUDA Memory allocation issue on GTX 850M on Windows 7 (64)

Hi guys,

We have run into a weird issue where we are unable to allocate more than 2GB of memory on 850M that has 4 GBs. Just to make sure that there is nothing wrong with our program, we wrote a small test app that keeps alloc’ing memory until it receives out of memory error. It works fine on Windows 8, however on Windows 7, it throws error after allocating 2 Gbs, and yet it shows it has 2 Gb free memory available. My question is why I am unable to allocate more than 2Gb memory on Windows 7? Another clue: I rolled back my driver and found out that I get this behavior starting from nvidia driver 352.86 all the way to the latest driver. On older drivers, it works like a charm even on Win 7.

Thanks in advance.

Best

If the driver’s behavior has changed between two driver revisions, you might want to file a bug. You have to be a member of nVidia’s registered CUDA developer program ( there’s no cost involved, just a a registration ).

This may be related to how the Windows Display Driver Model (WDDM) manages memory. I just find it odd that there has been a change between specific CUDA driver revisions.

Based on your description, this sounds like a regression in the driver. I would suggest filing a bug report to bring this to NVIDIA’s attention.

The bug reporting form is linked from the CUDA developer website, log in at https://developer.nvidia.com/

Edit: cudaMemGetInfo(free,total) on an sm_52 device doesn’t seem to adjust the free count after a single very large allocation. Yet the free count is adjusted on the two sm_50 devices in my 358.50 + Win7/x64 system.

If this is acceptable behavior then the function’s description should probably spell this out.


@Usmann, what is cudaMemGetInfo() reporting?

You could try running this test – it’s probably similar to yours: https://gist.github.com/allanmac/9263979

If you haven’t already, you should verify whether you’re building a 32-bit or 64-bit application.

Example on a 4GB GTX 980 (sm_52) + 358.50 + Win7/x64:

32-bit arch

$> malloc
GeForce GTX 980 (16)
free/total (MB):    3066 /    3072
cudaMalloc (MB):    2810
free/total (MB):    3066 /    3072 <--- THIS IS UNEXPECTED

64-bit arch

$> malloc
GeForce GTX 980 (16)
free/total (MB):    3935 /    4096
cudaMalloc (MB):    3679
free/total (MB):    3935 /    4096 <--- THIS IS UNEXPECTED

Example on a 2GB GTX 750 Ti (sm_50):

32-bit arch

$> malloc 2
GeForce GTX 750 Ti ( 5)
free/total (MB):    1651 /    2048
cudaMalloc (MB):    1395
free/total (MB):     255 /    2048 <--- OK

64-bit arch

$> malloc 2
GeForce GTX 750 Ti ( 5)
free/total (MB):    1650 /    2048
cudaMalloc (MB):    1394
free/total (MB):     255 /    2048 <--- OK

A Quadro K620 w/2GB (sm_50) shows results similar to the GTX 750 Ti.

@cbuchner1, njuffa: Thanks for the reply

@allanmac: This is what I am getting:
GeForce GTX 850M ( 5)
free/total (MB): 4011 / 4096
cudaMalloc (MB): 3755
free/total (MB): 4011 / 4096

There is a slight issue in the code. You are not checking for the return code of cudaMalloc call, in my case I am getting memory exception and therefore nothing was actually allocated. So the third line makes sense and is valid. I am quite sure that on your GTX 980, its the same, you are unable to allocate anything and hence the free/total memory stays the same. Please slightly adjust the code to see if you get error on cudaMalloc on GTX 980.

@all:

I have logged a bug on NVidia drivers

Oops… Fixed!

However no errors are being returned and cudaMemGetInfo() is now reporting the expected result every single time.

The only difference between today and yesterday is that I rebooted! :)

Thanks for filing a bug.