cudaMalloc failed with unknown error after only 491656bytes

Accelerated Computing CUDA CUDA Programming and Performance
1

I calculated how much is allocated after each cudaMalloc call. I get unknown error in a call to cudaMalloc, at this point of the program, only 491656bytes were allocated.
There’s 15 successful cudaMalloc calls before the call that fail.
What could be the cause of the problem?
Thanks any suggestions are appreciated.

I’m runing windows vista 64bit
I’m using vs2005
I’m compiling in 32bit mode
I have 9800GT

2

I am guessing you have a 512Mb Geforce 9800GT and you ran it out of memory.

3

And to answer the follow-on question: The CUDA driver (and the display driver) reserve some of the device memory for their use, so your programs cannot allocate all 512 MB. You can use cuMemGetInfo() to determine how much free memory is actually available to you.

4

Thanks avidday, but isn’t 491656bytes like too little to cause memory to run out? I have a 1GB 9800GT

Thanks seibert, actually i’m using C for Cuda instead of the Driver API, i think there’s no alternative to cuMemGetInfo in C for Cuda, but i keep track of how many bytes allocated after every cudaMalloc call, am I missing something?

5

Ehm, 491656bytes = 480kB right? Shouldn’t be a problem unless you made a typo :)

N.

6

That’s exactly what i meant. :)

Thanks

7

Sorry, I read 491656kb :)

That makes no sense. I routinely allocate large two and three dimensional finite difference meshes where a single degree of freedom uses 40-60Mb and they work with no problem. But 15 is suspiciously close to a power of two, which might mean there is some bizarre 4 bit counter somewhere…

8

Thanks everyone and I’m sorry.
It’s my fault
since there’s some kernel calls before the call to cudaMalloc that fails, when i tried this after the kernel calls, and before the cudaMalloc call
cutilSafeCall(cudaThreadSynchronize());
I got the unknow error, which means the kernel may have accessed to wrong memory. I’ll check further.
Thanks again.

9

Despite the naming, it is safe to call cuMemGetInfo() from the runtime API. (Maybe someday NVIDIA will add cudaMemGetInfo() so people won’t be confused anymore…)

10

Thanks External Media