I originally posted this in linux, but its also happening in windows as well. Mac osx is the only system that exhibits the correct behavior
cudaGLUnregisterBufferObject does NOT free up GPU memory that cudaGLRegisterBufferObject allocates.
simplest example of the problem:
static void print_mem()
{
CUresult res;
uint free, total;
res = cuMemGetInfo(&free, &total);
cout << "Free memory: " << (free>>20) << " MB" << endl;
}
//ignore the safecall routines, they do the same thing as cutil but put up a dialog box
static void test()
{
const size_t num_verts = 1024*1024*50;
print_mem();
GLuint vbo = 0;
glGenBuffers(1, &vbo);
glBindBuffer(GL_ARRAY_BUFFER, vbo);
glBufferData(GL_ARRAY_BUFFER, num_verts*sizeof(float)*4, 0, GL_DYNAMIC_DRAW);
glBindBuffer(GL_ARRAY_BUFFER, 0);
cbctSafeCall(cudaGLRegisterBufferObject(vbo));
print_mem();
cbctSafeCall(cudaGLUnregisterBufferObject(vbo)); //doesn't free a damn thing
print_mem();
glDeleteBuffers(1, &vbo);
print_mem();
}
this code produces:
Total memory: 1023MB
Free memory: 852MB
Free memory: 852 MB
Free memory: 52 MB
Free memory: 52 MB
Free memory: 52 MB
My app is completely useless to me without this working correctly. In windows. the memory will update correctly after the unregistration, just once however, and in the strangest of circumstances outside of any of my code when I bring up a dialog, menu or tooltip within my app’s gui.
I thought this at first, but indeed new allocations will fail. Driver versions vary from 2.0 to 2.2. happens on vista/xp & linux.
To make things worse, I cannot update my boss’s laptop drivers so I’m stuck with version 2.0 on his laptop, all other driver updates I’ve tried won’t detect his card.
I should also say that while re-registering a buffer will not cause an allocation failure (in support of your theory), any other kind will such as a cuda array allocation. The point at which the update finally occurs is very random in nature and to my belief dependent upon some arcane aspect of the OpenGL driver/context. For example on windows, bringing up any menu item or tool tip or dialog within my own app causes an update of the GPU memory whereas minimizing or going to other windows and back does not. On linux, I have to create a 3D texture and then wait for the display callback to be called. In both circumstances, the change occurs outside of my program, ie I cannot trace it to any one single line of code, except suddenly in one of my callbacks it will have been updated. I’ve tried “makecurrent” calls, glFlush(), just about everything. I’m really looking for a “hack” of a solution for now since I cannot upgrade my boss’s display driver on his 130m
Thanks. Thats good. If I can manage to get my boss’s acer laptop running 3.0 I’ll give it a shot. I suppose the only thing I can do in the meantime is avoid deregistering & registering new buffers and try to re-use whats there. Another pretty crazy alternative would be to completely copy all cuda memory to host buffers, destroy the entire cuda context and start over.
edit: aaand I’m an idiot. While its still a bug, its not nearly as severe as it was before. I wasn’t unmapping some pointers prior to my deregistration, thus the leak.
The problem still persists in that allocations can still fail if you do them RIGHT after deregistration and the driver takes a little while to reclaim the memory, but at least it does eventually.