controlling multiple GPUs from a single MATLAB "mex" file?

I’m trying to write a MATLAB “mex” file that uses more than one GPU. Has anyone done this?

My understanding is that you have to live with the following constraints:

[list=1]

(CUDA constraint) Each GPU must be accessed from a different thread.

(MATLAB constraint) Only the main thread can make “mx” / “mex” calls back into MATLAB.

If anybody knows of any pitfalls I’d love to hear about them.

(I’ll be trying this under Linux first, using pthreads.)

Jim

Jim,

I am also using CUDA with MEX functions, so far with good results.

First, where does it say that “each GPU must be accessed from a different thread”? I am not aware of such a limitation. You can play with PushCtx/PopCtx to change the CUDA contexts from the MEX/Matlab thread.

Mike

Thanks for the reply. True, I guess I could push and pop contexts as well. I just made a separate pthread per GPU/context.

It seems to be working fine!

A warning for anyone using pthreads + CUDA: make sure that when a pthread terminates, your main thread has called either pthread_detach or pthread_join (or start the thread in ‘detachable’ state in the first place). Otherwise thread resources are not freed, which seems to include a lot of CUDA resources.

Jim

Two more pitfalls:

[list=1]

If you’re going to simultaneously use two different MEX files that both access CUDA, make sure they don’t have any texture reference names or constant symbol names in common.

When you clear a CUDA MEX file from memory (with the MATLAB “clear” command), there are still open files left behind. (At least, under Linux.) Even if you have called cudaThreadExit(). You can find them with “lsof | grep nvidia”. If you repeatedly clear and reload your MEX file, eventually you will run out of file handles. There seems to be no way to close these files without exiting MATLAB. But as long as you don’t clear your MEX file from memory, CUDA will keep using the same file handles, even if you reinitialize your context.

Jim

Hi Jim,

My experience is that if you repeatedly clear and reload your MEX file you will eventually exhaust the memory on the graphic card, and cuCreateCtx or cuMemAlloc will fail. On my GTX 480 this happens after about 12 unloads. The only solution is then to restart Matlab.

So far I have no solution to this very annoying problem. It is probably a bug of the CUDA driver. Can you think of any workaround? I am using the driver API driver mode.

Mike

Mike,

I haven’t encountered that problem. I’m still using CUDA 2.3 so maybe it’s a 3.0+ problem?

If you’re properly closing your CUDA context and there is still memory left allocated on the card, I unfortunately have no suggestions.

Jim

Shameless plug… but I can’t help but chime in here that this sounds like a lot of hassle that could be avoided with a few hundred dollar purchase of Jacket’s Multi-GPU package, plus you’ll get a bunch of other MATLAB GPU functions :)

Check out Jacket’s Multi-GPU goodness here: http://www.accelereyes.com/products/jacketmgl

It’s fairly straightforward now that I know what not to do (detailed above). MEX file is running smoothly now.

Besides, it’s for this software project:
http://cbcl.mit.edu/jmutch/cns/

(Cost = $0.)