texture references / global memory and multiple GPUs


Texture references have to be declared globally in the .cu file.
With CUDA 4.0 I am now able to lauch the same kernel on different GPUs just using cudaSetDevice()without the need to have a multi-threaded (CPU) application.
I have found some hints that for a multi-threaded application with one GPU per thread this works. But I want a single threaded application with multiple GPUs.

How are texture references correctly handled in this context.
Is it sufficient to just bind/unbind the same texture reference for each GPU in turn?

And what about constant memory, also declared globally ? Can it be used at all ?

Are there any experiences with this ?
Any hint to some documentation would be highly appreciated ?!

regards Rolf

Since at least for me it is not 100% clear how this works (and it does), and I gt no answers so far, I’ll just write down my experiences:

All global device variables (global, deevice, constant) seem to exist for each device you are using and are distinct for each context.
I assume(!) they are “switched” when you call cudaSetDevice().

The usage of textures seems to be no problem.

  • you declare one global texture reference
  • you bind this texture to different memories on different GPUs
  • you can use it for different kernels on the dfferent devices.

What remains completely unclear to me is that

  • textures are declared globally on the host
  • they have public members you need to set before binding the texture
  • what happens if you bind the same texture with different settings for the different GPUs ?

Probably not a problem when using the texture on the device via tex1/2/3D() functions.
But you should probably(?!) never try to check the settings on the host, they are probably(?!) merely for passing
parameters to the texture binding functions?!

Unfortunately this is completely undocumented as to my knowledge.

regards Rolf