Strange issues with GL interop with OptiX

Currently, I have a project that uses CUDA, GLSL and OptiX. OpenGL’s PBO interop with OptiX does not give me any problems, however, when i use createTextureSamplerFromGLImage for OpenGL textures, there is an error that appear when my OptiX kernel is executing.

I tested the code with a simple snippet as such:

GLuint tempTex2D;
  glGenTextures(1, &tempTex2D);
  glBindTexture(GL_TEXTURE_2D, tempTex2D);
  glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
  glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
  glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
  glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
  glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA32F_ARB, max_voxel_size, max_voxel_size, 0, GL_RGBA, GL_FLOAT, NULL);
  glBindTexture(GL_TEXTURE_2D, 0);

  //register the gaussian framebuffer output into the single scatter texture
  TextureSampler tempSampler = Ctx->createTextureSamplerFromGLImage( tempTex2D ,RT_TARGET_GL_TEXTURE_2D);
  tempSampler->setWrapMode(0, RT_WRAP_CLAMP_TO_EDGE);
  tempSampler->setWrapMode(1, RT_WRAP_CLAMP_TO_EDGE);
  tempSampler->setIndexingMode(RT_TEXTURE_INDEX_NORMALIZED_COORDINATES);
  tempSampler->setFilteringModes(RT_FILTER_NEAREST, RT_FILTER_NEAREST, RT_FILTER_NONE);
  tempSampler->setReadMode(RT_TEXTURE_READ_NORMALIZED_FLOAT);
  tempSampler->setMaxAnisotropy(1.0f);
  Ctx["temp_tex"]->setTextureSampler(tempSampler);

rtTextureSampler temp_tex is declared in my kernel, but I did not used it or access, since I was testing if the interop would work for me:
rtTextureSampler<float4, 2> temp_tex;

The error occured the moment my Optix kernel was launched and produced the error:
OptiX Error: Invalid value <Details: Function “_rtContextLaunch2D” caught exception: GL error: invalid operation, [10420288]>

When I test by adding the same code snippet to the collision example in the Optix SDK. I was not able to replicate the error mainly because the rendering pass was in OpenGL. However, the original green outline surrounding the spheres became black, as well as the black lines used to connect the nearest colliding sphere.


Image on the left is the original application. The one on the right happens after I add a code snippet into the initScene() function in CollisionOptiX.cpp. I did realized that irregardless whether “rtTextureSampler<float4, 2> temp_tex” was declared in the collision_aux.cu file, the result was the same.

Have anyone encountered the same bug as me when using interop with OpenGL and OptiX?

tempSampler->setReadMode(RT_TEXTURE_READ_NORMALIZED_FLOAT); doesn’t make too much sense with a floating point texture. I would use RT_TEXTURE_READ_ELEMENT_TYPE for floating point textures.

OptiX checks for OpenGL errors when using OpenGL internally. The OpenGL error is just another state inside the OpenGL state machine. Once it occured it doesn’t go away until glGetError() is called.
Make sure that there are no OpenGL errors left over from your own OpenGL calls before calling into OptiX.

It’s unclear where you added your code to make the original collision example change its rendering.
Mind that to be able to reproduce something you experienced, the minimum required information should contain the OS (32-/64-bit), graphics hardware, driver version, OptiX version, and precise reproducer steps.

Thanks Detlef for the feedback regarding the OpenGL error, I will do some checking on my part for that.

As for information regarding the collision example:

My OS is Windows 7 64bit
My Graphics Hardware is Geforce GTX 680M
Driver version: 314.07
OptiX version: 3.0
Cuda Version: 5.0

As for the reproducer steps for the Collision Example, only the initScene() method was modified in CollisionOptiX.cpp and the following the code was inserted at Line 139

GLuint tempTex2D;
  glGenTextures(1, &tempTex2D);
  glBindTexture(GL_TEXTURE_2D, tempTex2D);
  glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
  glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
  glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
  glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
  glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA32F_ARB, 32, 32, 0, GL_RGBA, GL_FLOAT, NULL);
  glBindTexture(GL_TEXTURE_2D, 0);
  
  TextureSampler tempSampler = Ctx->createTextureSamplerFromGLImage( tempTex2D ,RT_TARGET_GL_TEXTURE_2D);
  tempSampler->setWrapMode(0, RT_WRAP_CLAMP_TO_EDGE);
  tempSampler->setWrapMode(1, RT_WRAP_CLAMP_TO_EDGE);
  tempSampler->setIndexingMode(RT_TEXTURE_INDEX_NORMALIZED_COORDINATES);
  tempSampler->setFilteringModes(RT_FILTER_NEAREST, RT_FILTER_NEAREST, RT_FILTER_NONE);
  tempSampler->setReadMode(RT_TEXTURE_READ_ELEMENT_TYPE);
  tempSampler->setMaxAnisotropy(1.0f);
  Ctx["tempSampler"]->setTextureSampler(tempSampler);