Optix 7.2 Ray-Triangle Intersection Failure

I am developing SBR algorithm utilizing Optix 7.2. I send rays from an observation plane towards meshes. For some meshes and some observation angles, ray-triangle intersection works unexpectedly.
For example, considering the sphere, each ray must hit the surface at most once or miss.
However in my case, when I send rays from theta 0 and phi 270 plane, some of the rays don’t intersect the triangles at the surface of the sphere, but intersect the triangles inside.
Interestingly, using same mesh, when I send rays from theta 0 and phi 0 plane, all rays behave expectedly and there is no misintersection. I added images of these scenarios respectively. What would be the reason of this problem? Any suggestions is appreciated. Thanks in advance.

Cuda 11, Visual Studio 2019, Optix 7.2

  1. What’s your GPU and display driver version? (Especially RTX or non-RTX would be important.)
    There are multiple CUDA 11 versions Which one is it exactly?
  2. Are you using build-in triangles or custom primitives?
  3. There are no anyhit programs involved and you’re only determining the closest hit?
  5. The mesh is fully closed? No T-vertices, no seams at the 360 to 0 degrees connection? (But latter would look different in the error case.)

The hardware triangle intersection is watertight and should produce the closest hit when not doing optixIgnoreIntersection calls inside anyhit programs or when not using OPTIX_RAY_FLAG_TERMINATE_ON_FIRST_HIT.
Since this is view dependent, which means the BVH traversal happens in a different order, either of this could have that effect.

If that is not it, I would guess the intersection wasn’t watertight.
Could you isolate some of the protruding rays and see if they all lie exactly on the border or corners of adjacent triangles?
If that is the case, we would require a minimal reproducer in failing state to see what is going.

  1. Quadro RTX6000, CUDA 11.2
  2. Built-in-triangle
  3. OPTIX_GEOMETRY_FLAG_DISABLE_ANYHIT is set. When I found the closest hit, I updated the ray origin and direction using hit point and new direction calculated by surface normal. Then called the optixTrace again recursively.
  4. OPTIX_RAY_FLAG_TERMINATE_ON_FIRST_HIT is set in optixTrace function
  5. Yes, the mesh is fully closed.

I want to make a calculation for each ray-triangle intersection. When I did this with Any-Hit, the program was running slow. For this reason, I call the optixTrace function recursively by terminating the ray at the first hit and updating the ray origin and direction.

OPTIX_RAY_FLAG_TERMINATE_ON_FIRST_HIT is set in optixTrace function
For this reason, I call the optixTrace function recursively by terminating the ray at the first hit and updating the ray origin and direction.

That won’t work that way with OPTIX_RAY_FLAG_TERMINATE_ON_FIRST_HIT set.
Instead that calls into the closest hit program with the first hit, not necessarily the closest hit!

Which one that is, depends on the BVH traversal which depends on the ray origin and direction. (Means it’s view dependent for camera rays.)

That flag is normally used for a faster visibility tests than with anyhit programs and optixTerminateRay.
For such a visibility test you would not need an anyhit or a closesthit program at all. That can be done with just a miss program which indicates nothing was in in the way of the visibility ray [t_min, t_max] range.

If you always want the closest hit inside the closesthit program, please try removing the OPTIX_RAY_FLAG_TERMINATE_ON_FIRST_HIT from the optixTrace call.

When possible it’s also recommended to use iterative over recursive ray tracing. That will save device side stack space and perform better.
Reaching the closesthit program is the end of the current ray anyway. If you want to continue a single path with another single ray from there, you can also return to the ray generation program and set up the next path segment ray there.

Follow this link and the links in there for further threads about processing all intersections along a ray:

Thank you for the answer. It was my misunderstood of “first hit”.