Prime program - unexpected results

I have an optix prime program where each vertex in my triangle mesh is a pixel in the resulting image. Currently, I trace a ray to the middle of each triangle. I then set a hit flag to true for each vertex in the triangle that is hit. This works perfectly, and I get the expected results.

What I would like to do instead is trace a ray to each vertex. When I do this, however, for about half the vertices, Optix Prime is returning triangles that do not even include the vertex. The triangle id seems to be off by 1. I would expect the results to be very similar to tracing rays to the center of the triangles, but they’re not. What am I missing?

You’re shooting rays exactly to the corners of each triangle which basically lies between adjacent triangles and depending on the floating point accuracy that may hit any of the adjacent triangles at this point.

What problem do you need to solve?
If it’s a binary visibility test, you could also shoot rays into the opposite direction from the vertex to the eye and you would know which vertex or triangle ID started it.

Right, it could hit any of the adjacent triangles. However, an adjacent triangle should include the vertex in question, correct?

Yes, it’s a binary visibility test. I think you’re right that shooting a ray from vertex to eye is a better idea. Thanks.

On further thought, I don’t think shooting rays from vertex to eye will work. I’m using Optix Prime so if the vertex is visible (no hits), I get no information. This leaves me no way to determine if the vertex is visible because the eye is seeing it from behind the surface.

I don’t know what information you mean or what surface is involved. If anything is between the beginning and end of the ray you shoot, an anyhit query will return true. No matter if that is from eye to vertex or from vertex to eye. You know which vertex it was because of the construction of the ray buffers.

If that is not what you need, then maybe describe your desired algorithm with more detail.

Right, but what if the eye is behind the surface? The anyhit query will return nothing for that indicating it is visible, but that is incorrect. Since no hit is returned, I cannot determine if the ray is behind the surface.

I figured out how to only include rays that are on the correct side of the surface so now I have an implementation where the rays are going from the vertices to the eye. Unfortunately, the results are very bad, and just like my original question, make no sense. I have 1,000,000 vertices that I am checking for visibility. With an implementation where I shoot the rays from the eye to the middle of each triangle in the mesh, 9,999,995 vertices are visible. With the implementation where the rays are going from each vertex to the eye, 5438 vertices are visible. Obviously, Optix is returning many, many hits when it shouldn’t.

Your terminology is confusing. Vertices are parts of the triangle, if you mean the triangle mesh is your surface and you only want to receive hits from the front face, which you either indicate with the vertex winding or a face normal, that would be clearer.
OptiX Prime doesn’t care about winding or face normals.

Not enough information to help further.
If you’re sure that there is nothing wrong with your algorithm, you could provide a minimal reproducer in failing state accompanied by all necessary system configuration information to setup the app for someone who doesn’t program (QA) to be able to file this as bugreport for investigation.

My problems in relation to my original question are because Optix Prime is not returning hits when it should. If a ray hits a vertex in the triangle mesh, what does Optix Prime do?

What’s your t_min when starting from the vetex?
It shouldn’t be 0.0f or you might hit the starting triangle. Add a scene_epsilon to the start offset then.

Again, an exact reproducer would be more helpful to analyze your case.

I am shooting rays from the eye to the vertex. As I stated originally, this should work fine even with floating point error as the ray will hit one of the triangles that contains the vertex. The problem is Optix Prime will often say that the ray didn’t hit anything, which is impossible. I already have a test case written to demonstrate the problem, I’m just waiting for it to be transferred to where I can send it to you guys. Where do we file bug reports?

I was referring to these numbers:

The second case needs a scene epsilon to not hit the starting triangle surface.

If you do not want to attach a reproducer here, using the paper clip icon which appears when hovering the mouse over submitted posts, please have a look at the OptiX release notes how to contact the OptiX team directly to exchange confidential or bigger data.