I have some questions regarding ray intersection in a triangle as described above figures:

When shooting a ray from (a) towards (b), the closest hit is (c).
When shooting a ray from (b) towards (a), the closest hit is (a).
When shooting a ray from (d) towards (e), miss.
When shooting a ray from (e) towards (d), the closest hit is (d).

Why is this happening?
What I want is for the results of (a) - (b) to be the same, and for (d) - (e) to be the same.
The problem is that although the rays are fired towards each other, the results are different.

I move the ray’s origin in the direction of the triangle normal vector by epsilon (1e-4f).
I think it’s a difference in ray origin, is that correct?
The ray’s length is 1e16f. Adjusting the ray’s length might solve the problem, but I want to understand the reasons behind it.

Is there a reference on how closest hit and intersection work?
I understand that intersection is implemented in hardware, is that correct?

Yes triangle intersection is implemented in hardware if you’re using an RTX-enabled GPU. The best reference for how closest-hit and intersection work is probably the OptiX Programming Guide.

So first off, the only hope we can have for a symmetric result between two triangles is if the ray is exactly the same in both directions. If our ray direction is not bitwise identical, with only the sign bits changed, then we can’t expect to have symmetric results. What this means specifically is that if we offset the ray along the normal of the triangle the ray starts from, then we will have two different rays, and we should expect the hit/miss results to reflect this asymmetry.

Second, always keep in mind that floating point quantizes your answers and introduces error. It is not possible in general with a floating point representation to guarantee that a ray originating from point (a) and heading to point (b) will hit (b) exactly at the point you aimed at. It’s very likely to end up nearby, and there is floating point error introduced in multiple places along the way, for example during ray direction calculations, during intersection, and during the reconstruction of your hit point after intersection.

Third, remember that casting a ray in the plane of a triangle will produce a miss. This means that you can easily construct a situation where the centroid of one triangle is in the plane of the other triangle, and in this case we should expect an asymmetric result. And similarly, the plane of a triangle could include the offset ray origin that you used. So maybe you’re experiencing this case by accident.

What is the scale of your triangles, how long is an edge typically? I’m curious about the magnitude of your ray origin, and whether adding a number like 1e-4 will work. I’m asking because the scale of your ray length 1e16 sounds disproportionately large. I’m not sure that causes any problems, but it might. It’s usually a problem to do floating point math on numbers with very different exponents, for example 1e16f + 1e-4f == 1e16f in fp32 floating point, so if your ray origin is anywhere near the scale of your ray direction, then the offset of 1e-4 is probably not doing anything.

The OptiX Programming Guide mentions the occurrence of the closest hit, but it doesn’t provide details on how it is determined, especially in the context of triangle intersection.
If there’s no RTX-enabled GPU, could it be that triangle intersections are not implemented in hardware, and a different algorithm is used?

Based on your first answer, it seems that you have a problem because the two triangles have different offsets and directions for the ray. To solve this issue, you will likely need to investigate further.

Regarding your second answer, I’m not sure how to proceed with this problem.

For the third answer, this is less likely to be the problem since you checked if the ray was in a plane by taking the dot product of the direction vector and the normal vector

The example points for the triangle are provided as follows:
(-650.12, 487.89, 855.70)
(-648.02, 480.29, 913.30)
(-658.41, 535.65, 887.00)
edge size : 60

The object’s bounding box size is (2445, 2750, 1680).

The ray offset seems to be working because the results are different for 1e-6f,1e-4f and 0.

You increased the ray length because setting it equal to the direction vector’s magnitude (normalized x) caused an issue where the ray didn’t reach the center of the triangle. Could this be related to floating-point precision issues?

If there’s no RTX-enabled GPU, could it be that triangle intersections are not implemented in hardware, and a different algorithm is used?

If your GPU is not RTX enabled, then yes there is a software fallback for the triangle intersector. It is not guaranteed to produce bit-exact identical results, if I recall correctly, but it uses the same algorithm and should generally give you the same answer. The issue here isn’t a difference between results with RTX and non-RTX GPUs is it?

Right I think your summary #1 is correct - this just needs more investigation. Mostly I wanted to make clear that when expecting a symmetric result, and receiving an asymmetric result, the problem might not be the program, it might be somewhere in the expectation and assumptions. Mathematically, it’s easy to assume that you can reverse a ray and get the inverse answer, but in practice, floating point math can be very tricky when it comes to meeting such expectations. For exploring #2, you can take a look at your hit points and compare them to your computed centroids; I’m guessing you will notice a difference between them. This might help explain why you’re not getting exactly what you expected. With a single specific triangle and a specific ray, you can also solve for the ray intersection with the triangle on paper or using a one-off program with double precision. By computing the exact hit point, you can compare it to the single-precision hit point you’re computing and find out how far away it is from the high precision hit point.

Based on the scale of your triangle, I assume that your triangle normal is not normalized? I guess that if it were normalized, then the ray offsets in the range of 1e-4 or 1e-6 wouldn’t work.

You increased the ray length because setting it equal to the direction vector’s magnitude (normalized x) caused an issue where the ray didn’t reach the center of the triangle. Could this be related to floating-point precision issues?

That sounds like it might be a case of t_max being too small in the first case? I don’t think increasing the ray length would solve any precision issues, but it will allow the ray to travel further given the same t_max value.

The normal vector of the triangle is normalized (unit vector).
But what does it mean to normalize based on triangle size?

That was just describing the case what would happen when the face normal was not normalized inadvertently.
Then the ray origin offsets along the unnormalized face normal would have been differently far away from the triangle surface depending on the triangle size. If your face normals are normalized, all is fine.

I have a triangle mesh whose offset is not behaving as expected (causing self-intersection).

Again, these offsets used for self-intersection avoidance are scene size dependent and might need to be tuned to your scene coordinate range. If they are not behaving as you would expect, then they are either not the appropriate value for your scene, or there is something incorrect inside your algorithm.

Finally, please just implement the visibility ray method I described multiple times. It’s exactly the algorithm you need and will be faster than any of your closest hit experiments so far.
Mind that the visibility test between the offset ray origin and end points along the face normal direction above the two tested triangle sample points would actually return true for two adjacent coplanar triangles!
Not sure if you want that. If not, you would need to filter out these results by checking if the two triangles are coplanar when the visibility test succeeded.

The normal vector of the triangle is normalized (unit vector).
But what does it mean to normalize based on triangle size?

So just to explain what I meant… The smallest magnitude number in the example triangle you gave is 480.29 for the 2nd point’s Y value. Using IEEE-754’s FP32, the smallest representable delta between this value and it’s two floating point neighbors is 0.000030518, or ~3e-5. If you multiply a normalized vector by 1e-6f and then add that to any of the values in your example triangle, then none of the numbers will change, because 1e-6f is smaller than 0.5 ULP (Unit of Last Place, aka Least Significant Bit) of any value that is 480.29 or larger in magnitude (And all 3 channels of your example triangle centroid, or any hitpoint on the triangle, are guaranteed to be equal or larger in magnitude). In other words, using an offset factor of 1e-6f will not move or affect your hit point, and thus cannot help solve any self-intersection problems. I expect that an offset of 1e-4f did work, but that you should see no difference between an offset factor of 1e-6 and zero. If you did see a difference between 1e-6 and zero, then something about your setup may be different than you expect, which could potentially help explain some of the discrepancies you’ve noticed.