Some questions about your algorithm:
1.) Are your meshes considered to be solid?
That is, should triangle 18 and 10 be visible to each other or is a ray going through the the mesh volume considered a miss?
Meaning is your mesh object a closed volume with consistent triangle winding to define the front faces (that is , the outside of the surface) correctly and considered solid, so that visibility should only be started on the front faces of triangles?
Figure 2 in your previous post and the result table above seems to indicate otherwise. Is that really what you want?
2.) Are you doing any self-intersection avoidance when connecting the two triangles with a visibility ray?
That is, are you preventing that the triangle (e.g. 18) you’re starting the visibility ray from is hit itself by offsetting the ray origin away from the triangle surface to the correct side, or by offsetting the tmin by a positive epsilon value (or more robust but expensive by checking the primitive ID inside an anyhit program)?
3.) Similar for the destination sample point on the other triangle (e.g. 10).
If you’re checking the visibility between two geometric primitives, are you limiting the ray [tmin, tmax] interval to the distance between the two sample points on the two triangles tested for visibility?
(Or is your ray supposed to actually hit the destination triangle? Don’t do that, that would be slower.)
In case the visibility ray should only check if anything is between the two triangle sample points, then the ray tmax needs to be offset as well, this time with a negative epsilon to not result in the destination triangle to be hit and be considered as blocking the visibility test.
(This might be why 14 is a miss and 15 is not. It’s a floating point accuracy thing.)
Again, if you need to determine the condition of “any visibility” between two triangles on the surface of a mesh, your current algorithm connecting the centroids of triangles will only work with such simple geometry as in the pictures above. It will not work for the Stanford bunny or the duck you used as other examples.
For those you need to sample a lot more points between each triangle pair to make sure you resolve partial visibility conditions, like David explained before.
The fastest visibility ray implementation you can do in OptiX is described here:
https://forums.developer.nvidia.com/t/anyhit-program-as-shadow-ray-with-optix-7-2/181312/2
It should be rather straightforward to implement. It just needs a lot of rays, though these are the fastest kind.