the speedup I got switching from our own Polygon-intersection routines to the build-in Triangle-intersection routine is really great. It seems to be that it also fixed the initial single-precision related errors we got.
Yet, realizing there is a new problem.
We launch rays from a starting polygon. Without a large enough scene epsilon there are many self-intersections with the origin of the ray. Previously I caught these with an ID check of the primitive. Scene epsilon has the problem, that some rays just pass through other polygons/triangles which are very close to the rays origin, yet still of interest.
Taking Strange intersection issue into account I’d assume, that the best option – to keep making use of RTX features with the builtin algorithm – is to use an any-hit routine to filter out self-intersections.
Are there any other possibilities or could a self-intersection check via PrimitiveIDs be part as a feature for an upcoming OptiX version?