Without providing the code of your anyhit program, I would guess that you didn’t call optixIgnoreIntersection()
inside it to let the BVH traversal continue.
Note that the anyhit program is not called in ray direction order but in BVH traversal order.
When using closest hit programs to gather all hits along a ray in ray direction order, there is no need to implement that recursively. That can be done easily with an iterative path tracer following the ray direction with different ray origin or t_min values until there is no hit anymore.
Please read these two posts and follow all links in there for more information:
https://forums.developer.nvidia.com/t/ray-mesh-intersections/170090/5
https://forums.developer.nvidia.com/t/does-there-update-ray-information-at-next-intersection-point-after-call-optixignoreintersection-in-any-hit/120040/2