No, each ray can intersect with more than one triangle, but if that happens depends on how your application is implemented. Below is the longer explanation.
Note that all that information can be found inside the OptiX Programming Guide.
The acceleration structure (AS) traversal is started with an optixTrace call. That defines the current ray which is checked for intersections with the primitives inside the AS.
What happens with the current ray on intersections with the geometric primitives inside the AS depends on
- the OptixGeometryFlags used for the AS build inputs,
- how the shader binding table (SBT) is setup with what OptiX programs of what domains and
- what ray flags have been used in the optixTrace call.
The entry point of a ray tracing pipeline is always the ray generation program.
That shoots the primary ray with an optixTrace call and the SBT arguments in that call control which ray type to use by selecting the SBT index according to the formula at the beginning of the OptiX Programming Guide chapter 7.3 Acceleration Structures. That formula is crucial to understand how SBTs work.
The hit record inside the SBT can have three program domains: intersection, anyhit, and closesthit program. Which of them are needed and used depends on the application’s intent (means the above bullet points).
Built-in triangles have an implicit intersection program.
All other geometric primitives inside OptiX must set an intersection program, where the ones for built-in curves and spheres need to be queried from OptiX using optixBuiltinISModuleGet and only custom geometric primitives must implement their own intersection program.
Now the frequency of which OptiX program domains are called is different.
The intersection program is called for any axis aligned bounding box (AABB) over a geometric primitive with which the ray intersects during AS traversal. Means it’s the most often called program and should be implemented as performant as possible!
It’s called in AS traversal order, not in ray direction order!
Usually the intersection program changes the ray’s t_max values of the interval (t_min, t_max) defined by the optixTrace call, as long as a closer intersection is found, until it arrived at the closest hit. The intersection distance of that is the t_max value queried with the device function optixGetRayTmax.
Then depending on the intersection with the actual geometric primitive (triangle, curve, sphere, custom) the anyhit program is called when it’s defined inside the SBT hit record.
That means that anyhit program can be called multiple times per ray.
It can even be called multiple times per primitive if that was split into multiple AABBs during the AS build for performance optimizations. If you want to count primitives inside the anyhit program, you must set the OptixGeometryFlag OPTIX_GEOMETRY_FLAG_REQUIRE_SINGLE_ANYHIT_CALL.
Now if there is an anyhit program inside the SBT hit record, that can influence what happens with the currently traversed ray with the device calls optixTerminateRay after which the associated closesthit program is called. Mind that this is not necessarily the closest hit then.
This is one way to implement shadow/visibility ray types which only need to check if any geometry is inside the tested ray interval.
A similar but faster mechanism to achieve this (recommended) is the ray flag OPTIX_RAY_FLAG_TERMINATE_ON_FIRST_HIT which causes the very first hit, that is not ignored in anyhit, to abort further traversal, defining it as the closest hit. This is faster than calling into an anyhit program and the shadow ray implementation only requires a miss program then.
Demonstrated in this example: https://github.com/NVIDIA/OptiX_Apps/blob/master/apps/rtigo10/shaders/brdf_diffuse.cu#L188
The other device function which influence what happens with the current ray traversal is optixIgnoreIntersection. That causes the current potential intersection to be discarded, so t_max is not changed and the ray traversal continues.
If you reach the closest hit or miss program the current ray is done. Meaning the acceleration structure (bounding volume hierarchy) traversal started with the most recent optixTrace call is finished and nothing more is intersected by that ray.
Is there a limit to the coordinates of the triangle vertices? What requirements does the size of the triangle primitive need to meet to ensure that it can be detected by ray(sometimes the ray fails to intersect the triangle, and when I increase the size of the triangle, it succeeds)?
That cannot be answered without knowing the absolute values for the triangle and ray values of your case.
The OptiX ray is defined in 32 bit floating point precision. It’s most likely a floating point precision issue.
The effective floating point precision depends on the range of your values. Please search the web for “IEEE 754 floating point number precision” and you’ll find sites which explain what value range offers what precision.
The ray-triangle intersection routine itself is watertight in OptiX, means when shooting rays at the common edge of two adjacent triangles, the ray hits one of them.
In general it’s a good idea to make your world size extents reasonably small and to limit your ray t_max to that world size.