Material with alpha mapping

I want to build a material that is capable of reading an alpha map texture.

My anyhit program looks like this

RT_PROGRAM void any_hit()
    float3 alpha = make_float3(tex2D(alphaTex, texcoord.x, texcoord.y));
    float opacity = (alpha.x + alpha.y + alpha.z) / 3.0f;
    if(opacity < 0.5f)
        prd_shadow.attenuation = make_float3(0.0f);

This works with shadows. A ray is traced and the any program is called to test potencial intersections. The shadow rays go through the sections of the model where the alpha is 0, the problem is that even ignoring the intersection in the any hit program, the closest hit program doesn’t ignore the ray.

So the solution I found was, in the closest hit program, read the alpha map and trace another ray, like it the previous one was ignored.

Am I missing something?

You would need to assign such an any_hit program to both the radiance and the shadow raytype.

If I undestood it right, the any_hit program is called for any potencial intersection during the processing of an intersection program. I thought that the any_hit program would be always called.

In cases where a ray type doesn’t have an associated any_hit program, all calls to rtPotencialIntersection are going to return true?

That’s not quite right.

More precisely rtPotentialIntersection() is about the intersection with the basic geometry, being inside a t_min, t_max interval.
See the OptiX Programming Guide chapter 4.7.2. Reporting Intersections.

Note that the vertex attributes must be calculated between rtPotentialIntersection() and rtReportIntersection() and the latter is the one using the material index to pick the right any_hit and closest_hit assigned to a (material, ray_type) pair. These programs can then use these attributes like in your example, to look up an alpha test texture etc., so rtReportIntersection() is where the any_hit program can influence the behavior.

Directly from the OptiX Programming Manual: “Should that any_hit program invalidate the intersection via the rtIgnoreIntersection function, then rtReportIntersection will return false. Otherwise, it will return true.”

The default programs are empty, means for the any_hit program NOT calling rtIgnoreIntersection() or rtTerminateRay(), so all hits will reach the closest_hit program.
The OptiX Programming Guide paragraph 3.5.2 Material explains that.

Also note that any_hit invocations have no specific order. (That depends on the acceleration structure traversal.)

Thanks Detlef Roettger, now it is clear as crystal. I implemented an any_hit program associated with my closest_hit program and it is working fine.