Is that logical to combine optixTraverse with OPTIX_RAY_FLAG_ENFORCE_ANYHIT?

Gaming and Visualization Technologies Visualization OptiX
1

The benefit of optixTraverse is we don’t need actually launch the AH / CH shader. I am considering using optixTraverse for shadow ray test in my CH shader. Then, I found out optixInvoke doesn’t work here.

After removing optixInvoke, code compile and run. But the performance is even worse than a simple AH shader with optixTrace. My purpose is simply kill the ray without launch AH shader if hit object material is pure opaque.

But why optixTraverse could work with OPTIX_RAY_FLAG_ENFORCE_ANYHIT without crash? Does it stop at any random hit object?

Shadow ray code called in CH shader:

unsigned int u0, u1;
packPointer( prd, u0, u1 );

optixTraverse(handle, ray_origin, ray_direction, tmin, tmax, 0.0f, (mask),
        OPTIX_RAY_FLAG_ENFORCE_ANYHIT,
        RAY_TYPE_OCCLUSION,      // SBT offset
        RAY_TYPE_COUNT,          // SBT stride
        RAY_TYPE_OCCLUSION,      // missSBTIndex
        u0, u1);

if( optixHitObjectIsHit() ) {
    auto rt_data = (HitGroupData*)optixHitObjectGetSbtDataPointer();
    ShadowPRD* sprd = (ShadowPRD*)prd;
    if (rt_data->opacity == 1.0f) {
        sprd->attanuation = vec3(0);
        return;
    } else {
          // launch material shader?
    }
} else {
    // non block ray
}

Anyhit shader:

extern "C" __global__ void __anyhit__shadow()
{
    auto rt_data = (HitGroupData*)optixGetSbtDataPointer();
    auto prd = getPRD<ShadowPRD>();
    if (rt_data->opacity == 1.0f) {
        prd->attanuation = {};
        optixTerminateRay();
        return;
    }
// below code doesn't matter
2

Hi @iaomw, for shadow feeler rays, the fastest solution is

  optixTrace(...
OPTIX_RAY_FLAG_DISABLE_ANYHIT | OPTIX_RAY_FLAG_DISABLE_CLOSESTHIT | OPTIX_RAY_FLAG_TERMINATE_ON_FIRST_HIT,
...);

OPTIX_RAY_FLAG_TERMINATE_ON_FIRST_HIT will do what it says, that is stop at the first object along the ray, closest or not. The CH is disabled by OPTIX_RAY_FLAG_DISABLE_CLOSESTHIT .

See OptiX_Apps/apps/rtigo10/shaders/brdf_diffuse.cu at master · NVIDIA/OptiX_Apps for more details.