Questions about denoiser

[FTC]

Hello team:
I have tested the sample code of optixDenoiser. However, I don’t know how to generate the input exr files, especially soane-BSDF, soane-Flow and soane-Normal(they are in NVIDIA-OptiX-SDK-7.3.0-linux64-x86_64/SDK/optixDenoiser/motiondata/). So I began to modify some basic code to have a try.
In optixSphere.cpp, I change the code as below (I want to add another sphere), but I can still see only one sphere:

            CUdeviceptr hitgroup_record;
            size_t      hitgroup_record_size = sizeof( HitGroupSbtRecord );
            CUDA_CHECK( cudaMalloc(
                    reinterpret_cast<void**>( &hitgroup_record ),
                    hitgroup_record_size * 2
                    ) );
            HitGroupSbtRecord hg_sbt[2];
            hg_sbt[0].data.sphere.center = { 0.8f, 0.0f, 0.0f };
            hg_sbt[0].data.sphere.radius = 0.5f;
            hg_sbt[1].data.sphere.center = { -0.8f, 0.0f, 0.0f };
            hg_sbt[1].data.sphere.radius = 0.5f;
            OPTIX_CHECK( optixSbtRecordPackHeader( hitgroup_prog_group, &hg_sbt[0] ) );
            OPTIX_CHECK( optixSbtRecordPackHeader( hitgroup_prog_group, &hg_sbt[1] ) );
            CUDA_CHECK( cudaMemcpy(
                        reinterpret_cast<void*>( hitgroup_record ),
                        hg_sbt,
                        hitgroup_record_size*2,
                        cudaMemcpyHostToDevice
                        ) );

            sbt.raygenRecord                = raygen_record;
            sbt.missRecordBase              = miss_record;
            sbt.missRecordStrideInBytes     = sizeof( MissSbtRecord );
            sbt.missRecordCount             = 1;
            sbt.hitgroupRecordBase          = hitgroup_record;
            sbt.hitgroupRecordStrideInBytes = sizeof( HitGroupSbtRecord );
            sbt.hitgroupRecordCount         = 2;

To sum up, I want to solve the following problems:
1. How to render multiple spheres, and how to put triangles and spheres in one scene.
2. How to generate flow/normal/BSDF pictures by optix.
To contact me, you can also mail 1773701277@qq.com or fangtiancheng@sjtu.edu.cn. I would be very appreciated if you can tell me how to solve these problems.

[droettger]

However, I don’t know how to generate the input exr files

Please read the OptiX Programming Guide about the denoiser inputs to see what data it expects in the different denoiser models.
https://raytracing-docs.nvidia.com/optix7/guide/index.html#ai_denoiser#nvidia-ai-denoiser
It’s recommended to use the HDR or AOV models.

Note that the optixSphere example wouldn’t need a denoiser because that is not implementing a progressive Monte Carlo renderer which would produce any noise, neither does the optixWhitted renderer which shows some more custom primitives.

How to render multiple spheres, and how to put triangles and spheres in one scene.

OptiX supports triangles and curves (linear, quadratic, cubic B-splines) as built-in geometric primitives.
The triangles have a built-in intersection program which doesn’t need to be set inside the shader binding table hit record for these primitives.
For the curves primitives you would need to query one of the built-in curve intersection programs and set it inside the hit record.
All other primitive types, like spheres, are custom geometric primitives in OptiX for which you need to provide an axis aligned bounding box element per primitive and your own intersection program calculating the ray-primitive intersection. The OptiX SDK contains examples for sphere and parallelogram intersection programs in different apps.

Built-in and custom primitives cannot be inside the same geometry acceleration structure (GAS). That means you would need to build at least two different GAS, one for the triangle geometric primitives and one for the custom sphere geometric primitives.
To get these into a scene, you would place these under and instance acceleration structure (IAS) with the two GAS holding the triangles and the spheres as two instances in that IAS.
(This specific render graph structure is indicated as special case in the OptixPipelineCompileOptions traversableGraphFlags as OPTIX_TRAVERSABLE_GRAPH_FLAG_ALLOW_SINGLE_LEVEL_INSTANCING.)

This is explained inside the OptiX programming guide chapter on acceleration structures:
https://raytracing-docs.nvidia.com/optix7/guide/index.html#acceleration_structures#acceleration-structures

If you want to render many sphere primitives, the optixSphere example is not a good foundation, since it only contains a single hardcoded sphere primitive which parameters (center, radius) are directly encoded inside additional data of a per shader binding table entry for the hit record. You cannot simply add more hit records with other such sphere parameters and expect that to work because that additional primitive is not contained inside the geometry acceleration structure (GAS) which is used as top level traversable handle for the optixTrace calls.
Means you’re not seeing the sphere you’ve added in another hit record because it neither exists inside the GAS, nor would your second hit record ever be called because you didn’t adjust the number of shader binding table (SBT) entries that single GAS used, which in turn requires an SBT instance offset to select the correct SBT hit record entry for your different primitives.
https://raytracing-docs.nvidia.com/optix7/guide/index.html#shader_binding_table#shader-binding-table

This is not a good approach if you want to render many custom primitives. For that you would simply place the sphere parameters (float3 center, float radius) into an array of float4 data, then calculate the axis aligned bounding boxes (AABB) around these spheres which are required for the custom primitive build input in the optixAccelBuild call.

You could then use a single hit record for all these spheres inside that single GAS. The intersection, closest hit and optional any hit programs in that hit record would use the optixGetPrimitiveIndex function to determine which sphere primitive has been hit. This works the same way for any geometric primitive type. With that index you can retrieve the per-primitive data, in this case the float4 holding the center and radius of the sphere, which you could store as pointer to that data array inside the global launch parameters if there is only one, or at additional per-hit record entry data inside the SBT similar to how it’s done with the hardcoded parameters inside the sphere example you cited above, just as a CUdeviceptr.
That per SBT data is retrieved inside the device programs with the OptiX device function optixGetSbtDataPointer.
https://raytracing-docs.nvidia.com/optix7/guide/index.html#device_side_functions#device-side-functions

How to generate flow/normal/BSDF pictures by optix.

The optixDenoiser example inside the SDK shows what OptiX API entry point functions to use when you have your input images with float or half components by simply loading some already rendered EXR images which are normally using the half format. That works the same way when you have rendered these images and have that data in the resp. CUDA memory buffers.

If you’re looking for examples which actually render an image with a path tracer and use the (HDR) denoiser on that, please have a look at the sticky posts of this sub-forum for links to more examples:
https://forums.developer.nvidia.com/t/optix-7-3-release/175373
The last link to the OptiX 7.2 release in there contains links to the OptiX 7 SIGGRAPH Course examples and more advanced examples. Both repositories contain example programs using the denoiser on interactively rendered images.

As you can see from the links to the OptiX Programming Guide, you should work through that and the OptiX SDK examples to learn how things work together.
This developer forum also contains a lot of additional information on top of the OptiX Programming Guide, including questions about sphere intersections and how to manage them in acceleration structures.

[FTC]

Hello team:
Thanks to your reply and NVIDIA’s detailed programming manual, I successfully rendered the normal pictures:

First, I found that the shading process is controlled by some functions on the pipeline, whose entrance named “__raygen__rg” “__closesthit__occlusion” “__closesthit__radiance”( in file optixPathTracer/optixPathTracer.cu ). Correspondingly, I also written three functions “__raygen__rg__normal” “__closesthit__occlusion__normal” “__closesthit__radiance__normal” as follow:

extern "C" __global__ void __raygen__rg__normal()
{
    const int    w   = params.width;
    const int    h   = params.height;
    const float3 eye = params.eye;
    const float3 U   = params.U;
    const float3 V   = params.V;
    const float3 W   = params.W;
    const uint3  idx = optixGetLaunchIndex();
    const int    subframe_index = params.subframe_index;

    unsigned int seed = tea<4>( idx.y*w + idx.x, subframe_index );

    float3 result = make_float3( 0.0f );
    // The center of each pixel is at fraction (0.5,0.5)
    const float2 subpixel_jitter = make_float2( rnd( seed ), rnd( seed ) );

    const float2 d = 2.0f * make_float2(
            ( static_cast<float>( idx.x ) + subpixel_jitter.x ) / static_cast<float>( w ),
            ( static_cast<float>( idx.y ) + subpixel_jitter.y ) / static_cast<float>( h )
            ) - 1.0f;
    float3 ray_direction = normalize(d.x*U + d.y*V + W);
    float3 ray_origin    = eye;

    RadiancePRD prd;
    prd.emitted      = make_float3(0.f);
    prd.radiance     = make_float3(0.f);
    prd.attenuation  = make_float3(1.f);
    prd.countEmitted = true;
    prd.done         = false;
    prd.seed         = seed;
    traceRadiance(
            params.handle,
            ray_origin,
            ray_direction,
            0.01f,  // tmin       // TODO: smarter offset
            1e16f,  // tmax
            &prd
    );
    // **********
    // we only need radiance
    result = prd.radiance ; // important
    // **********

    const uint3    launch_index = optixGetLaunchIndex();
    const unsigned int image_index  = launch_index.y * params.width + launch_index.x;
//    float3         accum_color  = result / static_cast<float>( params.samples_per_launch );

//    if( subframe_index > 0 )
//    {
//        const float                 a = 1.0f / static_cast<float>( subframe_index+1 );
//        const float3 accum_color_prev = make_float3( params.accum_buffer[ image_index ]);
//        accum_color = lerp( accum_color_prev, accum_color, a );
//    }
//    params.accum_buffer[ image_index ] = make_float4( result, 1.0f);
    params.frame_buffer[ image_index ] = make_color ( result );
}

extern "C" __global__ void __closesthit__occlusion__normal()
{
    // we don't need occlusion, so we have nothing to do
    return;
}

extern "C" __global__ void __closesthit__radiance__normal()
{
    // modified by ftc to generate normal image
    HitGroupData* rt_data = (HitGroupData*)optixGetSbtDataPointer();

    const int    prim_idx        = optixGetPrimitiveIndex();
    const float3 ray_dir         = optixGetWorldRayDirection();
    const int    vert_idx_offset = prim_idx*3;

    const float3 v0   = make_float3( rt_data->vertices[ vert_idx_offset+0 ] );
    const float3 v1   = make_float3( rt_data->vertices[ vert_idx_offset+1 ] );
    const float3 v2   = make_float3( rt_data->vertices[ vert_idx_offset+2 ] );
    const float3 N_0  = normalize( cross( v1-v0, v2-v0 ) );

    const float3 N    = faceforward( N_0, -ray_dir, N_0 );
    const float3 P    = optixGetWorldRayOrigin() + optixGetRayTmax()*ray_dir;

    RadiancePRD* prd = getPRD();
    // 判断是否发光
#ifndef  USE_NORMAL
    if( prd->countEmitted )
        prd->emitted = rt_data->emission_color;
    else
        prd->emitted = make_float3( 0.0f );
#else
    prd->emitted = make_float3( 0.0f );
#endif


    unsigned int seed = prd->seed;

    {
        const float z1 = rnd(seed);
        const float z2 = rnd(seed);

        float3 w_in;
        cosine_sample_hemisphere( z1, z2, w_in );
        Onb onb( N );
        onb.inverse_transform( w_in );
        prd->direction = w_in;
        prd->origin    = P;

        // we only need to set prd->radiance to abs(normal) and return
        prd->attenuation *= rt_data->diffuse_color; // It's no use


        prd->countEmitted = false;
    }

    const float z1 = rnd(seed);
    const float z2 = rnd(seed);
    prd->seed = seed;

    ParallelogramLight light = params.light;
    const float3 light_pos = light.corner + light.v1 * z1 + light.v2 * z2;

    // Calculate properties of light sample (for area based pdf)
    const float  Ldist = length(light_pos - P );
    const float3 L     = normalize(light_pos - P );
    const float  nDl   = dot( N, L );
    const float  LnDl  = -dot( light.normal, L );

    float weight = 0.0f;
    if( nDl > 0.0f && LnDl > 0.0f )
    {
        const bool occluded = traceOcclusion(
                params.handle,
                P,
                L,
                0.01f,         // tmin
                Ldist - 0.01f  // tmax
                );

        if( !occluded )
        {
            const float A = length(cross(light.v1, light.v2));
            weight = nDl * LnDl * A / (M_PIf * Ldist * Ldist);
        }
    }

    // important
    prd->radiance = make_float3(abs(N.x), abs(N.y), abs(N.z));
}

Finally, I modify pipeline function entrance in optixPathTracer.cpp void createProgramGroups( PathTracerState& ) as follows:

#ifdef USE_NORMAL
        raygen_prog_group_desc.raygen.entryFunctionName = "__raygen__rg__normal";
#else
        raygen_prog_group_desc.raygen.entryFunctionName = "__raygen__rg";
#endif
// The rest is the same

Then I get:

[droettger]

prd->radiance = make_float3(abs(N.x), abs(N.y), abs(N.z));

Mind that the denoiser takes the full range normal components in the range [-1.0,1.0] as input.
(I would also use fabsf() on floats.)

If that was just for visualization, then the better approach would be to use the scaled and biased normal to be able to see that the floor and roof normal point into opposite directions and the left and right walls’ normal point to mostly the negative x-axis in camera space with that view.

prd->radiance = N * 0.5f + 0.5f; // Scale and bias from [-1.0, 1.0] to [0.0, 1.0] range.

This might require the resp. overloads for operator*() and operator+() with float3 and float arguments. E.g. in here:
https://github.com/NVIDIA/OptiX_Apps/blob/master/apps/intro_denoiser/shaders/vector_math.h

Also note that the denoiser also doesn’t take object space normals you calculated, but camera space normals.
Please read the OptiX Programming Manual denoiser chapter once more.

Example code calculating the camera space normals when given a pinhole camera position and left-handed UVW coordinate system can be found here:
https://github.com/NVIDIA/OptiX_Apps/blob/master/apps/intro_denoiser/shaders/raygeneration.cu#L145