Welcome to the OptiX forum.
That is a very usual thing to do and there are multiple ways to provide additional per vertex attributes when using an IAS->GAS render graph structure.
You’re right, the GAS itself is only built using the vertex positions and any additional data needs to be provided via some indirection.
Even when using just a single GAS there are different ways.
This first method is the most common:
Define a Shader Binding Table (SBT) hit record structure which contains additional user data.
For vertex attributes that additional user data should be a structure with the pointer(s) to the CUDA device addresses pointing to the memory holding your attributes.
If you interleave the vertex attributes that needs only one CUdeviceptr for the attributes.
If you keep the vertex attributes in separate arrays, you’d need one pointer per attribute inside that struct.
If the topology (e.g. a triangle mesh) is defined with indices, you also need a device pointer to that to resolve the primitive index to triangle vertices.
The GAS build inputs for the optixAccelBuild define how many SBT records each GAS has.
Let’s take the simplest case with only one build input and only one SBT record.
(It’s possible to have multiple build inputs and then each of them has at least one SBT record, and it’s also possible to have multiple SBT records for one build input when using the sbtIndexOffsetBuffer to define which SBT hit record to use per primitive.)
You need to understand these two chapters in the OptiX Programming Guide for that:
https://raytracing-docs.nvidia.com/optix8/guide/index.html#acceleration_structures#accelstruct
https://raytracing-docs.nvidia.com/optix8/guide/index.html#shader_binding_table#shader-binding-table
Now you need to build your SBT hit records in the correct order so that each of your GAS accesses its own hit record with the additional user data you provided.
Inside the device code, you can access that user data via the OptiX device function optixGetSbtDataPointer.
You cast that pointer to the type of your vertex attribute struct and voila, you have access to all vertex attributes inside the hit programs.
If you search the OptiX SDK source code for that optixGetSbtDataPointer function, you’ll find multiple cases where that is used for different data.
Here’s some example code which shows this concept, though in that example I’m using an SBT hit record per instance which allowed to assign different materials to the same GAS, but the principle is the same.
1.) I’m using interleaved vertex attributes which are defined here
https://github.com/NVIDIA/OptiX_Apps/blob/master/apps/intro_runtime/shaders/vertex_attributes.h
2.) I’m defining a SBT hit record structure with additional user data here:
https://github.com/NVIDIA/OptiX_Apps/blob/master/apps/intro_runtime/inc/Application.h#L111
3.) That GeometryInstanceData contains the device pointers to the triangle mesh indices and interleaved VertexAttribute array and two indices into the MaterialParameter and LightDefinition arrays. Pointers to those inside the launch parameter struct SystemParameter above that.
https://github.com/NVIDIA/OptiX_Apps/blob/master/apps/intro_runtime/shaders/system_parameter.h#L76C22-L76C22
4.) Those indices and attributes pointers are filled when building the GAS:
https://github.com/NVIDIA/OptiX_Apps/blob/master/apps/intro_runtime/src/Application.cpp#L1425
5.) Later these are assigned to the correct SBT hit records. (This looks a little involved because of the per instance material method. It’ll look simpler when using different closesthit programs per material shader.)
https://github.com/NVIDIA/OptiX_Apps/blob/master/apps/intro_runtime/src/Application.cpp#L2085C19-L2085C19
6.) And finally your per GAS (or here per instance) user data is read inside the closesthit program like this:
https://github.com/NVIDIA/OptiX_Apps/blob/master/apps/intro_runtime/shaders/closesthit.cu#L129
and then interpolates the vertex attributes at the hit position of the ray on each triangle using optixGetPrimitiveIndex and optixGetTriangleBarycentrics and transforms them into world coordinates.
Above example uses as many SBT hit records as there are instances to be able to reuse the same GAS with different materials. That is a little wasteful and although it just works (I tried scenes with 8 million cow.obj model instances without issues) I wanted to have a simpler SBT layout.
So the rtigo10 and MDL_renderer examples inside that repository are showing how to build the smallest possible SBT and I would recommend following that SBT layout in your application because it’s more elegant.
This second method shows how to build an SBT with one hit record per material shader and still being able to assign this to different geometry.
The crucial formula how the effective SBT hit record is calculated is shown in this chapter.
You must understand how the SBT works to be able to architect such different SBT layouts.
https://raytracing-docs.nvidia.com/optix8/guide/index.html#shader_binding_table#accelstruct-sbt
1.) For that I’m not storing any additional user data on the SBT records, so it’s just the material shader program header (32 bytes).
https://github.com/NVIDIA/OptiX_Apps/blob/master/apps/rtigo10/inc/Device.h#L183
2.) With the two-level IAS->GAS render graph structure, the OptixInstance is unique and controls which SBT record to use via the sbtOffset field.
https://github.com/NVIDIA/OptiX_Apps/blob/master/apps/rtigo10/src/Device.cpp#L1456
3.) All per instance data is stored inside an array of GeometryInstanceData structures this time (not in SBT data):
https://github.com/NVIDIA/OptiX_Apps/blob/master/apps/rtigo10/src/Device.cpp#L1550C14-L1550C40
And as you see there, the geometry indices and attribute pointers are stored inside that.
4.) That array of GeometryInstanceData structures is indexed via the user defined OptixInstance instanceId:
https://github.com/NVIDIA/OptiX_Apps/blob/master/apps/rtigo10/src/Device.cpp#L1453
5.) And finally that instanceId index is read inside the OptiX device programs via optixGetInstanceId to access the data again:
https://github.com/NVIDIA/OptiX_Apps/blob/master/apps/rtigo10/shaders/brdf_diffuse.cu#L49
So both methods allow to assign arbitrary user data to a GAS, respectively to an instance.
In the first method it’s a matter of assigning the correct SBT hit record with user data to the GAS.
In the second method it’s using SBT hit records with no additional data and and instead uses the unique OptixInstance of a two-level IAS->GAS render graph to assign material shaders with the instance sbtOffset and indexes any user data via the instanceId.
All above examples use one SBT record per GAS! The SBT offset calculation is a little different when using multiple SBT records per GAS. It’s possible, it just needs a prefix sum of previous records to calculate the right sbtOffset. Shown inside this chapter of the OptiX Programming Guide.