Index each gltf imported triangle with a unique primitiveID

[droettger]

Please read the whole OptiX 7 Programming Guide and look into the API Reference for individual functions.
https://raytracing-docs.nvidia.com/
https://raytracing-docs.nvidia.com/optix7/guide/index.html#introduction#

The OptiX 7 render graph setup with the different acceleration structure nodes (there are only two) is explained with diagrams here:
https://raytracing-docs.nvidia.com/optix7/guide/index.html#basic_concepts_and_definitions#acceleration-structures

optixGetInstanceId: https://raytracing-docs.nvidia.com/optix7/api/html/group__optix__device__api.html#ga4d70272186506b1a5cb4813f1047021f
optixGetInstanceIndex: https://raytracing-docs.nvidia.com/optix7/api/html/group__optix__device__api.html#gab0c73e334441d76fc2bbcf4c4cc4184c

If I want to be able the refer to the material of the primitive in the ch() program to determine parameters such as reflectivity, can I pass the material from the gltf file to OptiX, or would it be smarter to have an external referencefile in which look up my values from inside the ch()?

Not sure I understood that. You cannot access files from device programs. All data must reside in memory the device can access.
The glTF loader inside the utility scene functions of the OptiX SDK examples already handles materials defined inside the glTF file, but you’re free to pass any parameter to the hit programs you like.

The usual way to communicate data to specific programs is via the Shader Binding Table (SBT) record which effectively defines which program is executed on what ray tracing event (ray generation, any hit, closest hit, miss, exception).
Understanding the Shader Binding Table (SBT) is absolutely crucial when designing applications with OptiX 7, to be able to control which data is available to what program.
You must master this part: https://raytracing-docs.nvidia.com/optix7/guide/index.html#shader_binding_table#shader-binding-table
It looks more difficult than it is. The flexibility is quite astounding. There are multiple different ways to setup a SBT to achieve the same result and it depends on the application’s requirements what fits best.

Before I pass the data to OptiX I would have to “manually” add all the triangles into one mesh using world coordinates and then use the modified mesh to build my scene in OptiX?

Not in general.
This would only be required if you want to have all triangles uniquely defined inside a single geometry acceleration structure (GAS) with no instancing (aka. a completely flattened scene hierarchy.)

This is not always feasible because there is a limit of 2^29 primitives in one GAS and you will most likely run out of VRAM before that.
If you can instance identical geometry (means reusing the same GAS traversable handle in multiple OptixInstance children of an IAS) then it’s highly recommended to do that to save on AS build time and VRAM.
The instance transform in a two-level hierarchy is basically free on RTX boards because that is handled in hardware.

Does the Acceleration structure building rely on the gltf input (which, if I didn’t misunderstand it, already features some kind of bounding boxes/acceleration structures?)or will OptiX build acceleration structures for the triangles anyway independent of the input?

OptiX doesn’t know about any scene file formats.
You are responsible to provide the data for the acceleration build.
Your input defines how OptiX builds the acceleration structure.

There are two acceleration build inputs in OptiX 7.0.: For triangle primitives and for custom primitives.
The first gets triangle data, the second gets axis aligned bounding box (AABB) data you need to calculate on your own custom primitives. The custom primitives require a developer-defined intersection program, the triangles use a built-in one, on RTX board that’s in hardware.

Explained in more detail here: https://raytracing-docs.nvidia.com/optix7/guide/index.html#acceleration_structures#acceleration-structures