First, splitting a GAS into multiple GAS should be straightforward.
You use the same routine calling optixAccelBuild but input fewer triangles.
Make sure that you partition these triangles into reasonably disjunct spatial blocks to make sure the resulting AABB over each GAS overlap as little as possible. A simple 3D grid would do.
Building multiple smaller GAS will also require less temporary memory.
If you really have a scene where a single GAS would exceed the maximum primitive count of 2^29 (== 536,870,912) that is a pretty huge model.
The acceleration structures are not small. I think I got about 625 MTris onto a 48 GB VRAM board in the past.
What is your system configuration?
Second, the SBT is rather flexible and depending on what you want to do (like what kind of materials you want to assign to that geometry) this would go from a single SBT hit record entry (all primitives use the same material shader, potentially with different parameters) to one SBT hit record entry per material shader (different primitives use a different material shaders, again potentially with different parameters) to one or more SBT hit record per instance.
I’m going to try now to modify the optixTriangle example to render two triangles, each in its own GAS, but any suggestions would be appreciated.
All my OptiX Advanced Examples are using a top-level instance acceleration structure (IAS) over multiple GAS.
They are linked inside the sticky posts of this sub-forum and many of the following threads link to source code explaining the different SBT layouts used.
Definitely read this one: https://forums.developer.nvidia.com/t/passing-per-vertex-attribute-data-into-a-shader-program/279321/2
The topic of splitting big GAS into multiple smaller ones has been discussed on this forum quite often.
https://forums.developer.nvidia.com/t/should-i-break-a-large-gas-into-multiple-smaller-ones/211603
https://forums.developer.nvidia.com/t/2-gases-insert-into-1-ias-problem/244155
https://forums.developer.nvidia.com/t/question-about-instance-acceleartion-struction/283898
https://forums.developer.nvidia.com/t/memory-consumption-relation-on-gas/256454
https://forums.developer.nvidia.com/t/decomposing-bvh-to-accelerate-traversal/283911/2
There have been many threads about different SBT layouts on the forum.
Start here and read all threads linked inside these posts as well:
https://forums.developer.nvidia.com/t/question-about-instance-acceleartion-struction/283898/4
https://forums.developer.nvidia.com/t/explaining-hitgroup-records-creation-in-sutil-vs-path-tracer/201777/2
https://forums.developer.nvidia.com/t/basic-question-world-and-object-coordinates/169725/5