This recent thread and the links in there answer some of the AS size questions and options to influence that.
https://forums.developer.nvidia.com/t/how-to-speed-up-the-traversal-speed-of-bvh/256481/2
The main issue with bigger GAS is that building them needs a lot of temporary memory for the initial build and you probably won’t get very far with the 2GB on your Pascal board. Also compaction is much more efficient on RTX boards. Your mileage may vary.
I haven’t measured it but I would expect that many smaller GAS require more memory than one big one, before and after compaction, so merging them is a reasonable idea when possible. These can then also be instanced as a whole.
Mind the prefix sum necessary for the SBT offsets on the instances when using more than one SBT record in a GAS. The SBT index calculation and the specific example tables in this OptiX Programming Guide chapter explain that, specifically the last in section 7.3.5:
https://raytracing-docs.nvidia.com/optix7/guide/index.html#shader_binding_table#accelstruct-sbt
Having multiple materials inside one GAS means numSbtRecords > 1 and then you need the
sbtIndexOffsetBuffer data inside the AS build input to define which geometric primitive uses what SBT offset (== material) and that would need to be stored inside the AS. I never checked if there is a size difference. I’m normally using numSbtRecords == 1 because I used the instance SBT offset and user defined instance ID to select shaders and material parameters.
This is one of the cases where you need to simply benchmark that on your specific system configuration.