Question about SBTs

I am only dealing with regular triangle geometry and no instancing.

Ok, then depending on the number of primitives you have the option in OptiX to either put all triangles into one huge Geometry Acceleration Structure (GAS) or, if that exceeds the maximum primitive limit per GAS (see OptixDeviceProperty), under individual OptixInstance objects inside an Instance Acceleration Structure (IAS) with each instance holding a GAS, which in turn can be one or multiple meshes.

But multiple geometry types, each with their own set of multiple hit/miss programs.

Let’s call “geometry types” == “shapes”, then it’s less confusing with geometric primitive types (triangles, curves, custom).

Please clarify, you mean one hitgroup per shape or multiple hitgroups?
(The former means geometries are sorted by material)

I need to run different queries against the same geometry

That is normally handled with different pipelines and maybe different Shader Binding Tables, but depends on what you store along with the SBT records.

and/or swap out geometry during execution.

That effectively means different acceleration structures. Depending on what you store with the SBT data, your would normally need to update/rebuild these as well, for example to access the triangle indices and additional vertex attributes.

If you think about buildings, a wall getting hit during one query “means” something different than if I run a different query. Windows can be ignored sometimes, but the building hits after not… that sort of problem.

OptiX supports visibility masks which can be used to ignore specific geometry. These are stored at OptixInstance objects which means you could make good use of a top-level IAS for that.

My understanding is that there is one SBT record per Geometry/Mesh/Primitive per RayType. (The Optix 6 raytype translates to a set of Hit programs, a program group.)

Yes, when dealing with a single GAS you have at least one SBT entry per shape per ray type.

But you can index the SBT in a very flexible manner, that is a matter of understanding this formula. This is absolutely crucial! 7.3 SBT + AS

sbt-index = sbt-instance-offset + (sbt-geometry-acceleration-structure-index * sbt-stride-from-trace-call) + sbt-offset-from-trace-call

Have a look at this as well: SBT problem when using multiple GAS objects. - #3 by Mattivc

Basically there is one SBT for the whole scene.

Well, normally yes. It’s more like one SBT per pipeline per whatever you consider your set of scene traversables reachable from a top-level traversable.

So if I have one group of 5 meshes and another of 2, I’d have at least 7 records, 14 with 2 ray types.

Yes, for a single GAS.

And that really is the only/direct connection between meshes and hit programs.
The index of the geometry (or rather the indices in the vertex/index arrays) and the position in the SBT.

Yes, for a single GAS. The sbt-index inside the above formular selects the SBT entry.
There is more SBT flexibility when using instances because then you have two more values to work with in that formula.
(There is the instance index and the user defined instance ID you can abuse for indexing tricks.)

I guess when meshes in one group have 5 different ray types then all of the meshes have to have 5 SBT entries = 35 records total?

The question you need to ask yourself is, do you really need so many different ray types, or could you just have the same hitgroup shaders handling multiple cases?
What “type” of ray you’re shooting can be encoded in a per-ray payload value or flags bit.

Now, if I have different “sets” of programs that need to run depending on the query I run, is this where pipelines come in and I just swap those out to load a different set of programs/program groups, but maintain the SBT?

No, you cannot change just the pipeline. The SBT contains program records (the 32 bytes header you get with optixSbtRecordPackHeader) of the called programs and these need to match your pipeline.

If the max ray types is now lower, say 3, the way I understand the documentation then I’d still need to have 5 records per geometry, just pointing at different programs, some never used.

Well, the SBT layout might stay identical but you would need to exchange the SBT program record headers anyway when changing between pipelines with different modules and then it makes sense to have pairs of matching pipelines and SBTs.
(Look at how I exchange the hit group headers between opaque and cutout opacity shaders when toggling that material property in my intro examples.)

We’re getting in the “it depends” territory here. :-)

Is that the rough outline?
Basically I think I am trying to fit this into an entity relationship model in my head, which probably doesn’t work well to begin with. :)

Yes, for a single GAS with individual meshes and one or more SBT entry per mesh, the SBT indexing flexibility is limited.

Keep reading the Chapter 7 Shader Binding Tables and esp. the Chapter 7.3 Acceleration Structures in there as often as you need to understand all possibilities of that SBT indexing formula.

David posted a very nice link about that as well in another thread. It’s interactive!
Here’s an external blog post that compares shader binding tables in OptiX 7 to DXR and VKR: