Two programs launch on one GPU concurrently

[aister91]

I am developing the simulation system with OptiX for the RTX platform.

My problem is launching the two OptiX programs simultaneously.

Two programs are defined as:
A.ptx and B.ptx
(example. A.ptx BSDF renderer and B.ptx makes LiDAR ray casting system)

Previously, I made the optixContext and pipeline for each. In this case, I make two acceleration structures for the same geometry because the acceleration structure seems dependent on context.

In other questions, I could read that the pipeline depends on the SBT.

If then, Can I build the program in one OptiX context with one AS (Scene) and launch it using multi-stream to run concurrently?

It would be nice if I could see this type of example.

Thanks, best regards.

[dhart]

Hi @aister91,

I don’t yet understand the question or the problems you’re encountering, can you elaborate on what the goal is and what is blocking your progress?

Do you truly need 2 optix pipelines to execute concurrently? Is this because they need to communicate with each other? Can your system be simplified to remove the need for 2 OptiX launches to run? Or combined into a single program?

While some people experiment with this kind of thing, it’s not a generally recommend practice to require 2 kernels to run concurrently. You can always launch multiple kernels at the same time, but for optimal performance they will each tend to try to occupy the entire GPU until they are near completion, unless your code is specifically syncronizing on external work. However, synchronizing on external work can be difficult to do well and we should explore simpler alternatives before going down that road.

It is true that if you use multiple streams and launch kernels independently on each, then they will execute asynchronously and somewhat concurrently. If you want more concurrency, then another option to consider is CUDA Green Contexts, where you can partition your GPU into some number of SMs for one context and a different set of SMs for another context. I have not seen anyone using Green Contexts with OptiX yet, so I’m not sure if or how well it works.

As far as sharing SBT or AS memory across two different pipelines, that is fine do to as long as they are compatible with both pipelines. Both the SBT and the ASes do sometimes depends on how your pipeline is setup and what options you’ve used, but not always.

–
David.

[aister91]

Thank you for your response.

I wanted to divide the roles of the two ray generation programs because LiDar module can be plural, unlike the render camera.

Therefore, I designed the ptx shader for each and then loaded the module individually.

OPTIX_CHECK(optixModuleCreateFromPTX(optixContext,
&moduleCompileOptions,
&pipelineCompileOptions,
ptxCode.c_str(),
ptxCode.size(),
log, &sizeof_log,
&module
));

OPTIX_CHECK(optixAccelBuild(optixContext,
/* stream */stream,
&accelOptions,
triangleInput.data(),
(int)numMeshes,
tempBuffer.d_pointer(),
tempBuffer.sizeInBytes,
outputBuffer.d_pointer(),
outputBuffer.sizeInBytes,
&asHandle,
&emitDesc, 1
));

I understand that the SBT and AS are built over “OptixContext”(Maybe my misunderstanding). In my implementation, the OptiX context is made multiple.

OptixProgram renderer(“A.ptx”);
OptixProgram LiDAR(“B.ptx”);

So I build previously Scene AS twice because of the different programs. This implementation is not efficient in my vision.

As far as sharing SBT or AS memory across two different pipelines, that is fine do to as long as they are compatible with both pipelines. Both the SBT and the ASes do sometimes depends on how your pipeline is setup and what options you’ve used, but not always.

Can I understand that the LiDAR program (‘B.context’) can use the handle created in the renderer program (‘A.context’)?

B.LaunchParam.handle = A.LaunchParam.handle;

Do you truly need 2 optix pipelines to execute concurrently? Is this because they need to communicate with each other? Can your system be simplified to remove the need for 2 OptiX launches to run? Or combined into a single program?

In the real world, camera and LiDAR systems have different frequencies, so if I divide the program is more helpful. (example. Until 3 frame capture in LiDAR system, vision system only captures 1 frame)

Anyway, I think I can try like this:

enum CameraType{
pinhole,
LiDAR
}
enum RayType{
radiance,
occlusion,
LiDAR
}

and check the camera type on raygen shader,

extern “C” global void raygen(){
///
if(CameraType == pinhole){
// general render pipeline
}else{
optixTrace(…, RAY_TYPE_LiDAR)
}
}

Would this orientation be more appropriate for OptiX?

[dhart]

An OptiX context is meant for managing a GPU, so if you only use 1 GPU, you probably only need 1 OptiX context. You can have multiple raygen programs and multiple SBTs and multiple pipelines all in a single context.

You can share an AS across multiple pipelines in the same context as long as the AS is compatible with both pipelines, meaning the AS options match both. This means the same geometry and the same build options are used, such as the motion, compaction, update, and random access flags.

So you could still keep your LiDAR and visible light renders as separate launches while only using 1 context. You can also put your LiDAR and light renders into separate modules of a single pipeline as well. This might simplify your life a bit and save some compute time if you only need to build one BVH instead of two.

You can do as shown in your last code snippet, if you want. I don’t think it would hurt performance if you put both camera types into a single program and select between them at run time. Your camera type would be in the launch params, perhaps. Since all the threads in any given launch will make the same decision, this kind of setup will not cause extra divergence, and the runtime cost would be limited to checking the camera value (which will typically be a cache hit) and the conditional jump instruction.

But if you have other reasons to keep them separate and have 2 different raygen programs, that’s okay too. Based on what we’ve discussed so far I recommend seeing if you can use 1 context and 1 pipeline. With 2 raygen programs, you would technically have 2 OptixShaderBindingTable structs, but notice that only the raygenRecord is required to be different. You can optionally share the other record types, if you want to use the same shader programs for miss, hit, callables, and exceptions. If you want different shader programs, then you might have 2 separate SBTs, and that’s fine as well.

Okay, and back to the original question, in case I didn’t answer it yet. If you want 2 separate launches to run as concurrently as possible, just launch them asynchronously on 2 different streams. The optixLaunch() call is asynchronous by default. It’s up to you to ensure that you don’t have any accidental host-side synchronization calls. As long as the kernels aren’t accidentally serialized and they are independent, then they will run “concurrently”. This does not mean that they will spread out evenly, but they will run as quickly as possible, and have little or no gap between them.

–
David.

[aister91]

I think I’ve come to understand more about OptiX. Thank you.