Run two engines in IRuntime

Robotics & Edge Computing Jetson & Embedded Systems Jetson TX2
1

Hi,

I am using TensroRT on Jetson TX2/NX for object detection. I have one question, can I create two engines(models) in one IRuntime, and do inference one by one?

Some code like below to create two engines and do inference:

IRuntime* runtime = createInferRuntime(gLogger);
ICudaEngine* engine = runtime->deserializeCudaEngine(ModelStream, size);
IExecutionContext* context = engine->createExecutionContext();

ICudaEngine* engine2 = runtime->deserializeCudaEngine(ModelStream2, size);
IExecutionContext* context2 = engine2->createExecutionContext();**

context.enqueue(xxx);
//parse output of first model inference, then use it as input of the second model inference
context2.enqueue(xxx);

Is this possible? If not, how can I implement the two models inference one by one?

Thanks
Harry

3

Hi,

You can create two IRuntime to achieve this.

For example:

IRuntime* runtime1 = createInferRuntime(gLogger);
ICudaEngine* engine1 = runtime->deserializeCudaEngine(ModelStream, size);
IExecutionContext* context1 = engine->createExecutionContext();

IRuntime* runtime2 = createInferRuntime(gLogger);
ICudaEngine* engine2 = runtime2->deserializeCudaEngine(ModelStream2, size);
IExecutionContext* context2 = engine2->createExecutionContext();**

context1.enqueue(xxx);
//parse output of first model inference, then use it as input of the second model inference
context2.enqueue(xxx);

Thanks.