tensorRT3: is it possible to create two engines?

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

I want to split a network into two parts: the first part runs with int8 mode, and the second part runs with float mode. So two engines should be created, something like:

engine_1 = trt.lite.Engine(framework="caffe",
                             deployfile="model_1.prototxt",
                             modelfile="model_1.caffemodel",
                             max_batch_size=1,
                             max_workspace_size=(10 << 25),
                             input_nodes={"data_1":(CHANNEL,HEIGHT,WIDTH)},
                             output_nodes=["prob_1"],
                             data_type=trt.infer.DataType.INT8,
                             calibrator=int8_calibrator,
                             logger_severity=trt.infer.LogSeverity.INFO)

engine_2 = trt.lite.Engine(framework="caffe",
                             deployfile="model_2.prototxt",
                             modelfile="model_2.caffemodel",
                             max_batch_size=1,
                             max_workspace_size=(10 << 25),
                             input_nodes={"data_2":(CHANNEL,HEIGHT,WIDTH)},
                             output_nodes=["prob_2"],
                             logger_severity=trt.infer.LogSeverity.INFO)

It generates segfault.

I want to know whether it is possible to create two engines.

If not, what is the proper way to support mixed precision?

Thanks!

2

Hi,

Please noticed that INT8 is only available on GPU architecture 6.1 and 7.x, not for TX2.

We have checked dual TensorRT use-case before and it can run correctly.
For your use-case, it’s recommended to check each TensorRT functionality first.

Thanks.