Hi,
Issue seems to be due to “EXPLICIT_BATCH” setting in the code.
In TRT 7, ONNX parser supports full-dimensions mode only. Your network definition must be created with the explicitBatch flag set (when using ONNX parser).
Since you are using TRT 6, please replace it with below code
with trt.Builder(TRT_LOGGER) as builder, builder.create_network() as network, trt.OnnxParser(network, TRT_LOGGER) as parser:
I tested on both TRT 6 (After code changes) and TRT 7 (without changes), it seems to be working fine on my test onnx model.
Engine Created 1: <class 'tensorrt.tensorrt.ICudaEngine'>
[TensorRT] WARNING: Current optimization profile is: 0. Please ensure there are no enqueued operations pending in this context prior to switching profiles
Context executed <class 'tensorrt.tensorrt.IExecutionContext'>
[TensorRT] WARNING: Explicit batch network detected and batch size specified, use enqueue without batch size instead.
using fp32 mode:
cost time: 0.00426483154296875
Engine Created 2: <class 'tensorrt.tensorrt.ICudaEngine'>
[TensorRT] WARNING: Current optimization profile is: 0. Please ensure there are no enqueued operations pending in this context prior to switching profiles
Context executed <class 'tensorrt.tensorrt.IExecutionContext'>
[TensorRT] WARNING: Explicit batch network detected and batch size specified, use enqueue without batch size instead.
<class 'numpy.ndarray'>
using fp16 mode:
cost time: 0.010645389556884766
Engine Created 3: <class 'tensorrt.tensorrt.ICudaEngine'>
[TensorRT] WARNING: Current optimization profile is: 0. Please ensure there are no enqueued operations pending in this context prior to switching profiles
Context executed <class 'tensorrt.tensorrt.IExecutionContext'>
[TensorRT] WARNING: Explicit batch network detected and batch size specified, use enqueue without batch size instead.
using int8 mode:
cost time: 0.01060032844543457
Serialized engine
FLtask.trt ---> File generated at the end
Thanks