TensorRT optimization profile


Hello everyone,

I’m new in using TensorRT Python API.
Could you help me to migrate simple angle prediction model from Keras framework to TensorRT via ONNX?


TensorRT Version:
GPU Type: GeForce GTX 1060 6 GB
Nvidia Driver Version: 440.33.01
CUDA Version: 10.2
CUDNN Version: 7.1
Operating System + Version: Ubuntu 18.04
Python Version (if applicable): 3.6
TensorFlow Version (if applicable): 2.3.1

Relevant Files

Issue refer to this one:

My model has two dynamic input. I had set up optimization profiles, but I can not transfer it to engine bulder.

 profile.set_shape('input_5', (1, 64, 64, 3), (20, 64, 64, 3), (100, 64, 64, 3))
 profile.set_shape('input_6', (1, 3), (20, 3), (100, 3))

 engine = builder.build_cuda_engine(network)

On the last step I have an error:

[TensorRT] ERROR: Network has dynamic or shape inputs, but no optimization profile has been defined.

Could you please provide some examples with dynamic shapes on Python TensorRT and help me?

Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:

  1. validating your model with the below snippet


import sys
import onnx
filename = yourONNXmodel
model = onnx.load(filename)
2) Try running your model with trtexec command.

In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging

I have reply to you on the private message.

Hi @v.stadnichuk,

Hope following link will help you.

Thank you.