TensorRT optimization profile

Description

Hello everyone,

I’m new in using TensorRT Python API.
Could you help me to migrate simple angle prediction model from Keras framework to TensorRT via ONNX?

Environment

TensorRT Version: 7.2.3.4
GPU Type: GeForce GTX 1060 6 GB
Nvidia Driver Version: 440.33.01
CUDA Version: 10.2
CUDNN Version: 7.1
Operating System + Version: Ubuntu 18.04
Python Version (if applicable): 3.6
TensorFlow Version (if applicable): 2.3.1

Relevant Files

Issue refer to this one:

My model has two dynamic input. I had set up optimization profiles, but I can not transfer it to engine bulder.

 profile.set_shape('input_5', (1, 64, 64, 3), (20, 64, 64, 3), (100, 64, 64, 3))
 profile.set_shape('input_6', (1, 3), (20, 3), (100, 3))
 config.add_optimization_profile(profile)

 engine = builder.build_cuda_engine(network)

On the last step I have an error:

[TensorRT] ERROR: Network has dynamic or shape inputs, but no optimization profile has been defined.

Could you please provide some examples with dynamic shapes on Python TensorRT and help me?

Hi,
Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:

  1. validating your model with the below snippet

check_model.py

import sys
import onnx
filename = yourONNXmodel
model = onnx.load(filename)
onnx.checker.check_model(model).
2) Try running your model with trtexec command.

In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging
Thanks!

Hi NVES!
I have reply to you on the private message.

Hi @v.stadnichuk,

Hope following link will help you.
https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#work_dynamic_shapes

Thank you.