Build engine from onnx model, can not set dynamic shape right


I try to export my onnx(set dynmiac axes already) model to trt engine with dynamic shapes. I try to configured optimized profile to set the dynamic shapes, but failed.


TensorRT Version: 8…6.1
GPU Type: RTX3090
Nvidia Driver Version: 11.3
CUDA Version: 11.7
CUDNN Version:
Operating System + Version: ubuntu 20.04
Python Version (if applicable): 3.8.10
TensorFlow Version (if applicable):
PyTorch Version (if applicable): 1.12
Baremetal or Container (if container which image + tag):

Relevant Files

Please attach or include links to any models, data, files, or scripts necessary to reproduce your issue. (Github repo, Google Drive, Dropbox, etc.)
code: (4.5 KB)

decoder.onnx (2.4 MB)

Steps To Reproduce

Change the onnx_model_path and run, check engine.get_profile_shapes and profile.get_shape if are same

Please include:

  • Exact steps/commands to build your repro
  • Exact steps/commands to run your repro
  • Full traceback of errors encountered

Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:

  1. validating your model with the below snippet

import sys
import onnx
filename = yourONNXmodel
model = onnx.load(filename)
2) Try running your model with trtexec command.

In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging

The model and code I’ve already sent in Relevant Files block.
And I tried trtexec to convert the onnx, if I not set the min/opt/max dynamic shapes with command

${trtexec} --onnx=output/models/${model_name}.onnx \
    --saveEngine=output/models/${model_name}.engine \
    --dumpOutput \
    --dumpProfile \
    --dumpLayerInfo --useSpinWait \
    --separateProfileRun >output/models/log/trtexec.log

I will got:

If I set the dynamic shapes with command:


${trtexec} --onnx=output/models/${model_name}.onnx \
    --saveEngine=output/models/${model_name}.engine \
    --minShapes=element_states_batch:${min_batch}x136x128,element_hidden_states:${min_batch}x136x128,element_states_mask:${min_batch}x136,lane_states_batch:${min_batch}x128x128,lane_mask:${min_batch}x128,sparse_goals_2D:${min_batch}x1536x2,sparse_goals_2D_mask:${min_batch}x1536 \
    --optShapes=element_states_batch:${opt_batch}x136x128,element_hidden_states:${opt_batch}x136x128,element_states_mask:${opt_batch}x136,lane_states_batch:${opt_batch}x128x128,lane_mask:${opt_batch}x128,sparse_goals_2D:${opt_batch}x1536x2,sparse_goals_2D_mask:${opt_batch}x1536 \
    --maxShapes=element_states_batch:${max_batch}x136x128,element_hidden_states:${max_batch}x136x128,element_states_mask:${max_batch}x136,lane_states_batch:${max_batch}x128x128,lane_mask:${max_batch}x128,sparse_goals_2D:${max_batch}x1536x2,sparse_goals_2D_mask:${max_batch}x1536 \
    --dumpOutput \
    --dumpProfile \
    --dumpLayerInfo --useSpinWait \
    --separateProfileRun >output/models/log/trtexec.log

And I got this:

Whole trtexec log:
trtexec.log (10.0 KB)


Sorry for the delayed response, Are you still facing this issue?

Thank you.

That’s okay,
Already found operator problem and replace it, thanks for reply.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.