ONNX Slice Operator issue : Incorrect output shape when built by TensorRT

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

I have pytorch model that crops 46x146 input to multiple 32x32 region and each region is fed to classifiers.

The (simplified) model is exported as “model_dummy.onnx” .
I checked the onnx file by the visualizer and
I confirmed that the onnx “Slice” operator is used and it has expected attributes (axis, starts, ends).

When I build the model by tensorRT on Jetson Xavier,
The debug output shows that slice operator outputs 1x1 regions instead of 32x32 regions.
The build script is “trt_runner_dummy.py” and the log file is “trt_runner_dummy.py.log”.
Part of the log is :

[TensorRT] VERBOSE: 3:Slice → (3, 46, 146)
[TensorRT] VERBOSE: 4:Slice → (3, 46, 146)
[TensorRT] VERBOSE: 5:Slice → (3, 1, 146)
[TensorRT] VERBOSE: 6:Slice → (3, 1, 1)

In the project, I use CNN as classifiers. So the incorrect 1x1 shape results in broken predictions.
How Can I fix this issue?

Thanks.

Environment

Environment A (where Model is trained and ONNX model is exported)

TensorRT Version: None
GPU Type: RTX1080
Nvidia Driver Version: 410.48
CUDA Version: 10.0
CUDNN Version: 7
Operating System + Version: Ubuntu16.04
Python Version (if applicable): 3.5.2
TensorFlow Version (if applicable): None
PyTorch Version (if applicable): 1.0.0
Baremetal or Container (if container which image + tag): Container based on nvidia/cuda:10.0-cudnn7-devel-ubuntu16.04
pip package ONNX Version: 1.5

Environment B (where build script (using TensorRT) is run)

Jetson Xavier
Jetpack Version: 4.3
Python Version (if applicable): 3.6.9

Relevant Files

Steps To Reproduce

Premise:

  • tesnorrt python source is at ‘/usr/src/tensorrt/samples/python’
  • “model_dummy.onnx” is in current directory

Command:

python3 trt_runner_dummy.py

2

Hi,

Merge these four slices into one slice where
slice = {0, 1, 2, 3}, start = {0, 0, 14, 28}, end = {1, 3, 46, 60}

Thanks

3

Thanks for your response.

I merged 4x4 slice ops to 4 slice ops as you suggested
(Actually, I dont need slice for axes 0,1. So I omitted them)

the figure below is the visualization of sub-graph after merge:

SCREEN-SHOT
SCREEN-SHOT1750×605 98.4 KB

and the log by tensorrt is：

$ python3 trt_runner.py
[TensorRT] VERBOSE: 128:Slice → (3, 1, 18)
[TensorRT] VERBOSE: 132:Slice → (3, 1, 1)
[TensorRT] VERBOSE: 136:Slice → (3, 1, 1)
[TensorRT] VERBOSE: 140:Slice → (3, 1, 1)
[TensorRT] VERBOSE: /home/jenkins/workspace/TensorRT/helpers/rel-6.0/L1_Nightly/build/source/parsers/onnxOpenSource/builtin_op_importers.cpp:773: Convolution input dimensions: (3, 1, 18)
[TensorRT] VERBOSE: /home/jenkins/workspace/TensorRT/helpers/rel-6.0/L1_Nightly/build/source/parsers/onnxOpenSource/builtin_op_importers.cpp:840: Using kernel: (7, 7), strides: (2, 2), padding: (3, 3), dilations: (1, 1), numOutputs: 64
[TensorRT] VERBOSE: /home/jenkins/workspace/TensorRT/helpers/rel-6.0/L1_Nightly/build/source/parsers/onnxOpenSource/builtin_op_importers.cpp:841: Convolution output dimensions: (64, 1, 9)
[TensorRT] VERBOSE: 141:Conv → (64, 1, 9)
[TensorRT] VERBOSE: 142:BatchNormalization → (64, 1, 9)
[TensorRT] VERBOSE: 143:Relu → (64, 1, 9)
[TensorRT] VERBOSE: 144:MaxPool → (64, 1, 5)

As you can see, I got another incorrect shape after merge.

I looks like a bug of the build engine.

What should I do to fix this?

4

I remember this problem was due to the bug of onnx-tensorrt when importing onnx.

close this issue.

5

Hi,
Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:

  1. validating your model with the below snippet

check_model.py

import sys
import onnx
filename = yourONNXmodel
model = onnx.load(filename)
onnx.checker.check_model(model).
2) Try running your model with trtexec command.

In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging
Thanks!

8

Hi,
This issue has been resolved and no support is needed.

Thank you.

9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.