ONNX Slice Operator issue : Incorrect output shape when built by TensorRT

Description

I have pytorch model that crops 46x146 input to multiple 32x32 region and each region is fed to classifiers.

The (simplified) model is exported as “model_dummy.onnx” .
I checked the onnx file by the visualizer and
I confirmed that the onnx “Slice” operator is used and it has expected attributes (axis, starts, ends).

When I build the model by tensorRT on Jetson Xavier,
The debug output shows that slice operator outputs 1x1 regions instead of 32x32 regions.
The build script is “trt_runner_dummy.py” and the log file is “trt_runner_dummy.py.log”.
Part of the log is :

[TensorRT] VERBOSE: 3:Slice -> (3, 46, 146)
[TensorRT] VERBOSE: 4:Slice -> (3, 46, 146)
[TensorRT] VERBOSE: 5:Slice -> (3, 1, 146)
[TensorRT] VERBOSE: 6:Slice -> (3, 1, 1)

In the project, I use CNN as classifiers. So the incorrect 1x1 shape results in broken predictions.
How Can I fix this issue?

Thanks.

Environment

Environment A (where Model is trained and ONNX model is exported)

TensorRT Version: None
GPU Type: RTX1080
Nvidia Driver Version: 410.48
CUDA Version: 10.0
CUDNN Version: 7
Operating System + Version: Ubuntu16.04
Python Version (if applicable): 3.5.2
TensorFlow Version (if applicable): None
PyTorch Version (if applicable): 1.0.0
Baremetal or Container (if container which image + tag): Container based on nvidia/cuda:10.0-cudnn7-devel-ubuntu16.04
pip package ONNX Version: 1.5

Environment B (where build script (using TensorRT) is run)

Jetson Xavier
Jetpack Version: 4.3
Python Version (if applicable): 3.6.9

Relevant Files

Steps To Reproduce

Premise:

  • tesnorrt python source is at ‘/usr/src/tensorrt/samples/python’
  • “model_dummy.onnx” is in current directory

Command:

python3 trt_runner_dummy.py

Hi,

Merge these four slices into one slice where
slice = {0, 1, 2, 3}, start = {0, 0, 14, 28}, end = {1, 3, 46, 60}

Thanks

Thanks for your response.

I merged 4x4 slice ops to 4 slice ops as you suggested
(Actually, I dont need slice for axes 0,1. So I omitted them)

the figure below is the visualization of sub-graph after merge:

and the log by tensorrt is:

$ python3 trt_runner.py
[TensorRT] VERBOSE: 128:Slice -> (3, 1, 18)
[TensorRT] VERBOSE: 132:Slice -> (3, 1, 1)
[TensorRT] VERBOSE: 136:Slice -> (3, 1, 1)
[TensorRT] VERBOSE: 140:Slice -> (3, 1, 1)
[TensorRT] VERBOSE: /home/jenkins/workspace/TensorRT/helpers/rel-6.0/L1_Nightly/build/source/parsers/onnxOpenSource/builtin_op_importers.cpp:773: Convolution input dimensions: (3, 1, 18)
[TensorRT] VERBOSE: /home/jenkins/workspace/TensorRT/helpers/rel-6.0/L1_Nightly/build/source/parsers/onnxOpenSource/builtin_op_importers.cpp:840: Using kernel: (7, 7), strides: (2, 2), padding: (3, 3), dilations: (1, 1), numOutputs: 64
[TensorRT] VERBOSE: /home/jenkins/workspace/TensorRT/helpers/rel-6.0/L1_Nightly/build/source/parsers/onnxOpenSource/builtin_op_importers.cpp:841: Convolution output dimensions: (64, 1, 9)
[TensorRT] VERBOSE: 141:Conv -> (64, 1, 9)
[TensorRT] VERBOSE: 142:BatchNormalization -> (64, 1, 9)
[TensorRT] VERBOSE: 143:Relu -> (64, 1, 9)
[TensorRT] VERBOSE: 144:MaxPool -> (64, 1, 5)

As you can see, I got another incorrect shape after merge.

I looks like a bug of the build engine.

What should I do to fix this?