Having trouble converting Pytorch Faster-RCNN to TensorRT Engine

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

I run into some shape issues (with IShuffleLayer) when trying to run trtexec on my onnx model, which is a faster rcnn model provided by pytorch model zoo.

Environment

TensorRT Version: 8.4.3-1+cuda11.6
GPU Type: 1 Quadro RTX 6000
Nvidia Driver Version:
CUDA Version: 11.6
CUDNN Version: Running nvcc --version gives me 11.6
Operating System + Version: Ubuntu 18.02
Python Version (if applicable): 3.9.12
PyTorch Version (if applicable): 1.12.1+cu116
Baremetal or Container (if container which image + tag): baremetal

Relevant Files

Please attach or include links to any models, data, files, or scripts necessary to reproduce your issue. (Github repo, Google Drive, Dropbox, etc.)

Code to Reproduce Faster-RCNN Model and Export to ONNX Model (named faster_rcnn_base.onnx):
export_pytorch_onnx.py (1.6 KB)

Steps To Reproduce

Please include:

  • Ran export_pytorch_onnx.py to create Faster-RCNN and export it to ONNX format.
  • Ran /usr/src/tensorrt/bin/trtexec --onnx=faster_rcnn_base.onnx --saveEngine=faster_rcnn_base_engine.trt --verbose
    Traceback can be found in this file:
    original_traceback.txt (127.2 KB)

Some additional steps I tried using to fix:
I tried using polygraphy to fold constants outlined in another post, but I ran into another error with with IShuffleLayer.

Polygraphy I ran: polygraphy surgeon sanitize --fold-constants faster_rcnn_base.onnx -o faster_rcnn_base_sanitized.onnx
Traceback after running trtexec --onnx=faster_rcnn_base.onnx --saveEngine=faster_rcnn_base_engine.trt --verbose can be found in this file:
polygraphy_traceback.txt (307.8 KB)

2

Hi ,
We recommend you to check the supported features from the below link.

You can refer below link for all the supported operators list.
For unsupported operators, you need to create a custom plugin to support the operation

Thanks!

5

Hi,

After we sanitize the model using Polygraphy, facing the following error.

[09/01/2022-08:30:24] [V] [TRT] Registering layer: Reshape_1330 for ONNX node: Reshape_1330
[09/01/2022-08:30:24] [E] Error[4]: [graphShapeAnalyzer.cpp::analyzeShapes::1294] Error Code 4: Miscellaneous (IShuffleLayer Reshape_1330: reshape changes volume. Reshaping [946181376] to [1,4624].)
[09/01/2022-08:30:24] [E] [TRT] parsers/onnx/ModelImporter.cpp:773: While parsing node number 357 [Reshape -> "onnx::Sigmoid_2547"]:
[09/01/2022-08:30:24] [E] [TRT] parsers/onnx/ModelImporter.cpp:774: --- Begin node ---
[09/01/2022-08:30:24] [E] [TRT] parsers/onnx/ModelImporter.cpp:775: input: "onnx::Reshape_2545"
input: "onnx::Reshape_2546"
output: "onnx::Sigmoid_2547"
name: "Reshape_1330"
op_type: "Reshape"

[09/01/2022-08:30:24] [E] [TRT] parsers/onnx/ModelImporter.cpp:776: --- End node ---
[09/01/2022-08:30:24] [E] [TRT] parsers/onnx/ModelImporter.cpp:778: ERROR: parsers/onnx/ModelImporter.cpp:180 In function parseGraph:
[6] Invalid Node - Reshape_1330
[graphShapeAnalyzer.cpp::analyzeShapes::1294] Error Code 4: Miscellaneous (IShuffleLayer Reshape_1330: reshape changes volume. Reshaping [946181376] to [1,4624].)
[09/01/2022-08:30:24] [E] Failed to parse onnx file
[09/01/2022-08:30:24] [I] Finish parsing network model
[09/01/2022-08:30:24] [E] Parsing model failed
[09/01/2022-08:30:24] [E] Failed to create engine from model or file.
[09/01/2022-08:30:24] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8401] # trtexec --onnx=faster_rcnn_base_sanitized.onnx --verbose --workspace=10000

Please refer to the following similar issue and fix your onnx model.

Thank you.

7

Thanks for your reply. How exactly do I modify the Reshape op? Is there a way to modify the onnx graph itself?

I’m using the torchvision models provided by pytorch, so I think all the reshape ops are using -1 in the first axis.

8

Hi,

Please checkout ONNX GraphSurgeon — ONNX GraphSurgeon 0.3.16 documentation

Thank you.