Conversion of FasterRCNN ONNX model to TensorRT fails

AI & Data Science Deep Learning (Training & Inference) TensorRT
, ,
1

Description

I am trying to convert an ONNX FasterRCNN based object detection model to a TensorRT engine and running into an error with shapes in the ROI heads.

Screenshot 2024-11-12 at 11.07.16 AM
Screenshot 2024-11-12 at 11.07.16 AM1523×147 31.8 KB

Here is an example script that will allow one to recreate the error:

import os
import tensorrt as trt

if __name__ == '__main__':
    x_shape = (2,2,300,400)
    DIR_NAME = os.path.dirname(__file__)
    TRT_LOGGER = trt.Logger(trt.Logger.VERBOSE)
    trt.init_libnvinfer_plugins(TRT_LOGGER, '')

    builder = trt.Builder(TRT_LOGGER)
    network = builder.create_network()
    parser = trt.OnnxParser(network, builder.logger)

    onnx_path = os.path.realpath('model_zoo_rcnn.onnx')
    with open(onnx_path, "rb") as f:
        if not parser.parse(f.read()):
            TRT_LOGGER.log(trt.Logger.ERROR, f"Failed to load ONNX file: {onnx_path}")
            for error in range(parser.num_errors):
                TRT_LOGGER.log( trt.Logger.ERROR, parser.get_error(error))
            raise RuntimeError(f'Failed to load ONNX file: {onnx_path}')
        
    engine_path = os.path.realpath('model_zoo_rcnn.engine')
    engine_dir = os.path.dirname(engine_path)
    os.makedirs(engine_dir, exist_ok=True)

    inputs = [network.get_input(i) for i in range(network.num_inputs)]
    
    config = builder.create_builder_config()
    config.profiling_verbosity = trt.ProfilingVerbosity.DETAILED

    engine_str = builder.build_serialized_network(network, config)
    with open(engine_path, "wb") as f:
        f.write(engine_str)

And here is the model used for this example:

This occurs when using the default TensorRT plugin based ROIAlign node. It occurs even when an exported ONNX model includes a fixed batch size/input and output tensor shapes. These issues appear to occur with any RCNN based model, including ones from the ONNX Model Zoo.

Does the input ONNX file/graph need to be modified to use dynamic shapes in this node or set the node’s inputs/outputs to be fixed? If so, how can this be done? If this is not the issue, what is causing this error and how can it be fixed?

Environment

TensorRT Version: 10.5
GPU Type: Nvidia Jetson Orin
Nvidia Driver Version: 540.4.0
CUDA Version: 12.6
CUDNN Version:
Operating System + Version: Ubuntu 22.04.04
Python Version (if applicable): 3.10.15
Baremetal or Container (if container which image + tag): Baremetal

Relevant Files

Log from loading ONNX graph:
faster_rcnn_ex.log (1.5 MB)

1 Like
2

I see this previous post from May 2023. Is the ONNX to TRT converter fixed to support this file type?

3

I have been able to get this workaround to work. We can do this through our onnx_graphsurgeon tool by forcing the condition to be a constant. TRT is able to optimize out the conditional entirely in this case.Modification script:

import onnx_graphsurgeon as gs
import numpy as np

const_else = gs.Constant("const_else", np.array([0]).astype(np.bool_))

g = gs.import_onnx(onnx.load("fasterrcnn.onnx"))

# Overwrite condition to a constant.
for n in g.nodes:
    if n.op == "If":
        n.inputs = [const_else]
        
onnx.save(gs.export_onnx(g), "fasterrcnn_constelse.onnx")