Is there any overal introduction about how to convert pytorch detection model trt engine to accelerate by tensorrt？

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

I My knowledge about converting pytorch models to trt models is relatively confusing. I know that there are three ways. First, tensorflow’s built-in support, second, through onnx transfer, third, through python or c++ API to build the network layer by layer and build trt engine.

I use pytorch to train the network, and the latter two methods are optional. For the second method, detection networks generally have highly privatized post-processing, implemented in two parts? The inference part is converted through onnx to trt model, and the post-processing is implemented through CPU? This is more complicated. If the conversion from pytorch to onnx fails, do we need to build the onnx network structure layer by layer? Is there any reference code? I want to add the centernet head and dla-34 as the backbone network. How can I quickly achieve model conversion and trt acceleration? Looking forward to your reply

Environment

TensorRT Version：8.4.2:
GPU Type ：RTX a6000
Baremetal or Container (if container which image + tag):
pytorch/pytorch:22.08-py3

2

It is common to have problem converting pytorch model to onnx and to trt. Upgrading to newer version of pytorch/TRT might help. There is another way to convert pytorch to TRT using Torch-TRT.

4

I have a look at the torch2trt, this is a library about using tensorrt python api to convert torch model to trt engine. In my knowlodge, it register every converter of torch function and op, to attach some code after their invoking. However, in my case, there is a not supported layer, and I am confused how to build the model. The torch2trt lib is so simple about plugins, besides, my running platform is c++ env, how to construct python plugins in torch2trt to supprt the c++ env? Must i reconstruct the layer by using onnx ops, and fllow the pipeline of “torch–> onnx–>trt”? Is torch2trt sutable?

5

In torch2trt describe This includes support for some layers which may not be supported natively by TensorRT. Once this library is found in the system, the associated layer converters in torch2trt are implicitly enabled. which means when i build and install the plugins, and generate so lib file, the converter correspending to the plugin will be generated automatically?
I have build the “ReflectionPad2dPlugin” by cmake, but report import error, after “from torch2trt import torch_plugins”.