I am trying to implement yolact_edge using TensorRT c++ APIs. I convert original PyTorch model to INT8 .trt model with torch2trt. The original model is splited into modules, such like the backbone, the FPN, the protonet, the prediction head…
After the converting, I tried to load the converted .trt model using C++ APIs, but it raise erros like “Serialization Error in verifyHeader: 0 (Magic tag does not match)”. I noticed that this error always caused by using different TensorRT versions. But the version of TensorRT I used to convert and load model is the sam.
Then I decided to convert original PyTorch model to .onnx model first, however I failed to convert one of the modules to .onnx model. It just raised a segmentation fault (core dumped) while converting the fpn_phase_1 to .onnx model(you can check the exact fpn_phase_1 module in yolact.py file). I don’t know how to fix that error.
TensorRT Version: TensorRT 22.214.171.124 GPU Type: RTX 3080 Nvidia Driver Version: 460.91.03 CUDA Version: 11.1 CUDNN Version: 8.2.1 Operating System + Version: Ubuntu 20.04 LTS Python Version (if applicable): 3.7.11 TensorFlow Version (if applicable): PyTorch Version (if applicable): 1.9.0 Baremetal or Container (if container which image + tag):
The eval.py is used to load and inference. During the inference, the model will be splited into modules and each module will be converted to both .onnx and .trt model using torch.onnx.export and torch2trt respectively.
yolact.py is the main model script. I add some torch.onnx.export codes to convert modules to .onnx. So if you want to check the export part, just search torch.onnx. I didn’t complete the conversion of prediction, because i don’t know how to do it. It seems like there are several predition layers and torch2trt convert all of them together. I don’t know how to do it using torch.onnx.export.
Steps To Reproduce
Just set up the env following the instruction, and run inference
Thanks for your reply!
Do you mean that the " Serialization Error in verifyHeader: 0 (Magic tag does not match)" is caused by the incompability between python INT8 and C++ INT8? The main problem for me, however, is the .trt engine file converted from .pth model directly using torch2trt can’t be loaded using c++ APIs, and the whole procedure uses the same TensorRT version.
I will attach outputs while using trtexec to test the .trt engine files