Description
I’m trying to convert a CO-DETR model to TensorRT and encountering the following error:
[10/30/2024-15:34:42] [E] [TRT] ModelImporter.cpp:726: While parsing node number 6009 [TopK -> "/transformer/TopK_output_cast_0"]:
[10/30/2024-15:34:42] [E] [TRT] ModelImporter.cpp:727: --- Begin node ---
[10/30/2024-15:34:42] [E] [TRT] ModelImporter.cpp:728: input: "/transformer/TopK_input_cast_0"
input: "/transformer/Reshape_26_output_0"
output: "/transformer/TopK_output_cast_0"
output: "/transformer/TopK_output_1"
name: "/transformer/TopK"
op_type: "TopK"
attribute {
name: "axis"
i: 1
type: INT
}
attribute {
name: "largest"
i: 1
type: INT
}
attribute {
name: "sorted"
i: 1
type: INT
}
[10/30/2024-15:34:42] [E] [TRT] ModelImporter.cpp:729: --- End node ---
[10/30/2024-15:34:42] [E] [TRT] ModelImporter.cpp:732: ERROR: ModelImporter.cpp:168 In function parseGraph:
[6] Invalid Node - /transformer/TopK
This version of TensorRT only supports input K as an initializer. Try applying constant folding on the model using Polygraphy: https://github.com/NVIDIA/TensorRT/tree/master/tools/Polygraphy/examples/cli/surgeon/02_folding_constants
[10/30/2024-15:34:42] [E] Failed to parse onnx file
[10/30/2024-15:34:42] [I] Finish parsing network model
[10/30/2024-15:34:42] [E] Parsing model failed
[10/30/2024-15:34:42] [E] Failed to create engine from model or file.
[10/30/2024-15:34:42] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8501] # trtexec --onnx=end2end.onnx --saveEngine=end2end.trt --workspace=65535 --verbose --plugins=../mmdeploy/lib/libmmdeploy_tensorrt_ops.so
I’m using the following Co-DETR model:
The steps are:
-
Run
tools/deploy.py
from MMDeploy to output the ONNX file:python mmdeploy/tools/deploy.py \ mmdeploy/configs/mmdet/detection/detection_onnxruntime-fp16_dynamic.py \ mmdetection/projects/CO-DETR/configs/codino/co_dino_5scale_swin_l_16xb1_16e_o365tococo.py \ mmdetection/checkpoints/co_dino_5scale_swin_large_16e_o365tococo-614254c9.pth \ mmdetection/demo/demo.jpg \ --work-dir mmdetection/checkpoints \ --device cuda:0 --show
-
Convert the output ONNX file using
trtexec
trtexec --onnx=end2end.onnx --saveEngine=end2end.trt --workspace=65535 --verbose --plugins=../mmdeploy/lib/libmmdeploy_tensorrt_ops.so
- The above error occurs
Please let me know the cause of this error and how to respond to it.
Environment
TensorRT Version: 8.5.1.7
GPU Type: Tesla V100
Nvidia Driver Version: 525.105.17
CUDA Version: 11.8
CUDNN Version: 8.7.0
Operating System + Version: nvcr.io/nvidia/tensorrt:22.12-py3
Python Version (if applicable): 3.8
TensorFlow Version (if applicable): None
PyTorch Version (if applicable): 2.0.0+cu118
Baremetal or Container (if container which image + tag): nvcr.io/nvidia/tensorrt:22.12-py3
Python packages
mmcv: 2.0.0
mmdeploy: 1.3.1
mmdeploy-runtime-gpu: 1.3.1
mmdet: 3.3.0
mmengine: 0.10.5
onnx: 1.17.0
onnxconverter-common: 1.14.0
onnxruntime-gpu: 1.15.1
openmim: 0.3.9
Relevant Files
https://drive.google.com/drive/folders/1ga0HHb3xLr4nm15jetgF6pt_Sv1CVfHc