Hi Team,
When we ran the semantic segmentation model and converted to TensorRT it was successfull but when we added a transformer module(Penoptic segmentation model) to it, the Conversion to Trt is giving Segmentation fault error for TensorRT 8.6 EA version and Reshape error for lower version of TensorRT
We are using the trtexec file to convert the onnx file to TensorRT. Which is failing.
os.system(f"/usr/src/tensorrt/bin/trtexec --onnx={onnx_model_path} --saveEngine={trt_model_path} --fp16")
We also tried breaking the transformer model into multiple sub modules and try to run the TRT convertion seperately as below, but only the MaskAttention decoder layer is failing.
Below are the error logs for the same:
[06/07/2023-17:11:59] [I] [TRT] Searching for plugin: NonZero, plugin_version: 1, plugin_namespace:
[06/07/2023-17:11:59] [E] [TRT] ModelImporter.cpp:773: While parsing node number 640 [NonZero → “onnx::Transpose_1225”]:
[06/07/2023-17:11:59] [E] [TRT] ModelImporter.cpp:774: — Begin node —
[06/07/2023-17:11:59] [E] [TRT] ModelImporter.cpp:775: input: “onnx::NonZero_1224”
output: “onnx::Transpose_1225”
name: “NonZero_640”
op_type: “NonZero”
doc_string: “/home//Documents//_vision_transformer_deployment_onnx/lib/models/hf_transformers/models/mask2former/modeling_mask2former.py(1916): forward\n/home//anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py(1099): _slow_forward\n/home//anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py(1111): _call_impl\n/home//Documents//_vision_transformer_deployment_onnx/lib/models/hf_transformers/models/mask2former/modeling_mask2former.py(2112): forward\n/home//anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py(1099): _slow_forward\n/home//anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py(1111): _call_impl\n/home//Documents//_vision_transformer_deployment_onnx/lib/models/_mask2former.py(80): forward\n/home//anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py(1099): _slow_forward\n/home//anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py(1111): _call_impl\n/home//Documents//_vision_transformer_deployment_onnx/lib/models/_mask2former.py(196): forward\n/home//anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py(1099): _slow_forward\n/home//anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py(1111): _call_impl\n/home//anaconda3/lib/python3.8/site-packages/torch/jit/_trace.py(118): wrapper\n/home//anaconda3/lib/python3.8/site-packages/torch/jit/_trace.py(127): forward\n/home//anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py(1111): _call_impl\n/home//anaconda3/lib/python3.8/site-packages/torch/jit/_trace.py(1166): _get_trace_graph\n/home//anaconda3/lib/python3.8/site-packages/torch/onnx/utils.py(388): _trace_and_get_graph_from_model\n/home//anaconda3/lib/python3.8/site-packages/torch/onnx/utils.py(437): _create_jit_graph\n/home//anaconda3/lib/python3.8/site-packages/torch/onnx/utils.py(496): _model_to_graph\n/home//anaconda3/lib/python3.8/site-packages/torch/onnx/utils.py(716): _export\n/home//anaconda3/lib/python3.8/site-packages/torch/onnx/utils.py(113): export\n/home/**/anaconda3/lib/python3.8/site-packages/torch/onnx/init.py(319): export\n07_export_onnx.py(96): main\n07_export_onnx.py(99): \n”
[06/07/2023-17:11:59] [E] [TRT] ModelImporter.cpp:776: — End node —
[06/07/2023-17:11:59] [E] [TRT] ModelImporter.cpp:778: ERROR: builtin_op_importers.cpp:4890 In function importFallbackPluginImporter:
[8] Assertion failed: creator && “Plugin not found, are the plugin name, version, and namespace correct?”
[06/07/2023-17:11:59] [E] Failed to parse onnx file
[06/07/2023-17:11:59] [I] Finish parsing network model
[06/07/2023-17:11:59] [E] Parsing model failed
[06/07/2023-17:11:59] [E] Failed to create engine from model or file.
[06/07/2023-17:11:59] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8401] # /usr/src/tensorrt/bin/trtexec --onnx=export/_vision_transformer_v0.1_vision_transform