I have encountered some errors when trying to convert ONNX model to TensorRT.
I am using a pretrained SSD Lite MobileNet V2 model that I have retrained.
Firstly, I have converted my saved_model with the following command line:
Hi,
Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:
validating your model with the below snippet
check_model.py
import sys
import onnx
filename = yourONNXmodel
model = onnx.load(filename)
onnx.checker.check_model(model).
2) Try running your model with trtexec command.
In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging
Thanks!
According to your code, both ONNX models are valid.
Due to confidentiality issue, I cannot share my models with you, but you can find verbose text file output related to the TensorRT conversion of my UINT8 model (output.onnx) and my FLOAT32 model (output_float32.onnx).
Yes, I have seen other topics with UINT8 data issue.
I have also seen that there is a future update regarding this on this topic:
Do you know when it will be available please?
Because of the UINT8 data issue, I tried to modify my model input with FLOAT32 instead of UINT8, but I got the error (you can see the full error in the output_float32_onnx_trt_conversion_output.txt file) [graphShapeAnalyzer.cpp::nvinfer1::builder::anonymous-namespace’::ShapeNodeRemover::processCheck::587] Error Code 4: Internal Error ((Unnamed Layer* 43) [LoopOutput]_output: tensor volume exceeds (2^31)-1, dimensions are [2147483647,3])`
Do you have an idea how to fix errors for my FLOAT32 model please?
Currently, we do not have an idea of the approximate ETA.
As mentioned earlier, Currently, TRT does not support tensors with more than 2^31-1 elements. We do not have a workaround except for modifying the network.
I would like to share some updates regarding the conversion of my model to tensorrt.
I modified the overall steps like below:
I Took my saved model and specified its input shape to [1,576,720,3]
→ when converting from onnx to tensorrt: remove the “tensor volume exceeds (2^31)-1” error
I have converted its input to FLOAT32 (with the script I shared above)
I have converted NMS layers for TRT with the script below
I have converted this model to trt with the following command line: trtexec --onnx=ssd_lite_mobilenet_v2_input_shape_ops_13_float32_BatchedNMSDynamic_TRT.onnx --saveEngine=output.trt
The conversion of the ONNX model in TRT failed with the following error:
Could you please share ssd_lite_mobilenet_v2_input_shape_ops_13_float32.onnx?
The graphsurgeon script specifies the 2 required inputs, however the processed model has only one.
Thank you for your answers.
Actually, the “ssd_lite_mobilenet_v2_input_shape_ops_13_float32.onnx” model corresponds to the “test_nvidia_ops_13_input_shape_float32_BatchedNMSDynamic_TRT.onnx”, I just renamed it before shared it with you.
I did not try to modify inputs number on my own, is input number modified by ONNX conversion or somewhere else in scripts that I shared with you?
Could you please check and confirm again.
The above model has the Plugin nodes inserted by ONNX-graphsurgeon. We are looking for the source SSD model without the plugin nodes.
Based on the graphsurgeon script, we can see that ssd_lite_mobilenet_v2_input_shape_ops_13_float32_BatchedNMSDynamic_TRT.onnx is the output model (which you likely renamed). We need the input model below: