How to convert saved_model to onnx to run with Jetson Inference

I’m trying to retrain a mobilenet model and to run it on jetson-inference.

I follow this link: tensorflow-onnx/ConvertingSSDMobilenetToONNX.ipynb at main · onnx/tensorflow-onnx · GitHub to convert the saved_model to onnx.

I could do that successfully. But when I tried to run that using detecnet in my jetson nano, I get the error " Unsupported ONNX data type: UINT8"
That I could fix with the following conversion:
import onnx_graphsurgeon as gs
import onnx
import numpy as np

graph = gs.import_onnx(onnx.load(“model.onnx”))
for inp in graph.inputs:
inp.dtype = np.float32

onnx.save(gs.export_onnx(graph), “updated_model.onnx”)

So after that, when I run ‘python3 detectnet.py “images/*.jpeg” result_30/found_%i.jpeg --network=updated_model.onnx --threshold=0.3’
I Get:
[TRT] ModelImporter.cpp:125: Resize__159 [Resize] inputs: [Transpose__144:0 → (1, 3, -1, -1)], [Concat__158:0 → (4)],
[TRT] ImporterContext.hpp:141: Registering layer: Resize__159 for ONNX node: Resize__159
ERROR: builtin_op_importers.cpp:2549 In function importResize:
[8] Assertion failed: scales.is_weights() && “Resize scales must be an initializer!”
[TRT] failed to parse ONNX model ‘updated_model.onnx’
[TRT] device GPU, failed to load updated_model.onnx
[TRT] detectNet – failed to initialize.
jetson.inference – detectNet failed to load network

Is there a way to load this external model successfully?
Loading the onnx with tf, works, but it takes too much time to do the inference.

Hi,

ONNX use INT8 as input format but TensorRT requires a float32 buffer.
Please check below comment to update the input data type via ONNX graphsurgeon.

Thanks.

Hi @AastaLLL , that’s exactly what I did… But after that I still have issues, please check my first comment. I need to kwow how I can get a success object detection.

HI @AastaLLL @kayccc ,

After I run onnx-graphsurgeon I get the following error:

ERROR: builtin_op_importers.cpp:2549 In function importResize: [8] Assertion failed: scales.is_weights() && “Resize scales must be an initializer!” [TRT] failed to parse ONNX model ‘updated_model.onnx’ [TRT] device GPU, failed to load updated_model.onnx [TRT] detectNet – failed to initialize. jetson.inference – detectNet failed to load network

Before trying jetson-inference with your model, first make sure that trtexec utility can load it (you can find trtexec under /usr/src/tensorrt/bin). If trtexec can’t parse your ONNX, neither will jetson-inference.

Also, jetson-inference contains pre/post-processing code that is specific to the network and framework it was trained in. This includes the code for applying the same pre-processing as was done in the framework (like the pixel normalization/standardization coefficients, mean pixel subtraction, channel layout, ect) in addition to the post-processing code that interprets the output bounding box / confidence tensors and applies clustering.

Currently the SSD-Mobilenet ONNX support in detectNet is setup for the PyTorch model:

So if you are able to get your ONNX loading, you would need to make sure the pre/post-processing is the same as done in TensorFlow. If possible it may be easier to just use the pytorch_ssd to create the model and run that in TensorRT, like in the jetson-inference tutorial.