How to convert saved_model to onnx to run with Jetson Inference

Robotics & Edge Computing Jetson & Embedded Systems Jetson Nano
1

I’m trying to retrain a mobilenet model and to run it on jetson-inference.

I follow this link: tensorflow-onnx/ConvertingSSDMobilenetToONNX.ipynb at main · onnx/tensorflow-onnx · GitHub to convert the saved_model to onnx.

I could do that successfully. But when I tried to run that using detecnet in my jetson nano, I get the error " Unsupported ONNX data type: UINT8"
That I could fix with the following conversion:
import onnx_graphsurgeon as gs
import onnx
import numpy as np

graph = gs.import_onnx(onnx.load(“model.onnx”))
for inp in graph.inputs:
inp.dtype = np.float32

onnx.save(gs.export_onnx(graph), “updated_model.onnx”)

So after that, when I run ‘python3 detectnet.py “images/*.jpeg” result_30/found_%i.jpeg --network=updated_model.onnx --threshold=0.3’
I Get:
[TRT] ModelImporter.cpp:125: Resize__159 [Resize] inputs: [Transpose__144:0 → (1, 3, -1, -1)], [Concat__158:0 → (4)],
[TRT] ImporterContext.hpp:141: Registering layer: Resize__159 for ONNX node: Resize__159
ERROR: builtin_op_importers.cpp:2549 In function importResize:
[8] Assertion failed: scales.is_weights() && “Resize scales must be an initializer!”
[TRT] failed to parse ONNX model ‘updated_model.onnx’
[TRT] device GPU, failed to load updated_model.onnx
[TRT] detectNet – failed to initialize.
jetson.inference – detectNet failed to load network

Is there a way to load this external model successfully?
Loading the onnx with tf, works, but it takes too much time to do the inference.

3

Hi,

ONNX use INT8 as input format but TensorRT requires a float32 buffer.
Please check below comment to update the input data type via ONNX graphsurgeon.

Thanks.

4

Hi @AastaLLL , that’s exactly what I did… But after that I still have issues, please check my first comment. I need to kwow how I can get a success object detection.

5

HI @AastaLLL @kayccc ,

After I run onnx-graphsurgeon I get the following error:

ERROR: builtin_op_importers.cpp:2549 In function importResize: [8] Assertion failed: scales.is_weights() && “Resize scales must be an initializer!” [TRT] failed to parse ONNX model ‘updated_model.onnx’ [TRT] device GPU, failed to load updated_model.onnx [TRT] detectNet – failed to initialize. jetson.inference – detectNet failed to load network

6

Before trying jetson-inference with your model, first make sure that trtexec utility can load it (you can find trtexec under /usr/src/tensorrt/bin). If trtexec can’t parse your ONNX, neither will jetson-inference.

Also, jetson-inference contains pre/post-processing code that is specific to the network and framework it was trained in. This includes the code for applying the same pre-processing as was done in the framework (like the pixel normalization/standardization coefficients, mean pixel subtraction, channel layout, ect) in addition to the post-processing code that interprets the output bounding box / confidence tensors and applies clustering.

Currently the SSD-Mobilenet ONNX support in detectNet is setup for the PyTorch model:

So if you are able to get your ONNX loading, you would need to make sure the pre/post-processing is the same as done in TensorFlow. If possible it may be easier to just use the pytorch_ssd to create the model and run that in TensorRT, like in the jetson-inference tutorial.