Export Pytorch's Yolo5 model to ONNX

Autonomous Machines Jetson & Embedded Systems Jetson Nano
,
1

Hello!

  • Do You have official script or guide for converting Pytorch’s model trained with Yolo v5 network into TensorRT’s usable ONNX format?

  • Does Pytorch version matter for conversion? I infere with TensorRT 8.0.1 on Jetson NANO (plz see below).

Environment

TensorRT Version: 8.0.1
GPU Type: Jetson Nano (Maxwell)
CUDA Version: 10.2
CUDNN Version: 8.2.1
Operating System + Version: Ubuntu 18.04 (Jetpack 4.6)

2

Hi,
Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:

  1. validating your model with the below snippet

check_model.py

import sys
import onnx
filename = yourONNXmodel
model = onnx.load(filename)
onnx.checker.check_model(model).
2) Try running your model with trtexec command.

In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging
Thanks!

5

Hi,

Hope the following doc may help you.
https://pytorch.org/tutorials/advanced/super_resolution_with_onnxruntime.html

While converting please make sure you’re using the supported opset version.
For other prerequisites please refer to the following support matrix doc.

We also recommend you to use the latest TensorRT version to get better performance.

Thank you.

6

I succeeded to export from Yolo5 model to ONNX and to run inference, but on Jetson Nano RAM usage is 1.1 GB when inferring frames regardless resolution. Is it normal for Nano to have such high RAM usage?
Are there some benchmarks of RAM usage ?

Model.engine file is 46 MB only…FP32

7

Hi,

We are moving this post to the Jetson Nano forum to get better help.

Thank you.

9

Hi,

cuDNN default occupies some memory for loading the binary.
To save this memory, you can try to run TensorRT without using cuDNN.

/usr/src/tensorrt/bin/trtexec --tacticSources=-CUDNN ...

Thanks.

10

Ok, what are disadvantages of not using CuDNN ?

11

Hi,

It depends on your model architecture.
Some TensorRT layer depends on cuDNN to infer.

If your model can run without cuDNN, you can turn it off to save the memory.

Thanks.

13

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.