Do You have official script or guide for converting Pytorch’s model trained with Yolo v5 network into TensorRT’s usable ONNX format?
Does Pytorch version matter for conversion? I infere with TensorRT 8.0.1 on Jetson NANO (plz see below).
TensorRT Version: 8.0.1
GPU Type: Jetson Nano (Maxwell)
CUDA Version: 10.2
CUDNN Version: 8.2.1
Operating System + Version: Ubuntu 18.04 (Jetpack 4.6)
Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:
validating your model with the below snippet
filename = yourONNXmodel
model = onnx.load(filename)
2) Try running your model with trtexec command.
In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging
Hope the following doc may help you.
Does Pytorch version matter for conversion?
While converting please make sure you’re using the supported opset version.
For other prerequisites please refer to the following support matrix doc.
These support matrices provide a look into the supported platforms, features, and hardware capabilities of the NVIDIA TensorRT 8.6.1 APIs, parsers, and layers.
We also recommend you to use the latest TensorRT version to get better performance.
I succeeded to export from Yolo5 model to ONNX and to run inference, but on Jetson Nano RAM usage is 1.1 GB when inferring frames regardless resolution. Is it normal for Nano to have such high RAM usage?
Are there some benchmarks of RAM usage ?
Model.engine file is 46 MB only…FP32
We are moving this post to the Jetson Nano forum to get better help.
cuDNN default occupies some memory for loading the binary.
To save this memory, you can try to run TensorRT without using cuDNN.
/usr/src/tensorrt/bin/trtexec --tacticSources=-CUDNN ...
Ok, what are disadvantages of not using CuDNN ?
It depends on your model architecture.
Some TensorRT layer depends on cuDNN to infer.
If your model can run without cuDNN, you can turn it off to save the memory.
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.