TRT engine file bigger than original Darknet YOLO V3 model

AI & Data Science Deep Learning (Training & Inference) TensorRT
,
1

Hello all,

Description

I am testing objectDetector_Yolo from deepstream-6.0/sources/, my goal is to test the model latency using different input sizes for the trt engine, as expected the latency increases if I reduce the input image size, however I noticed that this is not the case for the engine file.

Below you’ll find the input sizes I have tried and the engine file I got from TensorRT.

Input Size Engine file size (MB) Deepstream FPS gst-launch FPS
416x416 289 19 6.4
384x384 289 22 7.4
352x352 133 27 9.0
320x320 132 30 10.8
288x288 121 – –
256x256 288 – –
224x224 176 – –

Is there any reason why the engine file decreases and then increases again ?
Does the engine file size has an impact on the % of GPU that will be used by the application ?

Environment

TensorRT Version: 8.2.1.8
Deepstream: 6.0
GPU Type: Jetson TX2NX
Jetpack: 4.62.2 [L4T 32.7.2]
CUDA Version: 10.2
CUDNN Version: 8.2.1.32
Operating System + Version: Ubuntu 18.04
Baremetal or Container (if container which image + tag): Baremetal

Steps To Reproduce

After following the instructions from the README file in /opt/nvidia/deepstream/deepstream-6.0/sources/objectDetector_FasterRCNN/ I change the width in height in yolov3.cfg with the values from the above table and finally, I run:

$ deepstream-app -c deepstream_app_config_yoloV3.txt
2

It is expected that the same engine rebuilt may have different serialization sizes (due to variability in layer timings, etc). The engine file size depends on the tactics selected when you build the tensorrt engine. Different tactic has impact on memory consumption and end to end runtime latency.

5

Thank you for this clarification.

Then the input size does not impact the final engine file size ?

There is any way to see the selected tactics for every model ? more precisely the difference of the tactics between 2 engines generated from the same onnx model ?

6

Hi,

Please refer to the following doc.

Thank you.