The problem of time - consuming jump appears in TensorRT 7.0 accelerated yolov5s model reasoning

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

Hello! I’ve recently been using the tensorRT7.0 accelerated YOLOv5 model.
I use project is: https://github.com/wang-xinyu/tensorrtx/tree/master/yolov
Following the tutorial, I can normally implement accelerated reasoning for the model.
However, when I evaluated the time consumption, I found that the inference time consumption test was carried out for the same frame of image for many times.
The time consumption was different and fluctuated greatly, as shown in the figure below:

0814
08141280×810 534 KB

Environment

TensorRT Version: 7.0.0
GPU Type: TeslaV100-SXM2-32GB
Nvidia Driver Version: 418.67
CUDA Version: 10.0
CUDNN Version: 7.6.5
Operating System + Version: Ubuntu18.04
Python Version (if applicable): python3.6
TensorFlow Version (if applicable): /
PyTorch Version (if applicable): 1.4
Baremetal or Container (if container which image + tag): /

Relevant Files

Please attach or include links to any models, data, files, or scripts necessary to reproduce your issue. (Github repo, Google Drive, Dropbox, etc.)

Steps To Reproduce

Please include:

  • Exact steps/commands to build your repro
  • Exact steps/commands to run your repro
  • Full traceback of errors encountered
2

Hi @1965281904,
I followed the provided link, and could not reproduce the issue.
I can suggest you to try the same on latest TRT release.

Thanks!

3

By the latest version you mean： [TensorRT 7.1.3.4] ？

image
image835×816 46.9 KB

4

Yes, 7.1.3.4 is the latest TRT release available.
Thanks!

5

hi，Does the tensorRT version have to be the same as the CUDNN version? Or as long as it’s consistent with the CUDA version？

Because I currently want to use CUDA10.2, CUDNN7.6.5 and Tensorrt7.1.3.4, but I find that TensorRT7.1.3.4 is 8.0 for CUDNN version. Does this have any effect?

TensorRT-7.1.3.4.Ubuntu-18.04.x86_64-gnu.cuda-10.2.cudnn8.0.tar.gz

6

Hi @1965281904,
Please have a look at the compatible versions in the support matrix.

Thanks!