BIggest Latency in TensorRT

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

BIggest Latency in TensorRT

image
image1506×912 166 KB

How to continue ?

Environment

TensorRT Version: 8.6.0.12
GPU Type: 3080 Latop
Nvidia Driver Version: 470.82.01
CUDA Version: 11.4
CUDNN Version:
Operating System + Version:
Python Version (if applicable):
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):

Relevant Files

Please attach or include links to any models, data, files, or scripts necessary to reproduce your issue. (Github repo, Google Drive, Dropbox, etc.)

Steps To Reproduce

Please include:

  • Exact steps/commands to build your repro
  • Exact steps/commands to run your repro
  • Full traceback of errors encounteredH
2

Hi,

Request you to share the model, script, profiler, and performance output if not shared already so that we can help you better.

Alternatively, you can try running your model with trtexec command.

While measuring the model performance, make sure you consider the latency and throughput of the network inference, excluding the data pre and post-processing overhead.
Please refer to the below links for more details:

Thanks!