Seeking Guidance on Proper Installation of TensorRT

Hello everyone,

I hope you’re all doing well. I’m currently in the process of installing TensorRT and would greatly appreciate your insights on a matter.

My question is whether downgrading some of the packages might be necessary to ensure a proper installation of TensorRT. I’ve been facing a bit of uncertainty regarding the correctness of my TensorRT installation. While I haven’t encountered the TensorRT-related warning in TensorFlow, I did encounter a series of challenges to get TensorRT up and running.

Presently, the latest PyTorch version (2.0.1) only functions with CUDA 11.8. Consequently, I had to carry out downgrades for CUDA, cuDNN, and TensorRT. I must mention that these packages were added to the Ubuntu repository. However, I received prompts to upgrade them, each time I sudo apt update && sudo apt upgrade, which I temporarily mark them by placing them “on hold” to avoid any potential conflicts.

If any of you could share your expertise and advice in this matter, it would be immensely helpful. Your experience and insights would contribute greatly to resolving my situation.

When I type:
$ dpkg -l | grep nvinfer
I get:

ii  libnvinfer-dispatch-dev                                       amd64        TensorRT development dispatch runtime libraries
ii  libnvinfer-dispatch8                                          amd64        TensorRT dispatch runtime library
ii  libnvinfer-headers-dev                                        amd64        TensorRT development headers
ii  libnvinfer-headers-plugin-dev                                 amd64        TensorRT plugin headers
ii  libnvinfer-lean-dev                                           amd64        TensorRT lean runtime libraries
ii  libnvinfer-lean8                                              amd64        TensorRT lean runtime library
ii  libnvinfer-plugin8                                            amd64        TensorRT plugin libraries
ii  libnvinfer-vc-plugin-dev                                      amd64        TensorRT vc-plugin library
ii  libnvinfer-vc-plugin8                                         amd64        TensorRT vc-plugin library
ii  libnvinfer8                                                   amd64        TensorRT runtime libraries

As you can see there are still some packages with cuda12.0 version! Should I downgrade each one of them as well?! How can I know I have a proper Installation of TensorRT anyway? I haven’t seen much of difference in performance, yet!


TensorRT Version: 8.6.1
GPU Type: 4060
Nvidia Driver Version:
CUDA Version: 535.86.05
CUDNN Version: 8.9.0
Operating System + Version: Ubuntu 22.04
Python Version (if applicable): 3.10
TensorFlow Version (if applicable): 2.13
PyTorch Version (if applicable): 2.0.1

Thank you in advance for your time and assistance.

Please refer to the installation steps from the below link if in case you are missing on anything

Also, we suggest you to use TRT NGC containers to avoid any system dependency related issues.