# TensorRT for Cuda 12.2 **URL:** \ **Category:** TensorRT\ **Created:** [October 11, 2023, 5:17am UTC](https://forums.developer.nvidia.com/t/tensorrt-for-cuda-12-2/269087 "2023-10-11T05:17:27Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![Argo\_sa](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Argo\_sa](https://forums.developer.nvidia.com/u/Argo_sa)\ **Post date:** [October 11, 2023, 5:17am UTC](https://forums.developer.nvidia.com/t/tensorrt-for-cuda-12-2/269087/1 "2023-10-11T05:17:27Z")
## Description Hey everyone! I have a fresh install of ubuntu 22.04 with Cuda 12.2. I installed Cuda Toolkit and Cudnn. After that I was able to use GPU for pytorch model training. Now I need to install TensorRT and I can’t find a version for Cuda 12.2. Version for 12.1 doesn’t work. What should I do? ## Environment **TensorRT Version** : \* TensorRT 8.6 GA for Ubuntu 22.04 and CUDA 12.0 and 12.1 DEB local repo Package **GPU Type** : Nvidia 3060 (12gb) **Nvidia Driver Version** : 535.113.01 **CUDA Version** : CUDA Version: 12.2 **CUDNN Version** : 8.9.0.5 **Operating System + Version** : Ubuntu 22.04.3 LTS **Python Version (if applicable)**: Python 3.11.4 **TensorFlow Version (if applicable)**: **PyTorch Version (if applicable)**: 2.2.0.dev20231010 **Baremetal or Container (if container which image + tag)**: Baremetal ## Steps To Reproduce - Download \* [TensorRT 8.6 GA for Ubuntu 22.04 and CUDA 12.0 and 12.1 DEB local repo Package](https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/secure/8.6.1/local_repos/nv-tensorrt-local-repo-ubuntu2204-8.6.1-cuda-12.0_1.0-1_amd64.deb) - `sudo dpkg -i nv-tensorrt-local-repo-ubuntu2204-8.6.1-cuda-12.0_1.0-1_amd64.deb ` - `sudo cp /var/nv-tensorrt-local-repo-ubuntu2204-8.6.1-cuda-12.0/nv-tensorrt-local-42B2FC56-keyring.gpg /usr/share/keyrings/` - `sudo apt-get update` - `sudo apt-get install tensorrt` I get `E: Unable to locate package tensorrt` ---
**Author:** ![Argo\_sa](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Argo\_sa](https://forums.developer.nvidia.com/u/Argo_sa)\ **Post date:** [October 11, 2023, 9:44am UTC](https://forums.developer.nvidia.com/t/tensorrt-for-cuda-12-2/269087/2 "2023-10-11T09:44:00Z")
I reinstalled everything with Cuda 12.1. I still can train pytorch models on cuda, but I get exactly the same error while trying to install tensorrt. I took instructions from off web-site ---
**Author:** ![AakankshaS](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/aakankshas/32/14047_2.png) [@AakankshaS](https://forums.developer.nvidia.com/u/AakankshaS)\ **Post date:** [October 16, 2023, 11:51am UTC](https://forums.developer.nvidia.com/t/tensorrt-for-cuda-12-2/269087/4 "2023-10-16T11:51:55Z")
Hi @Argo_sa , Are you following the official document for Tensorrt Setup? > **[Installation Guide :: NVIDIA Deep Learning TensorRT Documentation](https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html#gettingstarted)** > > This NVIDIA TensorRT 8.6.1 Installation Guide provides the installation requirements, a list of what is included in the TensorRT package, and step-by-step instructions for installing TensorRT. Thanks ---
**Author:** ![mikechen66881](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@mikechen66881](https://forums.developer.nvidia.com/u/mikechen66881)\ **Post date:** [November 5, 2023, 7:03pm UTC](https://forums.developer.nvidia.com/t/tensorrt-for-cuda-12-2/269087/5 "2023-11-05T19:03:40Z")
Dear AakankshaS: What’s your direct suggestion on Tensorrt 8.6.1 for the latest RTX 4060 Ti 16G released on July 2023? While installing Driver for the GPU. it matches either v535.98(roll back to v535.86.10 after installing CUDA Toolkit) or v535.86.10 that is compatible with CUDA Version 12.2. If trying to install lower version driver v530.41.03 (released on March 2023) for RTX 4060 Ti 16G, nvidia-smi shows ERR! on GPU Name, i.e, that is no GPU Name of GeForce RTX 4060 Ti in the nvidia-smi interface. So what I can install is Linux Driver v535.86.10 and defaults CUDA Toolkit 12.2 update 1. It seems that the CUDA Toolkit is not compatible with the latest Tensorrt 8.6.1. What I can do to install Tensorrt? Appreciate you give a direct reply. Thanks in advance, Mike ---
**Author:** ![spolisetty](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/spolisetty/32/14047_2.png) [@spolisetty](https://forums.developer.nvidia.com/u/spolisetty)\ **Post date:** [November 22, 2023, 4:58pm UTC](https://forums.developer.nvidia.com/t/tensorrt-for-cuda-12-2/269087/6 "2023-11-22T16:58:43Z")
Hi, You can also try TensorRT NGC container to avoid setup related issues. > **[TensorRT | NVIDIA NGC](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/tensorrt)** > > NVIDIA TensorRT is a C++ library that facilitates high-performance inference on NVIDIA graphics processing units (GPUs). TensorRT takes a trained network and produces a highly optimized runtime engine that performs inference for that network. Thank you. ---
**Author:** ![Argo\_sa](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Argo\_sa](https://forums.developer.nvidia.com/u/Argo_sa)\ **Post date:** [December 3, 2023, 12:10pm UTC](https://forums.developer.nvidia.com/t/tensorrt-for-cuda-12-2/269087/7 "2023-12-03T12:10:32Z")
Exactly, this part: > **[Installation Guide :: NVIDIA Deep Learning TensorRT Documentation](https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html#installing-debian)** > > This NVIDIA TensorRT 8.6.1 Installation Guide provides the installation requirements, a list of what is included in the TensorRT package, and step-by-step instructions for installing TensorRT. ```auto sudo apt-get install tensorrt Reading package lists... Done Building dependency tree... Done Reading state information... Done E: Unable to locate package tensorrt ``` I am getting the same thing starting over again. I can see that for some reason your instructions do not lead `nv-tensorrt-local-repo-ubuntu2204-8.6.1-cuda-12.0` to get into `/etc/apt/sources.list.d/`. Only when I run command `echo "deb [arch=amd64] file:///var/nv-tensorrt-local-repo-ubuntu2204-8.6.1-cuda-12.0 /" | sudo tee /etc/apt/sources.list.d/tensorrt.list` it gets there. And After that I am heading a new problem: ```auto sudo apt-get install tensorrt Reading package lists... Done Building dependency tree... Done Reading state information... Done Some packages could not be installed. This may mean that you have requested an impossible situation or if you are using the unstable distribution that some required packages have not yet been created or been moved out of Incoming. The following information may help to resolve the situation: The following packages have unmet dependencies: libnvinfer-dev : Depends: libcudnn8-dev but it is not installable libnvinfer8 : Depends: libcudnn8 but it is not installable E: Unable to correct problems, you have held broken packages. ``` What is missing? ```auto cat /usr/local/cuda/include/cudnn_version.h | grep CUDNN_MAJOR -A 2 #define CUDNN_MAJOR 8 #define CUDNN_MINOR 9 #define CUDNN_PATCHLEVEL 5 -- #define CUDNN_VERSION (CUDNN_MAJOR * 1000 + CUDNN_MINOR * 100 + CUDNN_PATCHLEVEL) /* cannot use constexpr here since this is a C-only file */ ``` Seems like I have everything needed installed. ```auto Driver Version: 530.30.02 CUDA Version: 12.1 Python 3.11.4 CUDNN 8.9.5 Triton Inference Server 23.07 ``` ---
**Author:** ![Argo\_sa](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Argo\_sa](https://forums.developer.nvidia.com/u/Argo_sa)\ **Post date:** [December 3, 2023, 12:24pm UTC](https://forums.developer.nvidia.com/t/tensorrt-for-cuda-12-2/269087/8 "2023-12-03T12:24:17Z")
Some more details: ```auto nvcc --version nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2023 NVIDIA Corporation Built on Tue_Feb__7_19:32:13_PST_2023 Cuda compilation tools, release 12.1, V12.1.66 Build cuda_12.1.r12.1/compiler.32415258_0 ``` `dpkg -l | grep nvinfer` is empty And TensorRT Python API is installed: ```auto python Python 3.11.4 (main, Jul 5 2023, 13:45:01) [GCC 11.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import tensorrt >>> print(tensorrt. __version__ ) 8.6.1 >>> assert tensorrt.Builder(tensorrt.Logger()) ``` ---
**Author:** ![mikechen66881](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@mikechen66881](https://forums.developer.nvidia.com/u/mikechen66881)\ **Post date:** [April 9, 2024, 12:16am UTC](https://forums.developer.nvidia.com/t/tensorrt-for-cuda-12-2/269087/9 "2024-04-09T00:16:14Z")
Hi Guys: Nvidia has finally released TensorRT 10 EA (early Access) version. In spite of Nvdia’s delayed support for the compatibility between TensorRt and CUDA Toolkit(or cuDNN) for almost six months, the new release of TensorRT supports CUDA 12.2 to 12.4. It can solve the previous trouble-making/time-consuming issues. Users need to log in the following TensorRT webpage to download. > **[NVIDIA TensorRT](https://developer.nvidia.com/tensorrt)** > > An SDK with an optimizer for high-performance deep learning inference. Cheers. ---
**Author:** ![umeda](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@umeda](https://forums.developer.nvidia.com/u/umeda)\ **Post date:** [May 29, 2024, 7:01am UTC](https://forums.developer.nvidia.com/t/tensorrt-for-cuda-12-2/269087/10 "2024-05-29T07:01:17Z")
How about installing deb files related to tensorrt, ignoring the dependencies as shown below? sudo dpkg --force-depends -i /var/nv-tensorrt-local-repo-ubuntu2204-8.6.1-cuda-12.0/\*.deb ---
**Author:** ![prp852](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@prp852](https://forums.developer.nvidia.com/u/prp852)\ **Post date:** [July 24, 2024, 12:48am UTC](https://forums.developer.nvidia.com/t/tensorrt-for-cuda-12-2/269087/11 "2024-07-24T00:48:47Z")
Hi, I am using CUDA 12.2 but TensorRT 10.2 does not support (I get serialize error when converting and loading), any recs?