TensorRT 8.2 + CUDA 11.4 -> which CuDNN version?

carlosgalvezp · November 8, 2021, 9:03am

Hi,

According to this page, TensorRT 8.2 is compatible with CUDA 11.4.2. It also says it’s only compatible with CuDNN 8.2.1.

However, there’s no CuDNN 8.2.1 package ever built for CUDA 11.4.2. There is only the following:

libcudnn8_8.2.1.32-1+cuda10.2_amd64.deb
libcudnn8_8.2.1.32-1+cuda11.3_amd64.deb

So my question is: how are we supposed to use TensorRT 8.2 with CUDA 11.4, if there’s no compatible CuDNN version?

Thanks!

NVES · November 8, 2021, 9:38am

Hi ,
We recommend you to check the supported features from the below link.

You can refer below link for all the supported operators list.
For unsupported operators, you need to create a custom plugin to support the operation

github.com

onnx/onnx-tensorrt/blob/main/docs/operators.md

<!--- SPDX-License-Identifier: Apache-2.0 -->

# Supported ONNX Operators

TensorRT 8.4 supports operators up to Opset 17. Latest information of ONNX operators can be found [here](https://github.com/onnx/onnx/blob/master/docs/Operators.md)

TensorRT supports the following ONNX data types: DOUBLE, FLOAT32, FLOAT16, INT8, and BOOL

> Note: There is limited support for INT32, INT64, and DOUBLE types. TensorRT will attempt to cast down INT64 to INT32 and DOUBLE down to FLOAT, clamping values to `+-INT_MAX` or `+-FLT_MAX` if necessary.

See below for the support matrix of ONNX operators in ONNX-TensorRT.

## Operator Support Matrix

| Operator                  | Supported  | Supported Types | Restrictions                                                                                                           |
|---------------------------|------------|-----------------|------------------------------------------------------------------------------------------------------------------------|
| Abs                       | Y          | FP32, FP16, INT32 |
| Acos                      | Y          | FP32, FP16 |
| Acosh                     | Y          | FP32, FP16 |
| Add                       | Y          | FP32, FP16, INT32 |

This file has been truncated. show original

Thanks!

carlosgalvezp · November 8, 2021, 10:00am

@NVES , if you are a bot, you really need some improvements :) I clearly linked to the webpage that you asked me to consult, so your answer is (once again) not useful at all.

carlosgalvezp · November 9, 2021, 7:57am

@ework This question is very tightly related to my previous one. In this case however I believe Nvidia should provide a way for users to use a “tested” configuration. How has Nvidia tested it if there are no packages available?

Looking forward to your thoughts.

ework · November 9, 2021, 8:37am

You’re correct cuDNN 8.2.1 does not have a CUDA 11.4 build available. Although the CUDA 11.3 build of cuDNN works fine with CUDA 11.4 applications and that is the version which has been used to test TensorRT 8.2.0. CUDA and deep learning libraries are moving towards supporting CUDA enhanced compatibility, which will allow builds from earlier CUDA toolkits to work with newer CUDA toolkits [1]. We are very close to having this fully supported by TensorRT and cuDNN.

[1] CUDA Compatibility :: NVIDIA Data Center GPU Driver Documentation

carlosgalvezp · November 10, 2021, 8:13am

Thanks for the clarification. As a user, it’s confusing to have a mix of libraries - we usually strive to have consistency and keep everything nice and tidy. In other words, use CUDA 11.4 for everything is what makes sense, not one library with 11.4 and another with 11.3.

Regarding compatibility, the link that you point to talks about “drivers”, not libraries, or? CUDA 11.5 requires driver 495.x, however it supports older drivers. I don’t find it stated that the CUDA libraries (not drivers) are compatible with one another.

system · November 24, 2021, 8:13am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.