Failed building TensorRT plan file from onnx using TensorRT8.0.1.6 with Ampere GPU

AI & Data Science Deep Learning (Training & Inference) TensorRT
,
1

Description

Failed to build TensorRT plan file on Ampere GPU (test on both RTX 6000 and RTX 3090) using FP16 mode with TensorRT 8.0.1.6.

Environment

TensorRT Version: 8.0.1.6
NVIDIA GPU: RTX A6000 and RTX 3090
NVIDIA Driver Version: 460.84
CUDA Version: 11.0
CUDNN Version: 8.0.4
Operating System: ubuntu 18.04
Python Version (if applicable): 3.6.9
Tensorflow Version (if applicable): None
PyTorch Version (if applicable): None
Baremetal or Container (if so, version): docker version: 20.10.6, image built using official docker file.

Relevant Files

onnx model: model.onnx - Google Drive

Steps To Reproduce

run with the following command using trtexec

trtexec --onnx=model.onnx --workspace=4096 --device=0 --saveEngine=model.engine --fp16

logs: click TensorRT logs to see logs

however, if I omit the --fp16 option, the onnx file can be built successfully, but the TensorRT library still print these WARNINGs:

.
.
.
[W] [TRT] Skipping tactic 0 due to kernel generation error.
nvrtc: error: invalid value for --gpu-architecture (-arch)

[W] [TRT] Skipping tactic 1 due to kernel generation error.
nvrtc: error: invalid value for --gpu-architecture (-arch)

[W] [TRT] Skipping tactic 2 due to kernel generation error.
nvrtc: error: invalid value for --gpu-architecture (-arch)

[W] [TRT] Skipping tactic 3 due to kernel generation error.
nvrtc: error: invalid value for --gpu-architecture (-arch)

[W] [TRT] Skipping tactic 4 due to kernel generation error.
nvrtc: error: invalid value for --gpu-architecture (-arch)
.
.
.
2

Hi,
Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:

  1. validating your model with the below snippet

check_model.py

import sys
import onnx
filename = yourONNXmodel
model = onnx.load(filename)
onnx.checker.check_model(model).
2) Try running your model with trtexec command.
https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/trtexec
In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging
Thanks!

4

@yuanfei481,

For RTX A6000 and RTX 3090, CUDA-11.1 is a minimal requirement, looks like you’re using CUDA-11.0. We recommend you to please install CUDA 11.3.

Thank you.

5

Is tha so? I checked the CMakeLists.txt cloned from the TensorRT repository ,which shows the the minimum requirement for cuda is >= 11.0, howerver, I’ll give it a try.

image
image1189Ă—736 56.7 KB

6

update cuda from 11.0 to 11.3 sovle the problem.
Tanks!

1 Like