Failed building TensorRT plan file from onnx using TensorRT8.0.1.6 with Ampere GPU

Description

Failed to build TensorRT plan file on Ampere GPU (test on both RTX 6000 and RTX 3090) using FP16 mode with TensorRT 8.0.1.6.

Environment

TensorRT Version: 8.0.1.6
NVIDIA GPU: RTX A6000 and RTX 3090
NVIDIA Driver Version: 460.84
CUDA Version: 11.0
CUDNN Version: 8.0.4
Operating System: ubuntu 18.04
Python Version (if applicable): 3.6.9
Tensorflow Version (if applicable): None
PyTorch Version (if applicable): None
Baremetal or Container (if so, version): docker version: 20.10.6, image built using official docker file.

Relevant Files

onnx model: model.onnx - Google Drive

Steps To Reproduce

run with the following command using trtexec

trtexec --onnx=model.onnx --workspace=4096 --device=0 --saveEngine=model.engine --fp16

logs: click TensorRT logs to see logs

however, if I omit the --fp16 option, the onnx file can be built successfully, but the TensorRT library still print these WARNINGs:

.
.
.
[W] [TRT] Skipping tactic 0 due to kernel generation error.
nvrtc: error: invalid value for --gpu-architecture (-arch)

[W] [TRT] Skipping tactic 1 due to kernel generation error.
nvrtc: error: invalid value for --gpu-architecture (-arch)

[W] [TRT] Skipping tactic 2 due to kernel generation error.
nvrtc: error: invalid value for --gpu-architecture (-arch)

[W] [TRT] Skipping tactic 3 due to kernel generation error.
nvrtc: error: invalid value for --gpu-architecture (-arch)

[W] [TRT] Skipping tactic 4 due to kernel generation error.
nvrtc: error: invalid value for --gpu-architecture (-arch)
.
.
.

Hi,
Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:
https://docs.nvidia.com/deeplearning/tensorrt/quick-start-guide/index.html#onnx-export

  1. validating your model with the below snippet

check_model.py

import sys
import onnx
filename = yourONNXmodel
model = onnx.load(filename)
onnx.checker.check_model(model).
2) Try running your model with trtexec command.

In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging
Thanks!

@yuanfei481,

For RTX A6000 and RTX 3090, CUDA-11.1 is a minimal requirement, looks like you’re using CUDA-11.0. We recommend you to please install CUDA 11.3.

Thank you.

Is tha so? I checked the CMakeLists.txt cloned from the TensorRT repository ,which shows the the minimum requirement for cuda is >= 11.0, howerver, I’ll give it a try.

update cuda from 11.0 to 11.3 sovle the problem.
Tanks!

1 Like