Convert fp16 killed

804884774 · June 7, 2022, 2:43am

Description

convert fp16 killed，please help

Environment

TensorRT Version: nvcr.io/nvidia/tensorrt:21.12-py3
GPU Type: V100
Nvidia Driver Version: 460.32.03
CUDA Version:
CUDNN Version:
Operating System + Version: Ubuntu 20.04.3 LTS
Python Version (if applicable):
TensorFlow Version (if applicable):
PyTorch Version (if applicable): 1.10.1
Baremetal or Container (if container which image + tag): nvcr.io/nvidia/tensorrt:21.12-py3

Relevant Files

debug.zip (83.4 MB)

Steps To Reproduce

env: docker-img

nvcr.io/nvidia/tensorrt:21.12-py3

unzip debug.zip && cd debug

test command1：

python onnx2trt_test.py # run successfully

test command2：

python onnx2trt_test.py --fp16_mode # run killed

NVES · June 7, 2022, 3:07am

Hi,
Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:

validating your model with the below snippet

check_model.py

import sys
import onnx
filename = yourONNXmodel
model = onnx.load(filename)
onnx.checker.check_model(model).
2) Try running your model with trtexec command.

In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging
Thanks!

804884774 · June 7, 2022, 4:35am

onnx model and script have been uploaded in debug,zip

1、check_model：

import sys
import onnx
filename = 'debug/weights/btr.onnx'
model = onnx.load(filename)
onnx.checker.check_model(model)

# success

2、trtexec info

trtexec --onnx='debug/weights/btr.onnx'  –-verbose

&&&& PASSED 


trtexec --onnx='debug/weights/btr.onnx'  --fp16 –-verbose

***********
[06/07/2022-04:12:17] [V] [TRT] --------------- Timing Runner: Div_298 (ElementWise)
[06/07/2022-04:12:17] [V] [TRT] Tactic: 1 Time: 0.006272
[06/07/2022-04:12:17] [V] [TRT] Fastest Tactic: 1 Time: 0.006272
[06/07/2022-04:12:17] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: ElementWise Tactic: 1
[06/07/2022-04:12:17] [V] [TRT] *************** Autotuning format combination: Float(64,1:32,1,1), Float(64,1:32,1,1) -> Float(64,1:32,1,1) ***************
[06/07/2022-04:12:17] [V] [TRT] --------------- Timing Runner: Div_298 (ElementWise)
[06/07/2022-04:12:17] [V] [TRT] Tactic: 1 Time: 0.006144
[06/07/2022-04:12:17] [V] [TRT] Fastest Tactic: 1 Time: 0.006144
[06/07/2022-04:12:17] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: ElementWise Tactic: 1
[06/07/2022-04:12:17] [V] [TRT] *************** Autotuning format combination: Half(2048,1,1,1), Half(2048,1,1,1) -> Half(2048,1,1,1) ***************
[06/07/2022-04:12:17] [V] [TRT] --------------- Timing Runner: Div_298 (ElementWise)
[06/07/2022-04:12:17] [V] [TRT] Tactic: 1 Time: 0.007296
[06/07/2022-04:12:17] [V] [TRT] Fastest Tactic: 1 Time: 0.007296
[06/07/2022-04:12:17] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: ElementWise Tactic: 1
[06/07/2022-04:12:17] [V] [TRT] *************** Autotuning format combination: Half(1024,1:2,1,1), Half(1024,1:2,1,1) -> Half(1024,1:2,1,1) ***************
[06/07/2022-04:12:17] [V] [TRT] --------------- Timing Runner: Div_298 (ElementWise)
[06/07/2022-04:12:17] [V] [TRT] Tactic: 1 Time: 0.007296
[06/07/2022-04:12:17] [V] [TRT] Fastest Tactic: 1 Time: 0.007296
[06/07/2022-04:12:17] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: ElementWise Tactic: 1
[06/07/2022-04:12:17] [V] [TRT] *************** Autotuning format combination: Half(256,1:8,256,256), Half(256,1:8,256,256) -> Half(256,1:8,256,256) ***************
[06/07/2022-04:12:17] [V] [TRT] --------------- Timing Runner: Div_298 (ElementWise)
[06/07/2022-04:12:17] [V] [TRT] Tactic: 1 Time: 0.007296
[06/07/2022-04:12:17] [V] [TRT] Fastest Tactic: 1 Time: 0.007296
[06/07/2022-04:12:17] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: ElementWise Tactic: 1
Killed

spolisetty · June 7, 2022, 4:43pm

Hi,

Could you please try on the latest TensorRT version and let us know if you still face this issue.

Thank you.

804884774 · June 8, 2022, 10:27am

Hello,
I try the newest container：

nvcr.io/nvidia/tensorrt:22.05-py3

the result is same，below is the output

***********
[06/08/2022-17:50:57] [V] [TRT] --------------- Timing Runner: Div_298 (ElementWise)
[06/08/2022-17:50:57] [V] [TRT] Tactic: 1 Time: 0.006396
[06/08/2022-17:50:57] [V] [TRT] Fastest Tactic: 1 Time: 0.006396
[06/08/2022-17:50:57] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: ElementWise Tactic: 1
[06/08/2022-17:50:57] [V] [TRT] *************** Autotuning format combination: Float(64,1:32,1,1), Float(64,1:32,1,1) -> Float(64,1:32,1,1) ***************
[06/08/2022-17:50:57] [V] [TRT] --------------- Timing Runner: Div_298 (ElementWise)
[06/08/2022-17:50:57] [V] [TRT] Tactic: 1 Time: 0.006272
[06/08/2022-17:50:57] [V] [TRT] Fastest Tactic: 1 Time: 0.006272
[06/08/2022-17:50:57] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: ElementWise Tactic: 1
[06/08/2022-17:50:57] [V] [TRT] *************** Autotuning format combination: Half(2048,1,1,1), Half(2048,1,1,1) -> Half(2048,1,1,1) ***************
[06/08/2022-17:50:57] [V] [TRT] --------------- Timing Runner: Div_298 (ElementWise)
[06/08/2022-17:50:57] [V] [TRT] Tactic: 1 Time: 0.007168
[06/08/2022-17:50:57] [V] [TRT] Fastest Tactic: 1 Time: 0.007168
[06/08/2022-17:50:57] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: ElementWise Tactic: 1
[06/08/2022-17:50:57] [V] [TRT] *************** Autotuning format combination: Half(1024,1:2,1,1), Half(1024,1:2,1,1) -> Half(1024,1:2,1,1) ***************
[06/08/2022-17:50:57] [V] [TRT] --------------- Timing Runner: Div_298 (ElementWise)
[06/08/2022-17:50:57] [V] [TRT] Tactic: 1 Time: 0.007168
[06/08/2022-17:50:57] [V] [TRT] Fastest Tactic: 1 Time: 0.007168
[06/08/2022-17:50:57] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: ElementWise Tactic: 1
[06/08/2022-17:50:57] [V] [TRT] *************** Autotuning format combination: Half(256,1:8,256,256), Half(256,1:8,256,256) -> Half(256,1:8,256,256) ***************
[06/08/2022-17:50:57] [V] [TRT] --------------- Timing Runner: Div_298 (ElementWise)
[06/08/2022-17:50:57] [V] [TRT] Tactic: 1 Time: 0.007168
[06/08/2022-17:50:57] [V] [TRT] Fastest Tactic: 1 Time: 0.007168
[06/08/2022-17:50:57] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: ElementWise Tactic: 1
Killed

it runs a long time and killed，please help

spolisetty · June 13, 2022, 11:57am

Hi,

We could successfully build the TRT engine on TensorRT 8.4 EA version and could not reproduce the issue. We recommend you to please install TensorRT 8.4 EA locally or on the NGC container.

&&&& PASSED TensorRT.trtexec [TensorRT v8400] # /usr/src/tensorrt/bin/trtexec --onnx=weights/btr.onnx --fp16 --verbose --workspace=10000

Thank you.

804884774 · June 15, 2022, 5:45am

Hi,
Thank you very much！ It is useful！

system · June 29, 2022, 5:46am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.