Efficientdet conversion from onnx to tensorrt engine freeze

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

Running example from this Github

Freezing on the same spot in the last shown log. I am able to make it work using a workstation that using a quadro gpu.

Environment

TensorRT Version: 8202
GPU Type: RTX 3070
Nvidia Driver Version: 511.65
CUDA Version: 11.6
CUDNN Version: 8.3.2
Operating System + Version: ubuntu20.04 (windows 11 wsl)
Python Version (if applicable): 3.8
TensorFlow Version (if applicable): 2.7.0
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag): nvcr.io/nvidia/tensorflow:22.01-tf2-py3

Relevant Files

Log:
freeze.log (4.3 MB)

Onnx :
model.onnx (16.1 MB)

Steps To Reproduce

#!/bin/bash
INPUT_SHAPE=8x512x512x3
trtexec
–onnx=model/onnx-model/model.onnx
–saveEngine=model/trt-model/engine.trt
–optShapes=input:$INPUT_SHAPE
–workspace=8000
–verbose

2

Hi,
Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:

  1. validating your model with the below snippet

check_model.py

import sys
import onnx
filename = yourONNXmodel
model = onnx.load(filename)
onnx.checker.check_model(model).
2) Try running your model with trtexec command.

In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging
Thanks!

4

Hi,

We recommend you to please use the latest TensorRT version 8.4.
Using the latest TRT version, we could successfully generate the TRT engine.
https://developer.nvidia.com/nvidia-tensorrt-8x-download

Thank you.

5

Hi,

Ive tried the solution by using version 8.4, it still freeze. Tested by using windows zip installation on windows 11

update: after clear and kill some background process that using gpu resources, manage to make it work.

update2: another go with wsl, checkout latest, make using dockerfile provide by the repo instead of pulling from nvidia catalog, still freeze as the same spot as provided log

6

Hi,

We couldn’t reproduce the same error. We could successfully build the TRT engine.
Have you tried increasing workspace?
Could you please share with us Dockerfile to try from our end.

Thank you.

7

as mention, ive been using a container provide by nvidia but they didnt share the dockerfile, u need to pull this http://nvcr.io/nvidia/tensorflow:22.01-tf2-py3

but ive build another container suggest by official tensor rt repo and get the same freeze result, here is the shared dockerfile TensorRT/ubuntu-20.04.Dockerfile at main · NVIDIA/TensorRT · GitHub