Inference time using TF-TRT is the same as Native Tensorflow for Object Detection Models

AI & Data Science Deep Learning (Training & Inference) TensorRT
,
1

Description

I obtained the same inference time with my optimized model (sometimes slower the baseline model) using the Tensorflow TensorRT API.
I’ve included a set of two tests on both SSD Resnet640x640 and EfficientDetD0.

Environment

TensorRT Version: 8.2.2
GPU Type: Tesla T4
Nvidia Driver Version: 450.51.05
CUDA Version: 11.6
CUDNN Version: 7.0.0
Python Version: 3.8
TensorFlow Version: 2.7.0
Container : nvcr.io/nvidia/tensorflow:22.01-tf2-py3 (build 31081301)

Relevant Files

Models obtained from Zoo

Steps To Reproduce

Jupyter notebooks with results and steps to reproduce

2

Hi,
We recommend you to check the below samples links in case of tf-trt integration issues.

If issue persist, We recommend you to reach out to Tensorflow forum.
Thanks!

5

Hello!
Thanks for your reply, I’ve already seen the samples. The TF-TRT integration is working fine.
I’d like to know if TF-TRT doesn’t support optimization for those models, or what might be the root cause of the non inference speed-up compared to the example samples provided.
Thanks once again!

6

Any updates?

7

Hi,

Sorry for the delayed response. Yes, it’s supported.
If the issue still persists, We recommend you to reach out to the Tensorflow related forum to get better help.

Thank you.