Inference time using TF-TRT is the same as Native Tensorflow for Object Detection Models

Description

I obtained the same inference time with my optimized model (sometimes slower the baseline model) using the Tensorflow TensorRT API.
I’ve included a set of two tests on both SSD Resnet640x640 and EfficientDetD0.

Environment

TensorRT Version: 8.2.2
GPU Type: Tesla T4
Nvidia Driver Version: 450.51.05
CUDA Version: 11.6
CUDNN Version: 7.0.0
Python Version: 3.8
TensorFlow Version: 2.7.0
Container : nvcr.io/nvidia/tensorflow:22.01-tf2-py3 (build 31081301)

Relevant Files

Models obtained from Zoo

Steps To Reproduce

Jupyter notebooks with results and steps to reproduce

Hi,
We recommend you to check the below samples links in case of tf-trt integration issues.

If issue persist, We recommend you to reach out to Tensorflow forum.
Thanks!

Hello!
Thanks for your reply, I’ve already seen the samples. The TF-TRT integration is working fine.
I’d like to know if TF-TRT doesn’t support optimization for those models, or what might be the root cause of the non inference speed-up compared to the example samples provided.
Thanks once again!

Any updates?

Hi,

Sorry for the delayed response. Yes, it’s supported.
If the issue still persists, We recommend you to reach out to the Tensorflow related forum to get better help.

Thank you.