I obtained the same inference time with my optimized model (sometimes slower the baseline model) using the Tensorflow TensorRT API.
I’ve included a set of two tests on both SSD Resnet640x640 and EfficientDetD0.
TensorRT Version: 8.2.2
GPU Type: Tesla T4
Nvidia Driver Version: 450.51.05
CUDA Version: 11.6
CUDNN Version: 7.0.0
Python Version: 3.8
TensorFlow Version: 2.7.0
Container : nvcr.io/nvidia/tensorflow:22.01-tf2-py3 (build 31081301)
Models obtained from Zoo
Steps To Reproduce
Jupyter notebooks with results and steps to reproduce
We recommend you to check the below samples links in case of tf-trt integration issues.
If issue persist, We recommend you to reach out to Tensorflow forum.
Thanks for your reply, I’ve already seen the samples. The TF-TRT integration is working fine.
I’d like to know if TF-TRT doesn’t support optimization for those models, or what might be the root cause of the non inference speed-up compared to the example samples provided.
Thanks once again!
Sorry for the delayed response. Yes, it’s supported.
If the issue still persists, We recommend you to reach out to the Tensorflow related forum to get better help.