Performance using the integration TensorFlow-TensorRT vs direct TensorRT

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

hi all,

Are there some benchmarks that show the performance difference using TensorRT with integration TensorFlow-TensorRT versus direct TensorRT?

2

Hello,

We don’t currently have benchmarks comparing TRT vs. TFTRT.

3

Hi NVES, I found this plot showing TRT vs. TF-TRT. Is there some technical justification on why TF-TRT performs almost half of TRT’s throughput as shown in this plot?

846×576 71.5 KB

4

That’s based on very old TensorRT (version 3).

We are working on an updated TRT vs TFTRT comparison. I cannot share more information about future release here. Stay tuned.

5

Hi, I wonder @sanchezvr7 where did you find this plot image?. @nves, do you update any comparison results of TRT and TFTRT recently?

6

hi @masaf494d, I found that plot at https://devblogs.nvidia.com/tensorrt-integration-speeds-tensorflow-inference/. I haven’t be able to get good comparison between TRT and TFTRT, see the issue here Unexpected TensorRT5.1.2 Results vs TRTIS1.0.0 Results · Issue #239 · triton-inference-server/server · GitHub. Please keep me posted with your progress

1 Like
7

Since almost a year has passed, does NVIDIA has any update on the TFTRT vs TRT benchmark yet?