Performance comparison of TensorRT-optimized model between: (i) TF-TRT vs (ii) TensorRT C++ API?

AI & Data Science Deep Learning (Training & Inference) TensorRT
3

Hi All,

After starting to try TensorRT optimization and I personally found difficulties here and there, so, I decide to make a video tutorial here how we can optimize deep learning model obtained using Keras and Tensorflow. I also demonstrate to optimize YOLOv3. Hope it helps for those who begins trying to use TensorRT, and don’t encounter similar difficulties as I experienced before.

  1. Optimizing Tensorflow to TensorRT:
    01 Optimizing Tensorflow Model Using TensorRT with 3.7x Faster Inference Time - YouTube

  2. Visualizing model graph before and after TensorRT optimization:
    02 Visualizing Deep Learning Graph Before and After TensorRT Optimization - YouTube

  3. Optimizing Keras model to TensorRT:
    03 Optimizing Keras Model to TensorRT - YouTube

  4. Optimizing YOLOv3:
    06 Optimizing YOLO version 3 Model using TensorRT with 1.5x Faster Inference Time - YouTube

  5. YOLOv3 sample result, before and after TensorRT optimization:
    07 Another YOLOv3 Detection Result (Native Tensorflow vs TensorRT optimized) - YouTube

1 Like