optimization for object detection: no improvement in inference speed

gabriel0xFF · February 28, 2019, 12:00pm

Running the notebook with

MODEL = 'ssd_mobilenet_v2_coco'

and evaluating with “FP32” and “FP16”:
in both cases, 'Average runtime" will be at aroung 14-15ms, not showing any significant improvements.

Running provided samples for TensorRT optimization for image classification works just fine, improving inderence time by ~3-4 times.

Does TensorRT have some issues with object detection that are not apparent in plain image classification?

Topic		Replies	Views
Looking for Insight on Disappointing Results Optimizing an Object Detection Network with TensorRT TensorRT	1	893	March 29, 2019
TensorRT Optimization for Tensorflow-Unet-Image-segmentation TensorRT tensorrt , tensorflow , nano	1	1178	August 4, 2021
Inference time using TF-TRT is the same as Native Tensorflow for Object Detection Models TensorRT tensorrt , tf-trt	4	1014	March 31, 2022
Techniques to Imporve TensorRT Model Inference Speed TensorRT tensorrt , cudnn	1	35	June 2, 2025
Deploying Deep Neural Networks with NVIDIA TensorRT Technical Blog	17	662	October 8, 2017
Inference time speed up? TensorRT	1	469	August 3, 2021
How can I improve my prediction performance in TenserRt 3.0? TensorRT	3	933	April 26, 2018
Optimize Tensorflow with Tensor RT to improve inference timing Jetson Nano	2	639	October 18, 2021
FP16 does not improve latency of ssd_mobilenet_v2_coco TensorRT	0	498	August 9, 2019
No performance improvement for Tensorflow TensorRT model on converted on Jetsons Xavier NX Jetson Xavier NX tensorrt , tensorflow	2	681	October 18, 2021