Trtexec Throughput in QPS for ResNet50

AI & Data Science Deep Learning (Training & Inference) Triton Inference Server (archived)

nicks165 July 29, 2020, 10:45pm 1

I am running ResNet50 onnx model using trtexec on Nvidia T4. They print Throughput and latency. I am confused why the throughput in QPS instead of Images/Sec. And also the numbers are extremely bad compared to MLPerf.

1 Like

Topic		Replies	Views
Inference Speed Jetson Xavier NX pytorch	5	1140	March 30, 2023
Trtexec performance not close to benchmarks Jetson Orin NX tensorrt	1	583	December 19, 2023
Orin nano/nx ResNet-50 benchmark on R36.4.3(jetpack6.2) Jetson Orin Nano benchmarks	7	732	March 10, 2025
Same inference speed with Resnet50 for int8 and fp16 Jetson Xavier NX jetson-inference	3	828	November 30, 2020
An FCN with 360K parameters much slower compared to ResNet50 with 23M parameters TensorRT	3	481	April 21, 2020
TensorRT has less batching throughput improvement than PyTorch on Jetson Nano TensorRT tensorrt , jetson-inference	2	535	September 22, 2020
trtexec set input shape not working with TensorRT	2	5812	August 5, 2021
Onnx -> TensorRT. No speed difference between models of different sizes Jetson AGX Xavier tensorrt , onnx	5	974	July 12, 2021
Unexpected TensorRT5.1.2 Results vs TRTIS1.0.0 Results TensorRT	0	787	April 19, 2019
Why Orin's trtexec produces much better inference from onnx model than from engine model? Jetson Orin NX jetson-inference	6	731	August 31, 2022

Trtexec Throughput in QPS for ResNet50

Related topics