Hi,
We give it a try with the ResNet50.onnx model but were not able to reproduce this issue.
Xavier
$ sudo nvpmodel -q
NV Fan Mode:quiet
NV Power Mode: MODE_15W
2
$ /usr/src/tensorrt/bin/trtexec --onnx=/usr/src/tensorrt/data/resnet50/ResNet50.onnx
...
[04/28/2022-11:16:50] [I] === Performance summary ===
[04/28/2022-11:16:50] [I] Throughput: 63.1476 qps
[04/28/2022-11:16:50] [I] Latency: min = 15.7756 ms, max = 15.9119 ms, mean = 15.8206 ms, median = 15.8167 ms, percentile(99%) = 15.9011 ms
[04/28/2022-11:16:50] [I] End-to-End Host Latency: min = 15.7852 ms, max = 15.9302 ms, mean = 15.8358 ms, median = 15.8317 ms, percentile(99%) = 15.9097 ms
[04/28/2022-11:16:50] [I] Enqueue Time: min = 1.13879 ms, max = 1.62451 ms, mean = 1.2817 ms, median = 1.27686 ms, percentile(99%) = 1.42432 ms
[04/28/2022-11:16:50] [I] H2D Latency: min = 0.0385742 ms, max = 0.0421753 ms, mean = 0.0398231 ms, median = 0.0395508 ms, percentile(99%) = 0.0420532 ms
[04/28/2022-11:16:50] [I] GPU Compute Time: min = 15.7318 ms, max = 15.8689 ms, mean = 15.7789 ms, median = 15.7756 ms, percentile(99%) = 15.8577 ms
[04/28/2022-11:16:50] [I] D2H Latency: min = 0.00146484 ms, max = 0.00244141 ms, mean = 0.00182954 ms, median = 0.00195312 ms, percentile(99%) = 0.00244141 ms
[04/28/2022-11:16:50] [I] Total Host Walltime: 3.02466 s
[04/28/2022-11:16:50] [I] Total GPU Compute Time: 3.01378 s
[04/28/2022-11:16:50] [I] Explanations of the performance metrics are printed in the verbose logs.
[04/28/2022-11:16:50] [I]
&&&& PASSED TensorRT.trtexec [TensorRT v8201] # /usr/src/tensorrt/bin/trtexec --onnx=/usr/src/tensorrt/data/resnet50/ResNet50.onnx
Orin
$ sudo nvpmodel -q
NV Power Mode: MODE_15W
1
$ /usr/src/tensorrt/bin/trtexec --onnx=/usr/src/tensorrt/data/resnet50/ResNet50.onnx
...
[04/28/2022-03:15:21] [I] === Performance summary ===
[04/28/2022-03:15:21] [I] Throughput: 89.0295 qps
[04/28/2022-03:15:21] [I] Latency: min = 11.1973 ms, max = 11.301 ms, mean = 11.2604 ms, median = 11.2612 ms, percentile(99%) = 11.2981 ms
[04/28/2022-03:15:21] [I] Enqueue Time: min = 0.457764 ms, max = 0.522461 ms, mean = 0.471977 ms, median = 0.468628 ms, percentile(99%) = 0.50415 ms
[04/28/2022-03:15:21] [I] H2D Latency: min = 0.0643311 ms, max = 0.0765381 ms, mean = 0.0664937 ms, median = 0.065918 ms, percentile(99%) = 0.0721436 ms
[04/28/2022-03:15:21] [I] GPU Compute Time: min = 11.1248 ms, max = 11.2279 ms, mean = 11.1882 ms, median = 11.1887 ms, percentile(99%) = 11.2277 ms
[04/28/2022-03:15:21] [I] D2H Latency: min = 0.00408936 ms, max = 0.00708008 ms, mean = 0.00569752 ms, median = 0.00579834 ms, percentile(99%) = 0.00701904 ms
[04/28/2022-03:15:21] [I] Total Host Walltime: 3.0327 s
[04/28/2022-03:15:21] [I] Total GPU Compute Time: 3.02082 s
[04/28/2022-03:15:21] [I] Explanations of the performance metrics are printed in the verbose logs.
[04/28/2022-03:15:21] [I]
&&&& PASSED TensorRT.trtexec [TensorRT v8400] # /usr/src/tensorrt/bin/trtexec --onnx=/usr/src/tensorrt/data/resnet50/ResNet50.onnx
Thanks.