Hello,
I have a problem with benchmark XavierNX yolo model from this repository
https://github.com/NVIDIA-AI-IOT/jetson_benchmarks
Model for tests was downloaded from benchmark csv https://www.dropbox.com/s/ck9e40b57rd5o14/yolov3-tiny-416.zip
yolov3-tiny-416 model show in table 546.69 FPS which is great, but when I start tool by myself its show 0 FPS with error Error opening engine file: ./models/yolov3-tiny-416_b8_ws2048_gpu.engine
So I converted model from onnx to engine via command myself ( convertion command was extracted from benchmark script )
/usr/src/tensorrt/bin/trtexec --onnx=yolov3-tiny-416-bs8.onnx --explicitBatch --inputIOFormats=int8:chw+chw4+chw32 --int8 --saveEngine=yolov3-tiny-416_b8_ws2048_gpu.engine
And then start test via
/usr/src/tensorrt/bin/trtexec --onnx=yolov3-tiny-416-bs8.onnx --explicitBatch --inputIOFormats=int8:chw+chw4+chw32 --int8 --workspace=2048 --avgRuns=10 --duration=180 --loadEngine=yolov3-tiny-416_b8_ws2048_gpu.engine --useSpinWait
Results
[02/07/2023-14:40:06] [I] === Performance summary ===
[02/07/2023-14:40:06] [I] Throughput: 73.9664 qps
[02/07/2023-14:40:06] [I] Latency: min = 11.0469 ms, max = 114.109 ms, mean = 13.9611 ms, median = 11.2422 ms, percentile(90%) = 11.5312 ms, percentile(95%) = 22.8438 ms, percentile(99%) = 86.875 ms
[02/07/2023-14:40:06] [I] Enqueue Time: min = 0.320312 ms, max = 108.086 ms, mean = 3.67974 ms, median = 0.476562 ms, percentile(90%) = 1.57812 ms, percentile(95%) = 18.7031 ms, percentile(99%) = 80.4922 ms
[02/07/2023-14:40:06] [I] H2D Latency: min = 0.226624 ms, max = 11 ms, mean = 0.456758 ms, median = 0.304688 ms, percentile(90%) = 0.421875 ms, percentile(95%) = 0.882812 ms, percentile(99%) = 4.23438 ms
[02/07/2023-14:40:06] [I] GPU Compute Time: min = 10.4297 ms, max = 108.539 ms, mean = 13.068 ms, median = 10.5859 ms, percentile(90%) = 10.75 ms, percentile(95%) = 19.6875 ms, percentile(99%) = 81.4336 ms
[02/07/2023-14:40:06] [I] D2H Latency: min = 0.265625 ms, max = 11.8672 ms, mean = 0.436255 ms, median = 0.34668 ms, percentile(90%) = 0.375 ms, percentile(95%) = 0.395508 ms, percentile(99%) = 3 ms
[02/07/2023-14:40:06] [I] Total Host Walltime: 180.041 s
[02/07/2023-14:40:06] [I] Total GPU Compute Time: 174.027 s
[02/07/2023-14:40:06] [W] * GPU compute time is unstable, with coefficient of variance = 89.5551%.
Which is 45~100 FPS, but nothing near 500+ is there something wrong?
P.S. was enabled max power mode 20W 6CORE + jetson_clocks enabled