Hi,
I am trying to execute trtexec with the following parameters:
/trtexec --onnx=/ --int8 --batch=16 --iterations=100 --duration=120 --warmUp=1000 --avgRuns
I see the final summary as follows:
[03/12/2020-08:06:42] [I] Host latency
[03/12/2020-08:06:42] [I] min: 1.87109 ms (end to end 1.90234 ms)
[03/12/2020-08:06:42] [I] max: 2.09766 ms (end to end 2.25781 ms)
[03/12/2020-08:06:42] [I] mean: 1.94362 ms (end to end 2.05063 ms)
[03/12/2020-08:06:42] [I] median: 1.94531 ms (end to end 2.06641 ms)
[03/12/2020-08:06:42] [I] percentile: 2 ms at 99% (end to end 2.14062 ms at 99%)
I want to understand if the ‘end to end’ time (2.14062) is per batch (16 frames in this case) or per frame.