Latency for VGG16 inference

I am interested in the contents of the link below.

Time and Avg.Time in the answer. What does time mean?
And why does FC Layer’s time take place for a long time?
If there is a code that I can test myself, can you share it with us?
I’m using the Jetsontx2 board.



We share the data since the user doesn’t have a Jeston board.
Since you have a TX2 device already, you can measure it directly.

This score is the performance profiling from TensorRT trtexec binary.
Please add the --dumpProfile flag to output the layer-level profiling data:

$ /usr/src/tensorrt/bin/trtexec --onnx=[your/model] --dumpProfile

Time indicates the total execution time for the 551 iterations.
The avg time is the average time for each iteration.
Ex. 0.02 (Avg Time) = 10.92 (Time) / 551 (iterations)

Please note that TensorRT will automatically merge layers for better performance.
So you may find a fully-connected layer takes a long time since it also includes the operation from other layers.


This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.