Hello,
I am doing some analysis of how fast TensorRT works compared to others like TensorFlow.
I was wondering, is it possible to know how long it takes for each layer of a model to be computed by TensorRT?
Hello,
I am doing some analysis of how fast TensorRT works compared to others like TensorFlow.
I was wondering, is it possible to know how long it takes for each layer of a model to be computed by TensorRT?
Hi,
Request you to share the model, script, profiler, and performance output if not shared already so that we can help you better.
Alternatively, you can try running your model with trtexec command.
While measuring the model performance, make sure you consider the latency and throughput of the network inference, excluding the data pre and post-processing overhead.
Please refer to the below links for more details:
https://docs.nvidia.com/deeplearning/tensorrt/archives/tensorrt-803/best-practices/index.html#measure-performance
Thanks!
I am using jkjung’s YOLOv3-tiny implementation and want to know the time it takes to execute each layer.
github/jkjung-avt/tensorrt_demos/tree/master/yolo
Hi,
We can get to know using trtexec
command. Please use --dumpProfile
--separateProfileRun
options along with trtexec
to check per-layer performance.
Thank you.