hello, i have trained resnet50 classfier with myself dataset(number_classes=7). And i have tried to inference the trained model for int8 mode,but the inference speed is almost same with fp16 mode.I don’t know why. who can tell me the resons to cause it.THANKS.
Could you share more detail about test of the performance?
Do you use the TensorFlow or our TensorRT trtexec binary?
i used pytorch->onnx->tensorrt. So, how can i see the inference precision of each layer? i used python,how can i find the log?
You can get the layer-level performance result with trtexec + onnx input directly.
/usr/src/tensorrt/bin/trtexec --onnx=/usr/src/tensorrt/data/resnet50/ResNet50.onnx --dumpProfile
[11/30/2020-14:54:20] [I] === Profile (187 iterations ) === [11/30/2020-14:54:20] [I] Layer Time (ms) Avg. Time (ms) Time % [11/30/2020-14:54:20] [I] (Unnamed Layer* 0) [Convolution] + (Unnamed Layer* 2) [Activation] 86.29 0.46 2.7 [11/30/2020-14:54:20] [I] (Unnamed Layer* 3) [Pooling] 13.92 0.07 0.4 .. [11/30/2020-14:54:20] [I] (Unnamed Layer* 179) [ElementWise] 1.58 0.01 0.0 [11/30/2020-14:54:20] [I] (Unnamed Layer* 181) [Softmax] 1.66 0.01 0.1 [11/30/2020-14:54:20] [I] Total 3209.13 17.16 100.0
Or you can add your own profiler with below’s API: