tensorrt not improving the speed of yolov3 on 1080ti

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

I test yolov3 with tensorrt and witout tensorrt, I found the speed of backbone is no difference. I check my code is no problem, I guess the problem from model. Because the resnet50 can speed up with tensorrt

you can get test project from https://github.com/linssswww/bak_code

this is my envinronment and hardware:

1080Ti
cuda10.1
cudnn 7.5.1
ubuntu 16.04
tensorrt 5.1.5.0
pytorch 1.1.0

somebody meet same the issue
https://devtalk.nvidia.com/default/topic/1045913/tensorrt/no-improvement-in-fps-yolov3/post/5367749/#5367749

2

Hey dude, how do you port yolov3 to TensorRT runtime? I simply use a pytorch ver. of yolov3 and export it to onnx, I hope to build a TensorRT engine from the exported onnx model but failed. Seems that TensorRT does not support some operations of yolov3. Could you provide some insights into reimpl yolov3 with TRT?

3

I actually originally started with a pytorch yolov3.

Then i spent some time using the examples in tensorRT to convert yolov3 → ONNX → TensorRT
and it works :)

but sadly i got worse results than the pytorch implementation which was disappointing.

as far as im aware its only the yolo layer that isnt support but trt should optimise the other layers

this was on a gtx 1050 ti

Regards Andrew

4

Would recommend to try latest TRT version and let us know if the issue persists on the newer TensorRT releases.

Thanks

5

I am using TensorRT 8.6, and on YOLOv5 I face the same problem. The TRT model performs worst than the pytorch model