I’ve converted a yolov4 darknet model to an onnx using some conversion script I found online (https://github.com/Tianxiaomo/pytorch-YOLOv4). I then run the generated onnx using tensorRT. I had no errors during the conversion. All output from the tensorRT inference of yolov4 are as it should be so I don’t think I’m doing anything wrong there.
I’ve observed that the speed is not as it should be. I’m working under the assumption that the tensorRT engines should run faster than the darknet models.
Here are some numbers:
yolov4_416_416: 40ms (using darknet)
yolov4_416_416: 45ms (using onnx -> converted to tensorRT engine)
The yolov4 tensorRT engine seems to be running slower than the yolov4 darknet. Any reason why?
For comparison, I used a yolov3 onnx (converted using a different script). The tensorRT engine runs faster than the darknet in this case. I use the same inference script as I did for the yolov4.
yolov3_416_416: 38ms (using darknet)
yolov3_416_416: 30ms (using onnx - converted to tensorRT engine)
*I used input size of 416 * 416 * 3 and batch size of 4 for my tensorRT results.
TensorRT Version: 18.104.22.168
GPU Type: GTX 1060
Nvidia Driver Version: 22.214.171.12492
CUDA Version: 10.2
CUDNN Version: 7.6.5
Operating System + Version: Windows 10
Python Version (if applicable): Python 3.6.6
TensorFlow Version (if applicable): 1.5.0
PyTorch Version (if applicable): 1.4.0
Baremetal or Container (if container which image + tag):