Wrong SSD inference results using TensorRT python API

Hi,

I am using TensorRT python API. I was able to import a Caffe model of SSD network using the built-in TensorRT plugin layers and following user instructions. Unfortunately, when inferring the outputs are incorrect compared to the original Caffe implementation using the OpenCV DNN libraries: PriorBox layers seem to provide different outputs for the same inputs between the two implementations.

I decided to reproduce the problem on the well known TensorRT use case “sample SSD” located in “/usr/src/tensorrt/samples/sampleSSD”. This use case uses the TensorRT C ++ API and provides results comparable to the original Caffe implementation. But after translating the use case to use the TensorRT python API, the network outputs are again incorrect.

Here is the python implementation of the “Sample SSD” that I used : sampleSSD_python.tar.gz (685.0 KB)

I am stuck and would like to know if this is a bug in the TensorRT python API regarding the handling of PriorBox layers or if I misunderstood something important.

Thanks for reading and helping me to solve this issue.

Hardware : Jetson Nano
JetPack : 4.5
OS: Ubuntu 18.04
Cuda : version 10.2
cuDNN : version 8.0
TensorRT 7.1

Hi,

FYI, network outputs are also incorrect when using the python implementation of “Sample SSD” with the TensorRT engine generated by the C ++ implementation of “Sample SSD”. The problem therefore does not seem to be related to the TensorRT engine.

I confirm that the issue comes from the “PiorBox” layers that generate wrong outputs during inference compared to both the TensorRT C ++ implementation and the original Caffe implementation. So it looks like a real bug.

Can experts have a look at the implementation of “PriorBox” layer when called from the TensorRT python API ?

Hi,

Since python interface is just a wrapper, this is more like an implementation issue.

A common cause is from the zero padding in cvmat.
Could you help to confirm if you can get a similar result in the layer right before PriorBox?

Thanks.

Hi,

Yes, I confirm that the results are similar in the layer right before PriorBox.
Don’t hesitate to ask if I can perform other tests to help you to analyze and solve this issue.

Thanks

Hi,

It seems that you load a prebuilt TensorRT engine in the python example.
Could you try to create the TensorRT engine from caffemodel parser?

Since we do have a issue related to PriorBox accuracy and the root cause is from the serializer.
This issue is fixed internally and you won’t meet it in our next JetPack release.

Thanks.

Hi,

I confirm that when I build the TensorRT engine and then immediatly make an inference, the results are correct. If I try to serialize and save the engine, and then make an inference based on the engine reloaded from the file, the results are incorrect. So, I confirm that the issue is the same as the one you mentionned (sorry I didn’t see it before).

You said the issue is fixed internally, but will not be released soon.
In which JetPack can we expect to benefit from the fix ?

Thanks

Hi,

The fix is included in our next JetPack release.
We will let you know once it is available.

Thanks.

Hi,

That’s great, thanks a lot for the good news !