Using bin-file after TensorRT optimization

Autonomous Vehicles DRIVE AGX Xavier DRIVE AGX Xavier General
1

Hardware Platform: DRIVE AGX Pegasus™ Developer Kit
Software Version: DRIVE Software 10
Host Machine Version: Ubuntu 18.04
SDK Manager Version: 1.1.0.6343

Hello,

I have the following goal: I want to use a pre-trained yolo network within the DRIVE software stack. I noticed that one has the possibility to optimize onnx files and import the optimized bin-file into self-developed code by using the dwDNN_initializeTensorRTFromFileNew function.

Therefore, I successfully did the yolov3_onnx sample in /usr/src/tensorrt/samples/python/yolov3_onnx. I obtained a yolov3.onnx file, built a TensorRT engine with it and verified the results on the example image. Everything works fine so far.

Afterwards, I used the TensorRT optimization tool and obtained an optimized bin-file. I would like to verify and try it in the “sample_object_detector_tracker”. I tried starting the sample it in the following way:

./sample_object_detector_tracker --tensorRT_model=/home/.../optimized.bin

Unfortunately, I get the following error:

[23-06-2020 18:48:37] getPluginCreator could not find plugin ResizeNearest version 001 namespace 
[23-06-2020 18:48:37] Cannot deserialize plugin ResizeNearest

I don’t know why this error appears. How can this be fixed? I would appreciate any help!

I know that this problem has been addressed in deserialization problem using dwDNN_initializeTensorRTFromFileNew on drive software 10 - #2 by SivaRamaKrishnaNV. However, the problem was not solved.

Thanks in advance!

3

Dear @kn10,
It seems your model uses ResizeNearest TRT plugin. I could see in TRT 5.1 (onnx-tensorrt/builtin_op_importers.cpp at 55d75dec4f289d1c7a63d5cf578b4c6ab441c03e · onnx/onnx-tensorrt · GitHub) resizenearestplugin is used in Upsample layer. Where as in TRT 6.0 upsample layer is supported (onnx-tensorrt/builtin_op_importers.cpp at 397cdbafd898153f69a2b1a87dcc1c4dc5add18b · onnx/onnx-tensorrt · GitHub).
If it is due to upsample layer in ONNX and you are using DRIVE SW 10.0, you need to pick up the plugin implementation from TRT repo and create .so file similar to sample_dnn_plugin sample. The next Drive release(in couple of days) will support upsample layer natively. So your model is expected to work directly.

4

Dear @SivaRamaKrishnaNV,

thank you very much for your reply.

Which TRT repo do you mean in you reply above? I guess the TRT 5.1 repo, since only 5.1 is supported? Also, how can I create the .so-file?

5

Dear @kn10,
I guess the TRT 5.1 repo, since only 5.1 is supported?

Yes.
how can I create the .so-file?
Please check dnn plugin sample and CMakefiles for reference.