Custom Layer Implementation - TensorRT - Tensorflow model

Autonomous Vehicles DRIVE PX2 DriveWorks
#1

Hello,

Consider that I have a model with all the layers that are supported by TensorRT. The workflow for deploying this model on the DrivePX2 is

  1. Tensorflow model -> UFF model on Host
  2. Copy UFF file to PX2
  3. UFF model -> tensorRT engine on DPX2
  4. Load tensorRT engine using C++ API on DPX2 for inferencing

This is according to
https://devtalk.nvidia.com/default/topic/1030068/driveworks/-solved-tensorrt3-tensorflow-implementation-in-px2/
http://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#convert_model_tensorflow
and the sampleUFFMnist TensorRT sample

I now want to introduce dropout layers to this model. The dropout layers will be used during inference and this layer is currently unsupported by TensorRT.

What would be my workflow for the same?

Thanks,
serinvarghese

#2

Dear serinvarghese,
Can you check the below steps.

  • Implement the custom layer plugin in C++ as shown in samplePlugin
  • Use createUffParser to parse UFF file as shown in sampleUffMNSIT
  • Use parser->setPluginFactory() for custom layers as shown in samplePlugin
#3

There isnt a a function to use Plugins with C++ API with TRT 3.0.2(on PX2), has there been an update after this that allows the same

#4

Dear Dhingratul,
I did not get your question. If you are asking about using plugin layer in tensorRT, Can you please check samplePlugin sample.

#5

I am looking for a “samplePlugin” equivalent for UFF based models. In one of the posts, it is mentioned that there would be a SSD sample with UFF in TRT 4 GA, I am asking if that is available yet ?

#6

Yes. SSD sample with UFF is available in TRT 4 GA. Please check https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#uffssd_sample

#7

Yes, I did and I have a few bugs with it and TRT 4GA that are still unresolved, maybe you can help expedite the resolution for that.

https://devtalk.nvidia.com/default/topic/1026702/jetson-tx2/problem-with-custom-layers-and-python-uff-parser-in-tensorrt-3-0-rc/post/5275144/#5275144

https://devtalk.nvidia.com/default/topic/1036974/tensorrt/convert-to-uff-throws-invalid-data-type-typeerror-/post/5267989/#5267989

#8

Hi, Dhingratul

Okay. Let’s track that in the original topic.
Thanks.

#9

@AastaLLL any progress on either of the bugs ?

#10

Please check the original topic:
https://devtalk.nvidia.com/default/topic/1026702/jetson-tx2/problem-with-custom-layers-and-python-uff-parser-in-tensorrt-3-0-rc/3/?offset=39#5280597

Thanks.