TensorRT customized plugin

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

I have customized a plugin layer, and I can use it by building from source to libnvinfer_plugin.so.
I wanna ask if any possible we can use the plugin layer without building libnvinfer_plugin.so?
Can we just maybe “include” or "import " our customized plugin in some way?

2

Hi,
Please refer to below links related custom plugin implementation and sample:
https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/sampleOnnxMnistCoordConvAC

Thanks!

5

Hi, I knew the work flow about using plugin layer.
However, my question means whether we can skip building plugin library.(no compile). Instead, we just directly use(include) that plugin class we defined.

Can we skip this step?

Build the plugin library in TensorRT

6

Hi @p890040,

We can build plugin into a separate library and dlopen (or otherwise load) it. We don’t need to modify or recompile libnvinfer_plugin.so.

Thank you.

7

Hi @spolisetty
Thanks your response.
So we can compile our plugin to a library based on prebuilt tensorRT?
If it is. Where could I find some example like that?

Thank you a lot.

8

Hi @p890040,

Assuming the plugin is in some file called customPlugin.cpp , then it would just be something like:
nvcc customPlugin.cpp -shared -o my_plugin.so and then try loading my_plugin.so

Thank you.