# ONNX Plugin Layer implements **URL:** \ **Category:** TensorRT\ **Created:** [December 16, 2020, 7:08am UTC](https://forums.developer.nvidia.com/t/onnx-plugin-layer-implements/163792 "2020-12-16T07:08:19Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![disculus2012](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@disculus2012](https://forums.developer.nvidia.com/u/disculus2012)\ **Post date:** [December 16, 2020, 7:08am UTC](https://forums.developer.nvidia.com/t/onnx-plugin-layer-implements/163792/1 "2020-12-16T07:08:19Z")
## Description Hi, I have downloaded ssd.onnx and run with onnx parser to generate .trt file. But I met this error ``` While parsing node number 464 [NonMaxSuppression]: ERROR: ModelImporter.cpp:134 In function parseGraph: [8] No importer registered for op: NonMaxSuppression &&&& FAILED TensorRT.sample_onnx_mnist # ./test_onnx ``` It seems that NonMaxSuppressionPlugin is not available. Then I tried to implement plugin for custom layer. I follow nmsPlugin example and create a NonMaxSuppressionPlugin class. > 1. Make directory NonMaxSuppressionPlugin in plugin folder > 2. Copy nmsPlugin.cpp and nmsPlugin.h to folder NonMaxSuppressionPlugin > 3. Change nmsPlugin and nmsPluginCreator to NonMaxSuppressionPlugin and NonMaxSuppressionPluginCreator. > 4. Add into CMake list > 5. Add DEFINE\_BUILTIN\_OP\_IMPORTER(NonMaxSuppression) in parsers/onnx/builtin\_op\_importers.cpp > 6. Add initializePlugin\< nvinfer1::plugin::NonMaxSuppressionPluginCreator \>(logger, libNamespace) in InferPlugin.cpp > 7. Regenerate the in build folder by “cmake .. -DTRT\_LIB\_DIR=~/TensorRT/lib -DTRT\_BIN\_DIR=`pwd`/out -DBUILD\_PLUGINS=ON -DBUILD\_PARSERS=ON” > 8. make in build folder > 9. make install After rebuild I used onnx2trt ssd-10.onnx -o ssd\_trt.out to convert model It gives message : ``` Input filename: ../../../samples/data/ssd-10.onnx ONNX IR version: 0.0.4 Opset version: 10 Producer name: pytorch Producer version: 1.1 Domain: Model version: 0 Doc string: ---------------------------------------------------------------- Parsing model [2020-12-16 06:36:24 WARNING] /home/u5393118/TensorRT/parsers/onnx/onnx2trt_utils.cpp:235: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32. ..... [2020-12-16 06:36:24 WARNING] /home/u5393118/TensorRT/parsers/onnx/onnx2trt_utils.cpp:261: One or more weights outside the range of INT32 was clamped [2020-12-16 06:36:24 ERROR] INVALID_ARGUMENT: getPluginCreator could not find plugin NonMaxSuppressionONNXTRT_NAMESPACE version 001 ``` I’ not sure if I miss something in register the NonMaxSuppressionPlugin in to ONNX parser. Is any implementation example in ONNX plugin? ## Environment **TensorRT Version** : 7.0.0-1 **GPU Type** : Tesla V100 **Nvidia Driver Version** : 450.51.05 **CUDA Version** : 11.0 **CUDNN Version** : **Operating System + Version** : ubuntu 18.04 **Python Version (if applicable)** : 3.6.9 **TensorFlow Version (if applicable)** : **PyTorch Version (if applicable)** : **Baremetal or Container (if container which image + tag)** : ## Relevant Files ONNX model is downloaded from [https://github.com/onnx/models/tree/master/vision/object\_detection\_segmentation/ssd](https://github.com/onnx/models/tree/master/vision/object_detection_segmentation/ssd) ## Steps To Reproduce > 1. Make directory NonMaxSuppressionPlugin in plugin folder > 2. Copy nmsPlugin.cpp and nmsPlugin.h to folder NonMaxSuppressionPlugin > 3. Change nmsPlugin and nmsPluginCreator to NonMaxSuppressionPlugin and NonMaxSuppressionPluginCreator. > 4. Add into CMake list > 5. Add DEFINE\_BUILTIN\_OP\_IMPORTER(NonMaxSuppression) in parsers/onnx/builtin\_op\_importers.cpp > 6. Add initializePlugin\< nvinfer1::plugin::NonMaxSuppressionPluginCreator \>(logger, libNamespace) in InferPlugin.cpp > 7. Regenerate the in build folder by “cmake .. -DTRT\_LIB\_DIR=~/TensorRT/lib -DTRT\_BIN\_DIR=`pwd`/out -DBUILD\_PLUGINS=ON -DBUILD\_PARSERS=ON” > 8. make in build folder > 9. make install > 10. in build/parsers/onnx/ run onnx2trt ssd-10.onnx -o ssd.trt Please include: - Exact steps/commands to build your repro - Exact steps/commands to run your repro - Full traceback of errors encountered ---
**Author:** ![AakankshaS](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/aakankshas/32/14047_2.png) [@AakankshaS](https://forums.developer.nvidia.com/u/AakankshaS)\ **Post date:** [December 16, 2020, 7:18am UTC](https://forums.developer.nvidia.com/t/onnx-plugin-layer-implements/163792/2 "2020-12-16T07:18:38Z")
Hi @disculus2012, Please refer to below samples: > **[TensorRT/samples/opensource/sampleFasterRCNN at...](https://github.com/NVIDIA/TensorRT/tree/07ed9b57b1ff7c24664388e5564b17f7ce2873e5/samples/opensource/sampleFasterRCNN)** > > NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT. - NVIDIA/TensorRT > ---
**Author:** ![NVES](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/nves/32/14043_2.png) [@NVES](https://forums.developer.nvidia.com/u/NVES)\ **Post date:** [December 16, 2020, 7:37am UTC](https://forums.developer.nvidia.com/t/onnx-plugin-layer-implements/163792/3 "2020-12-16T07:37:30Z")
Hi, Request you to check the below reference links for custom plugin implementation. [https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/sampleOnnxMnistCoordConvAC](https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/sampleOnnxMnistCoordConvAC) > **[Estimating Depth with ONNX Models and Custom Layers Using NVIDIA TensorRT |...](https://developer.nvidia.com/blog/estimating-depth-beyond-2d-using-custom-layers-on-tensorrt-and-onnx-models/)** > > TensorRT is an SDK for high performance, deep learning inference. It includes a deep learning inference optimizer and a runtime that delivers low latency and high throughput for deep learning… Thanks! ---
**Author:** ![Sneaky\_Turtle](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Sneaky\_Turtle](https://forums.developer.nvidia.com/u/Sneaky_Turtle)\ **Post date:** [January 9, 2021, 1:47pm UTC](https://forums.developer.nvidia.com/t/onnx-plugin-layer-implements/163792/4 "2021-01-09T13:47:26Z")
@disculus2012 Did you ever get this solved? ---
**Author:** ![disculus2012](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@disculus2012](https://forums.developer.nvidia.com/u/disculus2012)\ **Post date:** [January 12, 2021, 3:46am UTC](https://forums.developer.nvidia.com/t/onnx-plugin-layer-implements/163792/5 "2021-01-12T03:46:46Z")
I have solved this problem by replacing plugin.so and parser.so in /usr/lib/x86\_64-linux-gnu/ from ${your\_path}/TensorRT/lib. cp ${your\_path}/TensorRT/lib/target.so /usr/lib/x86\_64-linux-gnu/ ---
**Author:** ![Sneaky\_Turtle](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Sneaky\_Turtle](https://forums.developer.nvidia.com/u/Sneaky_Turtle)\ **Post date:** [January 12, 2021, 3:51am UTC](https://forums.developer.nvidia.com/t/onnx-plugin-layer-implements/163792/6 "2021-01-12T03:51:02Z")
@disculus2012 that is the exact command that you used to solve this? “cp ${your\_path}/TensorRT/lib/target.so /usr/lib/x86\_64-linux-gnu/” ---
**Author:** ![disculus2012](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@disculus2012](https://forums.developer.nvidia.com/u/disculus2012)\ **Post date:** [January 12, 2021, 4:07am UTC](https://forums.developer.nvidia.com/t/onnx-plugin-layer-implements/163792/7 "2021-01-12T04:07:01Z")
The problem is that I didn’t update the new shared library I built. So the old shared library doesn’t contain the new plugin importer in ONNX parser. I’m using 7.0.0 version TensorRT so the plugin is 7.0.0 I update two .so files by > cp ~/TensorRT/lib/libnvinfer\_plugin.so.7.0.0 /usr/lib/x86\_64-linux-gnu/ > cp ~/TensorRT/lib/libnvonnxparser.so.7.0.0 /usr/lib/x86\_64-linux-gnu/ The ~/TensorRT/lib/libnvonnxparser.so.7.0.0 may be different in your system. You can check in your TensorRT folder. ---
**Author:** ![Sneaky\_Turtle](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Sneaky\_Turtle](https://forums.developer.nvidia.com/u/Sneaky_Turtle)\ **Post date:** [January 12, 2021, 4:08am UTC](https://forums.developer.nvidia.com/t/onnx-plugin-layer-implements/163792/8 "2021-01-12T04:08:50Z")
Okay, I thought just copying libraries would solve the lack of a NMS plugin. Does your plugin work for multiple models? ---
**Author:** ![disculus2012](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@disculus2012](https://forums.developer.nvidia.com/u/disculus2012)\ **Post date:** [January 12, 2021, 4:16am UTC](https://forums.developer.nvidia.com/t/onnx-plugin-layer-implements/163792/9 "2021-01-12T04:16:30Z")
Sorry, I’m still working on the plugin process part. ---
**Author:** ![Sneaky\_Turtle](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Sneaky\_Turtle](https://forums.developer.nvidia.com/u/Sneaky_Turtle)\ **Post date:** [January 12, 2021, 4:20am UTC](https://forums.developer.nvidia.com/t/onnx-plugin-layer-implements/163792/10 "2021-01-12T04:20:39Z")
lol @klinten someone also replied recently to me asking how to solve this, maybe it helps > [@Writing layer for NonMaxSuppression in onnx parser](https://forums.developer.nvidia.com/t/writing-layer-for-nonmaxsuppression-in-onnx-parser/121012/20): > > I found a working solution. The documentation fails to mention that BatchedNMSPlugin is modeled directly after TensorFlow CombinedNonMaxSuppression: as compared to So I modified my TF model to use CombinedNMS, then wrote a script using ONNX Graphsurgeon that convert nodes from CombinedNonMaxSuppression to BatchedNMSDynamic\_TRT based on the the following mapping from the tf2tensorrt code: ---
**Author:** ![disculus2012](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@disculus2012](https://forums.developer.nvidia.com/u/disculus2012)\ **Post date:** [January 12, 2021, 5:48am UTC](https://forums.developer.nvidia.com/t/onnx-plugin-layer-implements/163792/11 "2021-01-12T05:48:55Z")
Thanks for sharing. ---
**Author:** ![Sneaky\_Turtle](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Sneaky\_Turtle](https://forums.developer.nvidia.com/u/Sneaky_Turtle)\ **Post date:** [January 12, 2021, 6:18am UTC](https://forums.developer.nvidia.com/t/onnx-plugin-layer-implements/163792/12 "2021-01-12T06:18:51Z")
You’re welcome. Please drop by this thread again and let me know if you’ve made progress on getting a plugin working.