# YOLOv3-tiny TensorRT C++ Plugins Issue

**URL:** <https://forums.developer.nvidia.com/t/yolov3-tiny-tensorrt-c-plugins-issue/140366>\
**Category:** TensorRT\
**Tags:** yolo\
**Created:** [July 4, 2020, 11:04am UTC](https://forums.developer.nvidia.com/t/yolov3-tiny-tensorrt-c-plugins-issue/140366 "2020-07-04T11:04:17Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![mmubeen](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@mmubeen](https://forums.developer.nvidia.com/u/mmubeen)\
**Post date:** [July 4, 2020, 11:04am UTC](https://forums.developer.nvidia.com/t/yolov3-tiny-tensorrt-c-plugins-issue/140366/1 "2020-07-04T11:04:17Z")

</div>

## Environment

**TensorRT Version** : 6.0.1.10  
**DeepStream SDK Version** : 4.0.2  
**GPU Type** : Jetson Xavier running Jetpack 4.3  
**Nvidia Driver Version** :  
**CUDA Version** : 10.0  
**CUDNN Version** : 7.6.3  
**Operating System + Version** : Ubuntu 18.04  
**Baremetal or Container (if container which image + tag)**:

## Description

I am trying to load a TensorRT based engine file for Darknet YOLOv3-tiny in my C++ program. I converted my darknet weights to a TensorRT engine file using the examples for YOLO located in `/opt/nvidia/deepstream/deepstream-4.0/sources/objectDetector_Yolo/` directory. The major steps while doing so included: -

- Updating the `custom-network-config` and `model-file` parameters to point towards my yolov3-tiny cfg file and yolov3-tiny weights file respectively in `config_infer_primary_yoloV3_tiny.txt`
- Then running the application using the command `deepstream-app -c deepstream_app_config_yoloV3_tiny.txt`. It did run perfectly fine and I was able to see the output on the test stream (.mp4 file).

**Note:** I did go to the `nvdsinfer_custom_impl_Yolo` folder and updated the `nvdsparsebbox_Yolo.cpp` file according to use case and made the project resulting in the `libnvdsinfer_custom_impl_Yolo.so` file which was updated according to the use case.

Now I am trying to load this same engine file provided by deepstream to be loaded in my C++ program. I know that this engine file uses some custom tensorrt plugins. According to my understanding, these plugins could be found in the `libnvdsinfer_custom_impl_Yolo.so`, therefore I link my c++ program with this .so using the `-lnvdsinfer_custom_impl_Yolo` flag. Currently, my program is: -

```auto
#include <iostream>
#include <string>
#include <vector>
#include <array>
#include <sstream>
#include <fstream>
#include <cassert>
#include <numeric>
#include <stdexcept>
#include <stdio.h>

#include <unistd.h>

#include "NvInfer.h"
#include "NvInferPlugin.h"
#include <NvInferRuntimeCommon.h>

class Logger : public nvinfer1::ILogger
{
    void log(nvinfer1::ILogger::Severity severity, const char *msg) override
    {
        // suppress info-level messages
        if (severity != nvinfer1::ILogger::Severity::kINFO)
            std::cout << msg << std::endl;
    }
} gLogger;

int main(int argc, char const *argv[])
{
    initLibNvInferPlugins(&gLogger, "");

    nvinfer1::IRuntime *runtime{nullptr};
    nvinfer1::ICudaEngine *engine{nullptr};
    nvinfer1::IExecutionContext *context{nullptr};

    std::string engine_file_path = "model_b1_fp32.engine";

    std::stringstream gieModelStream; 
    gieModelStream.seekg(0, gieModelStream.beg); 
    std::ifstream cache( engine_file_path ); 
    gieModelStream << cache.rdbuf();
    cache.close();

    runtime = nvinfer1::createInferRuntime(gLogger); 
    assert(runtime != nullptr); 

    gieModelStream.seekg(0, std::ios::end);
    const int modelSize = gieModelStream.tellg(); 
    gieModelStream.seekg(0, std::ios::beg);
    void* modelMem = malloc(modelSize); 
    gieModelStream.read((char*)modelMem, modelSize);
    engine = runtime->deserializeCudaEngine(modelMem, modelSize, nullptr);
    assert(engine != nullptr); 
    free(modelMem);
    
    std::cout << "Loaded\n";

    context = engine->createExecutionContext();

    assert(engine->getNbBindings() == 2);

    context->destroy();
    engine->destroy();
    runtime->destroy();
    
    return 0;
}

```

The program compiles perfectly fine and I can load the plugins from the .so file (presumably). The output shows

```auto
mubeen@jetson-xavier-5157:~/workspace/yolo_trt$ ./main
Plugin Creator registration succeeded - GridAnchor_TRT
Plugin Creator registration succeeded - GridAnchorRect_TRT
Plugin Creator registration succeeded - NMS_TRT
Plugin Creator registration succeeded - Reorg_TRT
Plugin Creator registration succeeded - Region_TRT
Plugin Creator registration succeeded - Clip_TRT
Plugin Creator registration succeeded - LReLU_TRT
Plugin Creator registration succeeded - PriorBox_TRT
Plugin Creator registration succeeded - Normalize_TRT
Plugin Creator registration succeeded - RPROI_TRT
Plugin Creator registration succeeded - BatchedNMS_TRT
Plugin Creator registration succeeded - FlattenConcat_TRT
INVALID_ARGUMENT: getPluginCreator could not find plugin YoloLayerV3_TRT version 1
safeDeserializationUtils.cpp (259) - Serialization Error in load: 0 (Cannot deserialize plugin since corresponding IPluginCreator not found in Plugin Registry)
INVALID_STATE: std::exception
INVALID_CONFIG: Deserialize the cuda engine failed.
main: main.cpp:66: int main(): Assertion `engine != nullptr' failed.
Aborted (core dumped)

```

But as you can see I am still unable to load the model and it cannot find the `YoloLayerV3_TRT` plugin, shouldn’t this plugin be in the .so file, or am I missing something? Any help would be really appreciated.

---

<div class="post-metadata">

**Author:** ![AakankshaS](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/aakankshas/32/14047_2.png) [@AakankshaS](https://forums.developer.nvidia.com/u/AakankshaS)\
**Post date:** [July 4, 2020, 8:02pm UTC](https://forums.developer.nvidia.com/t/yolov3-tiny-tensorrt-c-plugins-issue/140366/2 "2020-07-04T20:02:13Z")

</div>

Hi @mmubeen,  
To use TensorRT registered plugins in your application, the libnvinfer\_plugin.so library must be loaded and all plugins must be registered. This can be done by calling `initLibNvInferPlugins(void* logger, const char* libNamespace)()` in your application code.  
Please note that If you have your own plugin library, you can include a similar entry point to register all plugins in the registry under a unique namespace. This ensures there are no plugin name collisions during build time across different plugin libraries.  
Please find the reference link for adding custom plugin

> **[Developer Guide :: NVIDIA Deep Learning TensorRT Documentation](https://docs.nvidia.com/deeplearning/tensorrt/archives/tensorrt-713/developer-guide/index.html#add_custom_layer)**
>
> This is the revision history of the NVIDIA TensorRT 8.4 Developer Guide.

Thanks!