Tutorial: How to run YOLOv7 on Deepstream

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Hi,

To help people run official YOLOv7 models on Deepstream here is some helper code.

You should first export the model to ONNX via this command (taken from the yolov7 README)

python export.py --weights ./yolov7-tiny.pt --grid --end2end --simplify --topk-all 100 --iou-thres 0.65 --conf-thres 0.35 --img-size 640 640

This command will create an ONNX model with an efficientNMS node. To be able to map the model outputs from efficientNMS to the NVIDIA NvDsObjectMeta data structure you need this code which needs to be added/compiled in /opt/nvidia/deepstream/deepstream-6.1/sources/libs/nvdsinfer_customparser/nvdsinfer_custombboxparser.cpp

extern "C"
bool NvDsInferParseCustomEfficientNMS (std::vector<NvDsInferLayerInfo> const &outputLayersInfo,
                                   NvDsInferNetworkInfo  const &networkInfo,
                                   NvDsInferParseDetectionParams const &detectionParams,
                                   std::vector<NvDsInferObjectDetectionInfo> &objectList) {
    if(outputLayersInfo.size() != 4)
    {
        std::cerr << "Mismatch in the number of output buffers."
                  << "Expected 4 output buffers, detected in the network :"
                  << outputLayersInfo.size() << std::endl;
        return false;
    }
    const char* log_enable = std::getenv("ENABLE_DEBUG");

    int* p_keep_count = (int *) outputLayersInfo[0].buffer;

    float* p_bboxes = (float *) outputLayersInfo[1].buffer;
    NvDsInferDims inferDims_p_bboxes = outputLayersInfo[1].inferDims;
    int numElements_p_bboxes=inferDims_p_bboxes.numElements;

    float* p_scores = (float *) outputLayersInfo[2].buffer;
    unsigned int* p_classes = (unsigned int *) outputLayersInfo[3].buffer;

    const float threshold = detectionParams.perClassThreshold[0];

    float max_bbox=0;
    for (int i=0; i < numElements_p_bboxes; i++)
    {
        if ( max_bbox < p_bboxes[i] )
            max_bbox=p_bboxes[i];
    }

    if (p_keep_count[0] > 0)
    {
        assert (!(max_bbox < 2.0));
        for (int i = 0; i < p_keep_count[0]; i++) {

            if ( p_scores[i] < threshold) continue;
            assert((unsigned int) p_classes[i] < detectionParams.numClassesConfigured);

            NvDsInferObjectDetectionInfo object;
            object.classId = (int) p_classes[i];
            object.detectionConfidence = p_scores[i];

            object.left=p_bboxes[4*i];
            object.top=p_bboxes[4*i+1];
            object.width=(p_bboxes[4*i+2] - object.left);
            object.height= (p_bboxes[4*i+3] - object.top);

            if(log_enable != NULL && std::stoi(log_enable)) {
                std::cout << "label/conf/ x/y w/h -- "
                << p_classes[i] << " "
                << p_scores[i] << " "
                << object.left << " " << object.top << " " << object.width << " "<< object.height << " "
                << std::endl;
            }

            object.left=CLIP(object.left, 0, networkInfo.width - 1);
            object.top=CLIP(object.top, 0, networkInfo.height - 1);
            object.width=CLIP(object.width, 0, networkInfo.width - 1);
            object.height=CLIP(object.height, 0, networkInfo.height - 1);

            objectList.push_back(object);
        }
    }
    return true;
}

You then just need to make sure your nvinfer stages point to the correct custom library and function like:

parse-bbox-func-name=NvDsInferParseCustomEfficientNMS
custom-lib-path=/opt/nvidia/deepstream/deepstream-6.1/lib/libnvds_infercustomparser.so

With that you should be good to go.

4 Likes
2

Thanks for your sharing!

3

Hello, could you help me please. I run this example deepstream_python_apps/apps/deepstream-rtsp-in-rtsp-out at master · NVIDIA-AI-IOT/deepstream_python_apps · GitHub
with your example.
I performed your instructions. When some object is detected, I get this:

python3: nvdsinfer_custombboxparser.cpp:589: bool NvDsInferParseCustomEfficientNMS(const std::vector&, const NvDsInferNetworkInfo&, const NvDsInferParseDetectionParams&, std::vector&): Assertion `(unsigned int) p_classes[i] < detectionParams.numClassesConfigured’ failed.

img_log

Screenshot_20221006_234653
Screenshot_20221006_2346531101×667 124 KB

4

Hi @xenonscrinium .

The issue is probably that you have not updated num-detected-classes=­91 to be correct for the model you are running. This assertion checks that any detected class from the model is less than this num-detected-classes variable.

1 Like
5

It works, thanks for your sharing.

6

Hello,
I followed the steps you mentioned above but I am not getting any detections, using deepstream_test3.
my config file:-

[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
model-color-format=0
onnx-file=yolov7-tiny.onnx
model-engine-file=yolov7-tiny.onnx_b2_gpu0_fp16.engine
labelfile-path=models/coco.txt
num-detected-classes=80
operate-on-class-ids=0;1;2;9
interval=0
gie-unique-id=1
batch-size=2
network-mode=2
process-mode=1
network-type=2
cluster-mode=2
maintain-aspect-ratio=1
parse-bbox-func-name=NvDsInferParseCustomEfficientNMS
custom-lib-path=libnvds_infercustomparser.so

[class-attrs-all]
nms-iou-threshold=0.45
pre-cluster-threshold=0.25
topk=300
7

operate-on-class-ids=0;1;2;9 is for secondary inference so might be breaking this?

8

Tried running without this line, nothing still :(

9

you could try export ENABLE_DEBUG=1 as it will show if there are actually any detections

10

Anyone willing to share a gst-launch example to try this?

11

Nevermind, works flawlessly!

1 Like
12

When I run ‘sudo -E make install’ I get the following problems which don’t seem realted to the code posted above (Jeston Xavier NX):

g++ -o libnvds_infercustomparser.so nvdsinfer_custombboxparser.cpp nvdsinfer_customclassifierparser.cpp -Wall -std=c++11 -shared -fPIC -I…/…/includes -I /usr/local/cuda-11.4/include -Wl,–start-group -lnvinfer -lnvparsers -Wl,–end-group
In file included from /usr/include/aarch64-linux-gnu/NvInferRuntimeCommon.h:19,
from /usr/include/aarch64-linux-gnu/NvInferLegacyDims.h:16,
from /usr/include/aarch64-linux-gnu/NvInfer.h:16,
from /usr/include/aarch64-linux-gnu/NvCaffeParser.h:16,
from …/…/includes/nvdsinfer_custom_impl.h:126,
from nvdsinfer_custombboxparser.cpp:25:
/usr/local/cuda-11.4/include/cuda_runtime_api.h:147:10: fatal error: crt/host_defines.h: No such file or directory
147 | #include “crt/host_defines.h”
| ^~~~~~~~~~~~~~~~~~~~
compilation terminated.
In file included from /usr/include/aarch64-linux-gnu/NvInferRuntimeCommon.h:19,
from /usr/include/aarch64-linux-gnu/NvInferLegacyDims.h:16,
from /usr/include/aarch64-linux-gnu/NvInfer.h:16,
from /usr/include/aarch64-linux-gnu/NvCaffeParser.h:16,
from …/…/includes/nvdsinfer_custom_impl.h:126,
from nvdsinfer_customclassifierparser.cpp:25:
/usr/local/cuda-11.4/include/cuda_runtime_api.h:147:10: fatal error: crt/host_defines.h: No such file or directory
147 | #include “crt/host_defines.h”
| ^~~~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:41: libnvds_infercustomparser.so] Error 1

Any idea?

1 Like
13

I was able to run the model using the provided tutorial on x86 architecture but when trying to run it in jetson docker, am facing the same issue. Seems like the C files are not available in the docker.

Please let me know if you were able to find a workaround.

Thank you

14

Jetson docker does not have the libraries to build the static library. You will need to build it on the host by installing deepstream directly on host.

15

Thank you so much for your kind response.

BTW I was able to run it on Jetson Docker.

17

From where did you get the files to compile?
Because from https://github.com/marcoslucianops/DeepStream-Yolo it does not work for me:

0:00:01.212635810 18436      0x1ccc720 WARN                     omx gstomx.c:2826:plugin_init: Failed to load configuration file: Valid key file could not be found in search dirs (searched in: /home/watchbot/.config:/etc/xdg/xdg-unity:/etc/xdg as per GST_OMX_CONFIG_DIR environment variable, the xdg user config directory (or XDG_CONFIG_HOME) and the system config directory (or XDG_CONFIG_DIRS)
ERROR: Deserialize engine failed because file path: /opt/nvidia/deepstream/deepstream-6.0/samples/configs/deepstream-app/models/basic/yolov7n_b4_dla0_int8.engine open error
0:00:03.011958049 18436      0x1ccc720 WARN                 nvinfer gstnvinfer.cpp:635:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1889> [UID = 1]: deserialize engine from file :/opt/nvidia/deepstream/deepstream-6.0/samples/configs/deepstream-app/models/basic/yolov7n_b4_dla0_int8.engine failed
0:00:03.012069058 18436      0x1ccc720 WARN                 nvinfer gstnvinfer.cpp:635:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1996> [UID = 1]: deserialize backend context from engine from file :/opt/nvidia/deepstream/deepstream-6.0/samples/configs/deepstream-app/models/basic/yolov7n_b4_dla0_int8.engine failed, try rebuild
0:00:03.012111843 18436      0x1ccc720 INFO                 nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1914> [UID = 1]: Trying to create engine from model files
YOLO config file or weights file is not specified

ERROR: Failed to create network using custom network creation function
ERROR: Failed to get cuda engine from custom library API
0:00:03.012937387 18436      0x1ccc720 ERROR                nvinfer gstnvinfer.cpp:632:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1934> [UID = 1]: build engine file failed
0:00:03.012992556 18436      0x1ccc720 ERROR                nvinfer gstnvinfer.cpp:632:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2020> [UID = 1]: build backend context failed
0:00:03.013025260 18436      0x1ccc720 ERROR                nvinfer gstnvinfer.cpp:632:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::initialize() <nvdsinfer_context_impl.cpp:1257> [UID = 1]: generate backend failed, check config file settings
0:00:03.013085133 18436      0x1ccc720 WARN                 nvinfer gstnvinfer.cpp:841:gst_nvinfer_start:<primary_gie> error: Failed to create NvDsInferContext instance
0:00:03.013120909 18436      0x1ccc720 WARN                 nvinfer gstnvinfer.cpp:841:gst_nvinfer_start:<primary_gie> error: Config file path: /opt/nvidia/deepstream/deepstream-6.0/samples/configs/deepstream-app/config_infer_primary_yoloV7.txt, NvDsInfer Error: NVDSINFER_CONFIG_FAILED
0:00:03.013308847 18436      0x1ccc720 WARN                GST_PADS gstpad.c:1149:gst_pad_set_active:<primary_gie:sink> Failed to activate pad
** ERROR: <main:658>: Failed to set pipeline to PAUSED
Quitting
ERROR from primary_gie: Failed to create NvDsInferContext instance

I understand with this that it is required wts and cfg, not ONNX.

18

Hi,
We uploaded the code here:

19

hello, I couldn’t detect any targets, did you solve them later？

20

I think the issue is caused because the exported model batch size > 1, it works with batch size =1 ,
for higher batch sizes I used this.