Tutorial: How to run YOLOv7 on Deepstream

Hi,

To help people run official YOLOv7 models on Deepstream here is some helper code.

You should first export the model to ONNX via this command (taken from the yolov7 README)

python export.py --weights ./yolov7-tiny.pt --grid --end2end --simplify --topk-all 100 --iou-thres 0.65 --conf-thres 0.35 --img-size 640 640

This command will create an ONNX model with an efficientNMS node. To be able to map the model outputs from efficientNMS to the NVIDIA NvDsObjectMeta data structure you need this code which needs to be added/compiled in /opt/nvidia/deepstream/deepstream-6.1/sources/libs/nvdsinfer_customparser/nvdsinfer_custombboxparser.cpp

extern "C"
bool NvDsInferParseCustomEfficientNMS (std::vector<NvDsInferLayerInfo> const &outputLayersInfo,
                                   NvDsInferNetworkInfo  const &networkInfo,
                                   NvDsInferParseDetectionParams const &detectionParams,
                                   std::vector<NvDsInferObjectDetectionInfo> &objectList) {
    if(outputLayersInfo.size() != 4)
    {
        std::cerr << "Mismatch in the number of output buffers."
                  << "Expected 4 output buffers, detected in the network :"
                  << outputLayersInfo.size() << std::endl;
        return false;
    }
    const char* log_enable = std::getenv("ENABLE_DEBUG");

    int* p_keep_count = (int *) outputLayersInfo[0].buffer;

    float* p_bboxes = (float *) outputLayersInfo[1].buffer;
    NvDsInferDims inferDims_p_bboxes = outputLayersInfo[1].inferDims;
    int numElements_p_bboxes=inferDims_p_bboxes.numElements;

    float* p_scores = (float *) outputLayersInfo[2].buffer;
    unsigned int* p_classes = (unsigned int *) outputLayersInfo[3].buffer;

    const float threshold = detectionParams.perClassThreshold[0];

    float max_bbox=0;
    for (int i=0; i < numElements_p_bboxes; i++)
    {
        if ( max_bbox < p_bboxes[i] )
            max_bbox=p_bboxes[i];
    }

    if (p_keep_count[0] > 0)
    {
        assert (!(max_bbox < 2.0));
        for (int i = 0; i < p_keep_count[0]; i++) {

            if ( p_scores[i] < threshold) continue;
            assert((unsigned int) p_classes[i] < detectionParams.numClassesConfigured);

            NvDsInferObjectDetectionInfo object;
            object.classId = (int) p_classes[i];
            object.detectionConfidence = p_scores[i];

            object.left=p_bboxes[4*i];
            object.top=p_bboxes[4*i+1];
            object.width=(p_bboxes[4*i+2] - object.left);
            object.height= (p_bboxes[4*i+3] - object.top);

            if(log_enable != NULL && std::stoi(log_enable)) {
                std::cout << "label/conf/ x/y w/h -- "
                << p_classes[i] << " "
                << p_scores[i] << " "
                << object.left << " " << object.top << " " << object.width << " "<< object.height << " "
                << std::endl;
            }

            object.left=CLIP(object.left, 0, networkInfo.width - 1);
            object.top=CLIP(object.top, 0, networkInfo.height - 1);
            object.width=CLIP(object.width, 0, networkInfo.width - 1);
            object.height=CLIP(object.height, 0, networkInfo.height - 1);

            objectList.push_back(object);
        }
    }
    return true;
}

You then just need to make sure your nvinfer stages point to the correct custom library and function like:

parse-bbox-func-name=NvDsInferParseCustomEfficientNMS
custom-lib-path=/opt/nvidia/deepstream/deepstream-6.1/lib/libnvds_infercustomparser.so

With that you should be good to go.

3 Likes

Thanks for your sharing!

Hello, could you help me please. I run this example deepstream_python_apps/apps/deepstream-rtsp-in-rtsp-out at master · NVIDIA-AI-IOT/deepstream_python_apps · GitHub
with your example.
I performed your instructions. When some object is detected, I get this:

python3: nvdsinfer_custombboxparser.cpp:589: bool NvDsInferParseCustomEfficientNMS(const std::vector&, const NvDsInferNetworkInfo&, const NvDsInferParseDetectionParams&, std::vector&): Assertion `(unsigned int) p_classes[i] < detectionParams.numClassesConfigured’ failed.

img_log

Hi @xenonscrinium .

The issue is probably that you have not updated num-detected-classes=­91 to be correct for the model you are running. This assertion checks that any detected class from the model is less than this num-detected-classes variable.

1 Like

It works, thanks for your sharing.

Hello,
I followed the steps you mentioned above but I am not getting any detections, using deepstream_test3.
my config file:-

[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
model-color-format=0
onnx-file=yolov7-tiny.onnx
model-engine-file=yolov7-tiny.onnx_b2_gpu0_fp16.engine
labelfile-path=models/coco.txt
num-detected-classes=80
operate-on-class-ids=0;1;2;9
interval=0
gie-unique-id=1
batch-size=2
network-mode=2
process-mode=1
network-type=2
cluster-mode=2
maintain-aspect-ratio=1
parse-bbox-func-name=NvDsInferParseCustomEfficientNMS
custom-lib-path=libnvds_infercustomparser.so

[class-attrs-all]
nms-iou-threshold=0.45
pre-cluster-threshold=0.25
topk=300

operate-on-class-ids=0;1;2;9 is for secondary inference so might be breaking this?

Tried running without this line, nothing still :(

you could try export ENABLE_DEBUG=1 as it will show if there are actually any detections

Anyone willing to share a gst-launch example to try this?

Nevermind, works flawlessly!

1 Like

When I run ‘sudo -E make install’ I get the following problems which don’t seem realted to the code posted above (Jeston Xavier NX):

g++ -o libnvds_infercustomparser.so nvdsinfer_custombboxparser.cpp nvdsinfer_customclassifierparser.cpp -Wall -std=c++11 -shared -fPIC -I…/…/includes -I /usr/local/cuda-11.4/include -Wl,–start-group -lnvinfer -lnvparsers -Wl,–end-group
In file included from /usr/include/aarch64-linux-gnu/NvInferRuntimeCommon.h:19,
from /usr/include/aarch64-linux-gnu/NvInferLegacyDims.h:16,
from /usr/include/aarch64-linux-gnu/NvInfer.h:16,
from /usr/include/aarch64-linux-gnu/NvCaffeParser.h:16,
from …/…/includes/nvdsinfer_custom_impl.h:126,
from nvdsinfer_custombboxparser.cpp:25:
/usr/local/cuda-11.4/include/cuda_runtime_api.h:147:10: fatal error: crt/host_defines.h: No such file or directory
147 | #include “crt/host_defines.h”
| ^~~~~~~~~~~~~~~~~~~~
compilation terminated.
In file included from /usr/include/aarch64-linux-gnu/NvInferRuntimeCommon.h:19,
from /usr/include/aarch64-linux-gnu/NvInferLegacyDims.h:16,
from /usr/include/aarch64-linux-gnu/NvInfer.h:16,
from /usr/include/aarch64-linux-gnu/NvCaffeParser.h:16,
from …/…/includes/nvdsinfer_custom_impl.h:126,
from nvdsinfer_customclassifierparser.cpp:25:
/usr/local/cuda-11.4/include/cuda_runtime_api.h:147:10: fatal error: crt/host_defines.h: No such file or directory
147 | #include “crt/host_defines.h”
| ^~~~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:41: libnvds_infercustomparser.so] Error 1

Any idea?

1 Like

I was able to run the model using the provided tutorial on x86 architecture but when trying to run it in jetson docker, am facing the same issue. Seems like the C files are not available in the docker.

Please let me know if you were able to find a workaround.

Thank you

Jetson docker does not have the libraries to build the static library. You will need to build it on the host by installing deepstream directly on host.

Thank you so much for your kind response.

BTW I was able to run it on Jetson Docker.