Get wrong infer results while testing yolov4 on deepstream 5.0

@pinktree3

Follow this guidline:
YoloV4: DarkNet or Pytorch -> ONNX -> TensorRT -> DeepStream

1 Like

Hi @ersheng
I followed this link [https://github.com/Tianxiaomo/pytorch-YOLOv4]
I generated the ONNX from the Darknet,
After that, I go to the Nvidia tensorrt container, and the execute the command :

trtexec --onnx=yolov4_1_3_608_608.onnx --explicitBatch --saveEngine=yolov4_1_3_608_608_fp16.engine --workspace=4096 --fp16

I get the following error at the end:
.
.
.
[libprotobuf ERROR google/protobuf/text_format.cc:298] Error parsing text-format onnx2trt_onnx.ModelProto: 1:1: Invalid control characters encountered in text.
[libprotobuf ERROR google/protobuf/text_format.cc:298] Error parsing text-format onnx2trt_onnx.ModelProto: 1:12: Invalid control characters encountered in text.
[libprotobuf ERROR google/protobuf/text_format.cc:298] Error parsing text-format onnx2trt_onnx.ModelProto: 1:14: Message type “onnx2trt_onnx.ModelProto” has no field named “pytorch”.
Failed to parse ONNX model from fileyolov4_1_3_608_608.onnx
[07/09/2020-10:16:02] [E] [TRT] Network must have at least one output
[07/09/2020-10:16:02] [E] [TRT] Network validation failed.
[07/09/2020-10:16:02] [E] Engine creation failed
[07/09/2020-10:16:02] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec # trtexec --onnx=yolov4_1_3_608_608.onnx --explicitBatch --saveEngine=yolov4_1_3_608_608_fp16.engine --workspace=4096 --fp16

any suggestions?

@pinktree3

What versions of Pytorch and TensorRT are you using?

@ersheng
I did configure batch size of both [streammux] and [primary-gie]. I will try agine and upload the error info later. Thanks

Hi, @pinktree3
post # 12 is useful. I just refered to it.
For the error you met, you can check your version of pytorch and tensorrt.

Darknet2ONNX

Pytorch 1.4.0 for TensorRT 7.0 and higher
Pytorch 1.5.0 and 1.6.0 for TensorRT 7.1.2 and higher

ONNX2TensorRT

TensorRT version Recommended: 7.0, 7.1

thanks @ersheng @jiejing_ma

@ersheng Followed your yolov4 repo to make TRT engine for yolov5 which was built successfully. Compared the output with pytorch mode and they are both same. But when I hook it up in DeepStream I am not getting any boxes. I have uploaded the code and relevant files here. Let me know if you have any pointers!

Thanks

Hi y14uc339,

Please help to open a new topic for your issue. Thanks

@kayccc I have opened a new topic its been 2 days! Do you mind having a look Yolov5 giving unexpected outputs

Thank you for the detailed steps. I followed all your steps.
I’m using custom yolov4 model. While I try to run deepstream-app. I got following error

nvinfer gstnvinfer.cpp:596:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::initResource() <nvdsinfer_context_impl.cpp:667> [UID = 1]: Detect-postprocessor failed to init resource because dlsym failed to get func NvDsInferParseCustomYoloV4 pointer

Solved:

@y14uc339 Thank you, I find out the problem.
@ersheng config_infer_primary_yoloV4.txt I changed
parse-bbox-func-name=NvDsInferParseCustomYoloV4 —> parse-bbox-func-name=NvDsInferParseYoloV4

Then it’s works.

@karthick define the parsing box function just like the other functions are declared in the custom impl cpp file and you will see off this error. This works:

extern "C" bool NvDsInferParseYoloV4(
    std::vector<NvDsInferLayerInfo> const& outputLayersInfo,
    NvDsInferNetworkInfo const& networkInfo,
    NvDsInferParseDetectionParams const& detectionParams,
    std::vector<NvDsInferParseObjectInfo>& objectList);



static NvDsInferParseObjectInfo convertBBoxYoloV4(const float& bx, const float& by, const float& bw,
                                     const float& bh, const uint& netW, const uint& netH)
{
    NvDsInferParseObjectInfo b;
    // Restore coordinates to network input resolution
    float xCenter = bx * netW;
    float yCenter = by * netH;

    float w = bw * netW;
    float h = bh * netH;

    float x0 = xCenter - w * 0.5;
    float y0 = yCenter - h * 0.5;
    float x1 = x0 + w;
    float y1 = y0 + h;

    x0 = clamp(x0, 0, netW);
    y0 = clamp(y0, 0, netH);
    x1 = clamp(x1, 0, netW);
    y1 = clamp(y1, 0, netH);

    b.left = x0;
    b.width = clamp(x1 - x0, 0, netW);
    b.top = y0;
    b.height = clamp(y1 - y0, 0, netH);

    return b;
}

static void addBBoxProposalYoloV4(const float bx, const float by, const float bw, const float bh,
                     const uint& netW, const uint& netH, const int maxIndex,
                     const float maxProb, std::vector<NvDsInferParseObjectInfo>& binfo)
{
    NvDsInferParseObjectInfo bbi = convertBBoxYoloV4(bx, by, bw, bh, netW, netH);
    if (bbi.width < 1 || bbi.height < 1) return;

    bbi.detectionConfidence = maxProb;
    bbi.classId = maxIndex;
    binfo.push_back(bbi);
}

static std::vector<NvDsInferParseObjectInfo>
decodeYoloV4Tensor(
    const float* detections, const uint num_bboxes,
    NvDsInferParseDetectionParams const& detectionParams,
    const uint& netW, const uint& netH)
{
    std::vector<NvDsInferParseObjectInfo> binfo;

    uint bbox_location = 0;
    for (uint b = 0; b < num_bboxes; ++b)
    {
        float bx = detections[bbox_location];
        float by = detections[bbox_location + 1];
        float bw = detections[bbox_location + 2];
        float bh = detections[bbox_location + 3];

        float maxProb = 0.0f;
        int maxIndex = -1;

        uint cls_location = bbox_location + 4;
        for (uint c = 0; c < detectionParams.numClassesConfigured; ++c)
        {
            float prob = detections[cls_location + c];
            if (prob > maxProb)
            {
                maxProb = prob;
                maxIndex = c;
            }
        }

        if (maxProb > detectionParams.perClassPreclusterThreshold[maxIndex])
        {
            addBBoxProposalYoloV4(bx, by, bw, bh, netW, netH, maxIndex, maxProb, binfo);
        }

        bbox_location += 4 + detectionParams.numClassesConfigured;
    }

    return binfo;
}

extern "C"  bool NvDsInferParseYoloV4(
    std::vector<NvDsInferLayerInfo> const& outputLayersInfo,
    NvDsInferNetworkInfo const& networkInfo,
    NvDsInferParseDetectionParams const& detectionParams,
    std::vector<NvDsInferParseObjectInfo>& objectList)
{
    if (NUM_CLASSES_YOLO != detectionParams.numClassesConfigured)
    {
        std::cerr << "WARNING: Num classes mismatch. Configured:"
                  << detectionParams.numClassesConfigured
                  << ", detected by network: " << NUM_CLASSES_YOLO << std::endl;
    }

    std::vector<NvDsInferParseObjectInfo> objects;

        const NvDsInferLayerInfo &layer = outputLayersInfo[0]; // num_boxes x (4 + num_classes)

        // 2 dimensional: [num_boxes, 4 + num_classes]
        assert(layer.inferDims.numDims == 2);
        // The second dimension should be 4 + num_classes
        assert(detectionParams.numClassesConfigured == layer.inferDims.d[1] - 4);

        uint num_bboxes = layer.inferDims.d[0];

        // std::cout << "Network Info: " << networkInfo.height << "  " << networkInfo.width << std::endl;

        std::vector<NvDsInferParseObjectInfo> outObjs =
            decodeYoloV4Tensor(
                (const float*)(layer.buffer), num_bboxes, detectionParams,
                networkInfo.width, networkInfo.height);

        objects.insert(objects.end(), outObjs.begin(), outObjs.end());

    objectList = objects;

    return true;
}

/* This is a sample bounding box parsing function for the sample YoloV3 detector model */
static NvDsInferParseObjectInfo convertBBox(const float& bx, const float& by, const float& bw,
                                     const float& bh, const int& stride, const uint& netW,
                                     const uint& netH)
{
    NvDsInferParseObjectInfo b;
    // Restore coordinates to network input resolution
    float xCenter = bx * stride;
    float yCenter = by * stride;
    float x0 = xCenter - bw / 2;
    float y0 = yCenter - bh / 2;
    float x1 = x0 + bw;
    float y1 = y0 + bh;

    x0 = clamp(x0, 0, netW);
    y0 = clamp(y0, 0, netH);
    x1 = clamp(x1, 0, netW);
    y1 = clamp(y1, 0, netH);

    b.left = x0;
    b.width = clamp(x1 - x0, 0, netW);
    b.top = y0;
    b.height = clamp(y1 - y0, 0, netH);

    return b;
}

static void addBBoxProposal(const float bx, const float by, const float bw, const float bh,
                     const uint stride, const uint& netW, const uint& netH, const int maxIndex,
                     const float maxProb, std::vector<NvDsInferParseObjectInfo>& binfo)
{
    NvDsInferParseObjectInfo bbi = convertBBox(bx, by, bw, bh, stride, netW, netH);
    if (bbi.width < 1 || bbi.height < 1) return;

    bbi.detectionConfidence = maxProb;
    bbi.classId = maxIndex;
    binfo.push_back(bbi);
}

/* Check that the custom function has been defined correctly */
CHECK_CUSTOM_PARSE_FUNC_PROTOTYPE(NvDsInferParseYoloV4);

@y14uc339 Thank you. Problem in config_infer_primary_yoloV4.txt file.
parse-bbox-func-name=NvDsInferParseYoloV4

I followed this solution and worked like a charm
However, I can’t seem to correctly modify the functions to work with Yolov4 tiny
Do you have any tips on what I can do?
Thanks

@y14uc339 @pinktree3 @jiejing_ma @gaylord @hymanzhu1983

Output format on https://github.com/Tianxiaomo/pytorch-YOLOv4 is now split from a single output
[batch_size, num_boxes, 4 + num_classes]
into
[batch_size, num_boxes, 1, 4] and [batch_size, num_boxes, num_classes].

For each bounding box, [x_center, y_center, H, W] mode is changed to [x1, y1, x2, y2].

So, there are corresponding updates in objectDetector_Yolo/nvdsinfer_custom_impl_Yolo/nvdsparsebbox_Yolo.cpp.

Follow this updated guide if you want to use the latest updates of YoloV4: YoloV4 Manual

2 Likes

@ersheng So, I used pytorch-YOLOv4 repo. Created an onnx model with batch size 2:

python demo_darknet2onnx.py cfg/yolov4-tiny.cfg yolov4-tiny.weights data/dog.jpg 2

then created a tensorrt engine:

trtexec --onnx=yolov4_2_3_416_416_fp16.onnx --explicitBatch --saveEngine=yolov4_2_3_416_416_fp16.engine --workspace=2048 --fp16 

Then made changes in the deepstream-app config file as:

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=3
uri=file:/opt/nvidia/deepstream/deepstream-5.0/sources/objectDetector_YoloV4/sixth_video.mp4
uri=file:/opt/nvidia/deepstream/deepstream-5.0/sources/objectDetector_YoloV4/sixth_video.mp4

#uri=file:/opt/nvidia/deepstream/deepstream-5.0/samples/streams/sample_720p.h264
num-sources=2

[streammux]
batch-size=2

In the detector config file:

[property]
batch-size=2

The error that I am getting is:

root@eca7bf78ed85:/opt/nvidia/deepstream/deepstream-5.0/sources/objectDetector_YoloV4# deepstream-app -c deepstream_app_config_yoloV4.txt 
WARNING: ../nvdsinfer/nvdsinfer_func_utils.cpp:34 [TRT]: Current optimization profile is: 0. Please ensure there are no enqueued operations pending in this context prior to switching profiles
0:00:02.490156301   128 0x5599102016d0 INFO                 nvinfer gstnvinfer.cpp:602:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1577> [UID = 1]: deserialized trt engine from :/opt/nvidia/deepstream/deepstream-5.0/sources/objectDetector_YoloV4/yolov4_2_3_416_416_fp16.engine
INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:685 [Implicit Engine Info]: layers num: 3
0   INPUT  kFLOAT input           3x416x416       
1   OUTPUT kFLOAT boxes           2535x1x4        
2   OUTPUT kFLOAT confs           2535x80         

0:00:02.490251454   128 0x5599102016d0 WARN                 nvinfer gstnvinfer.cpp:599:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::checkBackendParams() <nvdsinfer_context_impl.cpp:1518> [UID = 1]: Backend has maxBatchSize 1 whereas 2 has been requested
0:00:02.490269099   128 0x5599102016d0 WARN                 nvinfer gstnvinfer.cpp:599:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1689> [UID = 1]: deserialized backend context :/opt/nvidia/deepstream/deepstream-5.0/sources/objectDetector_YoloV4/yolov4_2_3_416_416_fp16.engine failed to match config params, trying rebuild
0:00:02.493503318   128 0x5599102016d0 INFO                 nvinfer gstnvinfer.cpp:602:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1591> [UID = 1]: Trying to create engine from model files
ERROR: ../nvdsinfer/nvdsinfer_model_builder.cpp:934 failed to build network since there is no model file matched.
ERROR: ../nvdsinfer/nvdsinfer_model_builder.cpp:872 failed to build network.
0:00:02.493790285   128 0x5599102016d0 ERROR                nvinfer gstnvinfer.cpp:596:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1611> [UID = 1]: build engine file failed
0:00:02.493815136   128 0x5599102016d0 ERROR                nvinfer gstnvinfer.cpp:596:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1697> [UID = 1]: build backend context failed
0:00:02.493829151   128 0x5599102016d0 ERROR                nvinfer gstnvinfer.cpp:596:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::initialize() <nvdsinfer_context_impl.cpp:1024> [UID = 1]: generate backend failed, check config file settings
0:00:02.494012819   128 0x5599102016d0 WARN                 nvinfer gstnvinfer.cpp:781:gst_nvinfer_start:<primary_gie> error: Failed to create NvDsInferContext instance
0:00:02.494030430   128 0x5599102016d0 WARN                 nvinfer gstnvinfer.cpp:781:gst_nvinfer_start:<primary_gie> error: Config file path: /opt/nvidia/deepstream/deepstream-5.0/sources/objectDetector_YoloV4/config_infer_primary_yoloV4.txt, NvDsInfer Error: NVDSINFER_CONFIG_FAILED
** ERROR: <main:651>: Failed to set pipeline to PAUSED
Quitting
ERROR from primary_gie: Failed to create NvDsInferContext instance
Debug info: gstnvinfer.cpp(781): gst_nvinfer_start (): /GstPipeline:pipeline/GstBin:primary_gie_bin/GstNvInfer:primary_gie:
Config file path: /opt/nvidia/deepstream/deepstream-5.0/sources/objectDetector_YoloV4/config_infer_primary_yoloV4.txt, NvDsInfer Error: NVDSINFER_CONFIG_FAILED
App run failed

So, I want to ask how to run yolov4 with batch-size>1 and process multiple streams parallely. Thanks

I got the same error, have you found the solution?

@hymanzhu1983 No not yet!

@hymanzhu1983 @y14uc339

Problem has been reproduced.
We are looking into the issue with inference via DeepStream when explicit_batch_size > 1