Pgie is chaotic when add videorate to deepstream-app

[hieuhv]

Because i don’t know fps of input source, so i want to set fps of input source is constant. I modified deepstream-app to add videorate element for each source (code is below for reproducing):

bin->video_rate = gst_element_factory_make (“videorate”, “video_rate”);
if (!bin->video_rate) {
NVGSTDS_ERR_MSG_V (“Could not create element ‘videorate’”);
goto done;
}
caps = //gst_caps_new_empty_simple (“video/x-raw”);
gst_caps_new_simple (“video/x-raw”,
“format”, G_TYPE_STRING, “RGBA”,
“framerate”, GST_TYPE_FRACTION, 10 , 1,
NULL);
feature = gst_caps_features_new (“memory:NVMM”, NULL);
gst_caps_set_features (caps, 0, feature);
…
NVGSTDS_LINK_ELEMENT (bin->nvvidconv, bin->video_rate);
NVGSTDS_LINK_ELEMENT (bin->video_rate, bin->cap_filter1);

However, pgie is chaotic after that. I think order of frame after pgie is wrong. It’s ok if pgie is disable. Can you help me?

• Hardware Platform (Jetson / GPU) jetson nano
• DeepStream Version 6.0
• JetPack Version (valid for Jetson only)
• TensorRT Version 8.0.3.1
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

[fanzh]

some questions:
1 I didn’t find bin->nvvidconv in 6.0 deepstream-app code, is that the whole code modification?
2 why do you want to set fps of input source is constant? are you testing one stream or multiple streams? “batched-push-timeout” of Gst-nvstreammux can cover the case of the inconstant fps.
3 what do you mean about “pgie is chaotic”?

[hieuhv]

1. I modified create_uridecode_src_bin function in source/apps/apps-common/src/deepstream_source_bin.c
2. I tested with multistream. Output of pgie is delayed because inference time of pgie is slow. If output not delay, i have to use “drop-frame-interval”, but I can’t know fps of input exactly so i can’t fix value for “drop-frame-interval”. Therefore, I need videorate element to fix fps input (here is 10).
3. Bboxs and objects are mismatch. I will send to you video output.

[fanzh]

1 ok , please let me see the video output.
2 what is your video source type? please provide the whole code diff, config file.

[hieuhv]

I added video output and config for deepstream-app
Patch file for deepstream_app:
a.patch (2.7 KB)

config.txt (4.4 KB)

[fanzh]

diff.patch (1019 Bytes)
please try this patch and your config.

[hieuhv]

It’s ok, thank for your help!
Can you explain why the order of pipeline is videorate->nvvidconv?