Deepstream 5.0 bug in gst-dsexample, plus Smart Record empty field

This message is for the Nvidia dev team. First, thanks for all the work on JetPack 4.4 and Deepstream 5.0!

Wanted to report a few bugs I’ve found so far:

  1. Seg fault in the gst-dsexample plug-in if you’re using full-frame=1 and scaling the processing width and height to a different size than the output of streammux. In source file (/opt/nvidia/deepstream/deepstream-5.0/sources/gst-plugins/gst-dsexample/gstdsexample_optimized.cpp), issue:

a) in scale_and_fill_data() line 600, the call to NvBufSurfaceMemSet() causes the seg fault. After debugging why, I noticed that an incompletely full batch results in the dsexample->batch_insurf.numFilled var to become out of sync and eventually references an index out of bounds (cause: it’s not being reset to 0 after submitting an incomplete batch for processing).

b) The fix is in gst_dsexample_submit_input_buffer(), the block starting at line 918 should NOT be inside that else block. It also needs to apply to the “(dsexample->process_full_frame)” case, so the fix is to move it out so it applies to both if/else cases.

/* Submit a non-full batch. */
    if (batch) {
      if (!convert_batch_and_push_to_process_thread (dsexample, batch.get ())) {
        return GST_FLOW_ERROR;
      batch.release ();
      dsexample->batch_insurf.numFilled = 0;

c) Also, I also noticed a lack of indexing on lines 808 and 809:

rect_params.width = in_surf->surfaceList->width;
rect_params.height = in_surf->surfaceList->height;

Since this is inside a loop, it should probably be:

rect_params.width = in_surf->surfaceList[i].width;
rect_params.height = in_surf->surfaceList[i].height;
  1. Smart Record is awesome. Thanks for adding it. It’s working great for me from two different RTSP sources, and it’s a clever idea to save the H264 stream directly to disk without needing to decode it first (probably buying my Jetson Nano some more fps!). A couple of issues I’ve found:

a) For the callback function (initParams.callback) when the recording ends, I’m finding that the duration field in NvDsSRRecordingInfo is always 0 when my callback function is called. So I’m tracking it myself as a workaround, but you should probably fix that.

b) It’s kinda painful to get to the SR context object (NvDsSRContext) for each source in the pipeline. I had to do some gymnastics to pass NvDsSrcParentBin references for each RTSP source into my custom DS plug-in at the time I’m setting up the GST pipeline. I pass a pointer to them as a gstreamer property of type gpointer. It works, it’s just annoying. I couldn’t find a more efficient way to work through GST or NvBatchMeta to get at all the sources and their SR context objects, but maybe I missed something. So I’m basically doing this on the receiving end in the DS plug-in I wrote:

NvDsSRContext *recordCtx = (NvDsSRContext *)plugin_ctx->initParams.multi_src_bin->sub_bins[frame_meta->source_id].recordCtx;

where plugin_ctx is my DS plugin’s context struct, and I added a field called ‘multi_src_bin’ to it (type NvDsSrcParentBin *). This was the easiest thing I found to pass my DS plug-in after configuring the GST pipeline in the main deepstream-app (reference app).

All that said, why not just attach a link to NvDsSRContext* inside of NvDsFrameMeta? You’ve already got source_id, source_frame_width, and other source-related fields there. Would shortcut a lot of pain! :)

My environment:

  • Jetson Nano
  • DS5.0
  • JP4.4
  • Other versions consistent with JP4.4

Would you mind to separate it to two topics.

It’s not letting me edit this post anymore. But I’ll post the Smart Record item as a separate topic now.

We have noted both the issues and engineering team will look into it and provide fixes as necessary. Thanks for reporting these.