How to trigger Smart Record based on detection of a particular Object

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) dGPU T4
• DeepStream Version 5.0
• JetPack Version (valid for Jetson only)
• TensorRT Version TRT7.0
• NVIDIA GPU Driver Version (valid for GPU only) 440.60

How to trigger Smart Record based on detection of a particular Object? A sample code would be helpful.

If you have a look at the deepstream-test5 sample application you’ll see how smart record is used.
In your own app you can have a probe somewhere after your nvinfer element. I typically add the probe to the tracker src pad.
In this probe you have access to all the metadata and can check if detections have been made and for what classes etc. in here you can call NvDsSRStart().

2 Likes

Thanks Jason. I must say you have done tremendous work on deepstream. Thanks.

1 Like

@jasonpgf2a hi, i have a same question on this. I’m trying to trigger a smart record on bbox_generated_probe_after_analytics probe, have you ever try on this? Would you mind to provide some way to access the bin within the probe?

@jasonpgf2a I have been struggling to get the Smart Record implemented. Any pointers are highly appreciated.

@Mike I have been able to trigger the smart record within the bbox_generated_probe_after_analytics probe. However, It seems like need to find out a way to trigger the start and hold the until record duration is end before trigger the NvDsSRStop.
Maybe we can find a way together…

static void
bbox_generated_probe_after_analytics (AppCtx * appCtx, GstBuffer * buf,
    NvDsBatchMeta * batch_meta, guint index)
{
  NvDsObjectMeta *obj_meta = NULL;
  NvDsSRSessionId sessId = 0;
  NvDsSrcParentBin *bin = &appCtx->pipeline.multi_src_bin;
  NvDsSrcBin *src_bin = &bin->sub_bins[index];
  guint startTime = 7;
  guint duration = 8;

  //g_print("subbins_rec:%d",src_bin->config->smart_rec_duartion);
  g_print("index:%d\n",index);
  if (src_bin->config->smart_rec_duration >= 0)
    duration = src_bin->config->smart_rec_duration;
    g_print("smart_rec_duartion:%d\n",duration);

  if (src_bin->config->smart_rec_start_time >= 0)
    startTime = src_bin->config->smart_rec_start_time;
    g_print("smart_rec_time:%d\n",startTime);
  if (src_bin->recordCtx && !src_bin->reconfiguring) {
    NvDsSRContext *ctx = (NvDsSRContext *) src_bin->recordCtx;
    if (ctx->recordOn) {
      NvDsSRStop (ctx, 0);
    } else {
      NvDsSRStart (ctx, &sessId, startTime, duration, NULL);
    }
  }

within the deepstream_source_bin.c it use the g_timeout_add to trigger the smart record_event_generator from every period of record duration from the settings. Need to find out how the g_timeout_add can be use based on detection of particular object.
I think one easiest way is to store the event source id to the appCTX which return by g_timeout_add , and stop the record after the particular object is not detected and frame out.

Hi,@cheesiangleow5284,have you solve this problem? I want to know if you just put the smart record here it worked or should i do some encode work? I am a beginner so i really confused about the smart record example.

Hi @shuiniu I just trigger NvDsSRStart in bbox_generated_probe_after_analytics when iterating the NvDsObjectMeta *obj_meta is specific object.

@cheesiangleow5284 Thank you for your reply.I will try it.And i want to know did it work well to get video?

1 Like

@shuiniu I’m not fully understand your work well, if you are asking about “Does the video recorded well” , well then I will say yes!

@cheesiangleow5284 are you working well with trigger in bbox_generated_probe_after_analytics, if oke, can you help me do that

thanks for you reply @cheesiangleow5284.
I just use your sample and change like below and want it record object 0,but it doesn’t work,what can i change?

static void
all_bbox_generated(AppCtx * appCtx, GstBuffer * buf,
NvDsBatchMeta * batch_meta, guint index)
{
NvDsSRSessionId sessId = 0;
NvDsSrcParentBin *bin = &appCtx->pipeline.multi_src_bin;
NvDsSrcBin *src_bin = &bin->sub_bins[index];
guint startTime = 4;
guint duration = 10;

for (NvDsMetaList * l_frame = batch_meta->frame_meta_list; l_frame != NULL;
 l_frame = l_frame->next)
{
   NvDsFrameMeta *frame_meta = (NvDsFrameMeta *) (l_frame->data);
   for (NvDsMetaList *l_obj = frame_meta->obj_meta_list; l_obj != NULL;
        l_obj = l_obj->next) {
       NvDsObjectMeta *obj_meta = (NvDsObjectMeta *) (l_obj->data);
       if (obj_meta->class_id == 0) {
          if (src_bin->recordCtx && !src_bin->reconfiguring) {
            NvDsSRContext *ctx = (NvDsSRContext *) src_bin->recordCtx;
            if (ctx->recordOn) {
                NvDsSRStop (ctx, 0);
             } else {
            NvDsSRStart (ctx, &sessId, startTime, duration, NULL);//replace NUll with src_bin but still doesn't work
             }
          }
         }
       }
}

}

//put it at create_pipeline in deepstream-app-main.c

create_pipeline (appCtx[i], NULL,
all_bbox_generated, perf_cb, overlay_graphics)

I’m getting below error when I call smart record in bbox_generated_post_analytics, any idea how to fix it?

ERROR: <RunUserCallback:199>: No video stream found

I’m calling it like

static void
bbox_generated_post_analytics (AppCtx * appCtx, GstBuffer * buf,
    NvDsBatchMeta * batch_meta, guint index)
{
  // smart record
  NvDsSrcParentBin *bin = &appCtx->pipeline.multi_src_bin;
  NvDsSrcBin *src_bin = &bin->sub_bins[frame_meta->source_id];
  g_timeout_add (10000, smartRecord, src_bin);

...
}

static gboolean
smartRecord (NvDsSrcBin *src_bin)
{
  NvDsSRSessionId sessId = 0;
  guint startTime = 7;
  guint duration = 8;

  if (src_bin->config->smart_rec_duration >= 0)
    duration = src_bin->config->smart_rec_duration;

  if (src_bin->config->smart_rec_start_time >= 0)
    startTime = src_bin->config->smart_rec_start_time;

  if (src_bin->recordCtx && !src_bin->reconfiguring) {
    NvDsSRContext *ctx = (NvDsSRContext *) src_bin->recordCtx;
    if (ctx->recordOn) {
      NvDsSRStop (ctx, 0);
    } else {
      NvDsSRStart (ctx, &sessId, startTime, duration, NULL);
    }
  }
  return TRUE;
}

Update: Solved it. turns out g_timeout_add (10000, smartRecord, src_bin) isn’t working, it’s calling smartRecord function everytime, which inadvertently stops the recording as ctx->recordOn fluctatues b/w 0 and 1 and no video is thus saved. For now, I’ve commented out NvDsSRStop(ctx, 0).

1 Like