Add a smart record to an existing pipeline

[me.kerem]

Please provide complete information as applicable to your setup.

**• Hardware Platform: **Jetson
**• DeepStream Version: ** 6.3
• JetPack Version: 5.1.2
**• TensorRT Version: ** 8.5.2
**• NVIDIA GPU Driver Version: ** R35

I have pipeline that built with nvmultiurisrcbin plugin for example:
nvmultiurisrcbin name=nvmultiurisrcbin smart-rec-default-duration=60 smart-record=1 smart-rec-dir-path='path' attach-sys-ts=false ! nvvidconv ! video/x-raw, format=(string)BGRx ! fakesink

I need to add a smart record for this, I already add message consumer with KAFKA:
gboolean parse_err = !parse_msgconsumer_yaml (&message_consumer_config, "message-consumer0", std::getenv("CONFIG_FILE_PATH"));

and use it:
GstElement* nvmultiurisrcbin = gst_bin_get_by_name(GST_BIN(pipeline), "nvmultiurisrcbin"); c2d_ctx = start_cloud_to_device_messaging (&message_consumer_config, NULL, nvmultiurisrcbin);

I create the smart record:

` NvDsSRInitParams params = {0};
NvDsSRContainerType ContainerType;
params.containerType = NVDSSR_CONTAINER_MP4;
params.dirpath = “path”;
params.cacheSize = 30;
params.defaultDuration = 300;
params.callback = smart_record_callback;

NvDsSrcParentBin *pBin = (NvDsSrcParentBin *) c2d_ctx->uData;
  if (!pBin) {
    NVGSTDS_WARN_MSG_V ("Null user data");
    return 1;
  }

  NvDsSRSessionId sessId = 0;
  gint sensorId;
  NvDsSRContext *srCtx = NULL;

if (NvDsSRCreate (&srCtx, &params) != NVDSSR_STATUS_OK) {
  NVGSTDS_ERR_MSG_V ("Failed to create smart record bin");
  g_free (params.fileNamePrefix);
  return 1;
}`

In this case, how do I link srCtx->recordbin to the pipeline?
Among the options I try are to unlink part of the pipeline and link the srCtx->recordbin in the middle.
Mp4 videos are created, but they contain only 616 bytes or are empty.

The MP4 file contains the following content:
ftypqt qt freemdatDmoovlmvhd�T �T�a



@
�trak\tkhda�T �T�


@�mdia mdhd�T �T�U�!hdlr�minf!hdlrdhlralis$dinfdref
alis
\stblstsdsttsstscstszstco=udta5meta!hdlrmhlrmdirilst=udta5meta!hdlrmhlrmdirilst

Thanks
Menahem

[fanzh]

please refer to the doc nvmultiurisrcbin and nvurisrcbin. you don’t need to create recordbin because the sub element nvurisrcbin already includes recordbin. please refer to this topic for how to use nvurisrcbin smartrecord.

if using message consumer, pleease refer to start_cloud_to_device_messaging in opt\nvidia\deepstream\deepstream-7.0\sources\apps\sample_apps\deepstream-app\deepstream_app.c, you can start/stop record in subscribe_cb.

[me.kerem]

@fanzh

if using message consumer, pleease refer to start_cloud_to_device_messaging in opt\nvidia\deepstream\deepstream-7.0\sources\apps\sample_apps\deepstream-app\deepstream_app.c, you can start/stop record in subscribe_cb.

It was confusing for me to know what parameter I needed to send to start_cloud_to_device_messaging.
This is the call to start_cloud_to_device_messaging in deepstream_app.c:
start_cloud_to_device_messaging (&config->message_consumer_config[i], NULL, **&appCtx->pipeline.multi_src_bin**);

pipeline.multi_src_bin is NvDsSrcParentBin struct that contains GstElement nvmultiurisrcbin

struct NvDsSrcParentBin
{
GstElement *bin;
GstElement *streammux;
GstElement *nvmultiurisrcbin;
GThread *reset_thread;
NvDsSrcBin sub_bins[MAX_SOURCE_BINS];
guint num_bins;
guint num_fr_on;
gboolean live_source;
gulong nvstreammux_eosmonitor_probe;
};

If I only have nvmultiurisrcbin itself, how can I access this?

[fanzh]

the first parameter NvDsMsgConsumerConfig is from parse_msgconsumer_yaml, the second is NULL. the third parameter uData will be a parameter of subscribe_cb. deepstream-app is openousrce. you can add log to check.

[me.kerem]

@fanzh

the third parameter uData will be a parameter of subscribe_cb

from subscribe_cb:

srCtx = (NvDsSRContext *) pBin->sub_bins[sensorId].recordCtx; if (!srCtx) { NVGSTDS_WARN_MSG_V ("Null SR context handle."); goto error; }

My understanding is that in the third parameter (uData), I need to send an element that contains a list of sub-bins (NvDsSrcBin). However, if I only have nvmultiurisrcbin, I am not sure how I can get this object with this list.

When I try to use: start_cloud_to_device_messaging (&message_consumer_config, NULL, nvmultiurisrcbin);

The start-recording command returns an error “Null SR context handle.” because this code:

srCtx = (NvDsSRContext *) pBin->sub_bins[sensorId].recordCtx; if (!srCtx) { NVGSTDS_WARN_MSG_V ("Null SR context handle."); goto error; }

[fanzh]

please refer to the topic in my first comment. after finding the nvurisrcbin, which is wrapped by nvmultiurisrcbin, you can send start-sr to start recording and send stop-sr to stop recording .

[me.kerem]

If nvmultiurisrcbin wraps nvurisrcbin, how do I retrieve it?

I try gst_bin_get_by_name(GST_BIN(nvmultiurisrcbin), "nvurisrcbin");

and gst_bin_iterate_elements(GST_BIN(nvmultiurisrcbin))

However, I have not been successful.

As far as I can tell, there is no example of a direct relation in the source code of nvmultiurisrcbin

[fanzh]

please refer to set_nvuribin_conv_prop in opt\nvidia\deepstream\deepstream\sources\libs\gstnvdscustomhelper\gst-nvmultiurisrcbincreator.cpp for how to get nvurisrcbin.

[me.kerem]

In case anyone is interested, I change the massage_cb to:

#include "gstdsnvurisrcbin.h"

NvDsSrcParentBin multi_src_bin;
NvDsC2DContext *c2d_ctx;

static gboolean
message_cb (GstBus * bus, GstMessage * message, gpointer user_data)
{
  switch (GST_MESSAGE_TYPE (message)) {
    case GST_MESSAGE_ERROR:{
      GError *err = NULL;
      gchar *name, *debug = NULL;

      name = gst_object_get_path_string (message->src);
      gst_message_parse_error (message, &err, &debug);

      g_printerr ("ERROR: from element %s: %s\n", name, err->message);
      if (debug != NULL)
        g_printerr ("Additional debug info:\n%s\n", debug);

      g_error_free (err);
      g_free (debug);
      g_free (name);

      g_main_loop_quit (loop);
      break;
    }
    case GST_MESSAGE_WARNING:{
      GError *err = NULL;
      gchar *name, *debug = NULL;

      name = gst_object_get_path_string (message->src);
      gst_message_parse_warning (message, &err, &debug);

      g_printerr ("ERROR: from element %s: %s\n", name, err->message);
      if (debug != NULL)
        g_printerr ("Additional debug info:\n%s\n", debug);

      g_error_free (err);
      g_free (debug);
      g_free (name);
      break;
    }
    case GST_MESSAGE_EOS:
      g_print ("Got EOS\n");
      if (c2d_ctx){
        stop_cloud_to_device_messaging (c2d_ctx);
      }

      g_main_loop_quit (loop);
      gst_element_set_state (pipeline, GST_STATE_NULL);
      g_main_loop_unref (loop);
      gst_object_unref (pipeline);
      break;
    case GST_MESSAGE_STATE_CHANGED: 
      GstState old_state, new_state, pending_state;
      gst_message_parse_state_changed(message, &old_state, &new_state, &pending_state);

      if (new_state == GST_STATE_PLAYING) {
        if (strncmp(GST_OBJECT_NAME (message->src) ,"dsnvurisrcbin", strlen("dsnvurisrcbin")) == 0) {
            g_print("dsnvurisrcbin play");
            std::string objectName(GST_OBJECT_NAME (message->src));
            int number = std::stoi(objectName.substr(strlen("dsnvurisrcbin")));
            GstDsNvUriSrcBin *nvUriSrcBin = (GstDsNvUriSrcBin *)GST_ELEMENT(message->src);
            multi_src_bin.sub_bins[number].recordCtx = nvUriSrcBin->recordCtx;


            if (c2d_ctx == NULL) {
              c2d_ctx = start_cloud_to_device_messaging (&message_consumer_config, NULL, &multi_src_bin);
              if (c2d_ctx == NULL) {
                g_print ("Failed to create message consumer");
              }
            }
        }
      }

      break;
    default:
      break;
  }

  return TRUE;
}

And it is work.