Store incoming video in chunks in files so those files can be associated with the metadata

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) GPU
• DeepStream Version 6.2
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only) 525.105.17
• Issue Type( questions, new requirements, bugs) Question
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

I need to store chunks of incoming video to file with the start (and end?) frameid, used in the metadata, as part of the filename(s). If frameid is not possible (or not optimal), then potentially a timestamp (as in the test5 smart-record sample) could be useful. I’d prefer not to use NvDsMultiSrcInput/NvDsSampleC2DSmartRecordTrigger/NvDsMsgBrokerC2DReceiver (as in test5), because I need to store all video and I don’t want to send start/stop messages continuously.

I’m trying to build a graph using Deepstream graph composer, which saves the incoming video stream to disk in separate files containing 10s of video duration each (duration of 10s is not crucial, could be a different duration). I’ve tried using the following set of components:

  1. nvidia::deepstream::NvDsMultiSrcInputWithRecord (configured with smart-rec-default-duration=10), associated to
  2. nvidia::deepstream::NvDsRecordAction
  3. nvidia::gxf::PeriodicSchedulingTerm (configured with recess_period=10s)

The idea is to trigger the NvDsRecordAction every 10 seconds, and store the video output with a duration of 10 seconds.

I could not find any documentation on how to configure and get working the components NvDsRecordAction and PeriodicSchedulingTerm. The Deepstream SDK documentation in the links below only describe the components’ individual configuration settings, and not how to connect or make these components work together in a graph.
https://docs.nvidia.com/metropolis/deepstream/dev-guide/graphtools-docs/docs/text/ExtensionsManual/StandardExtension.html#nvidia-gxf-periodicschedulingterm
https://docs.nvidia.com/metropolis/deepstream/dev-guide/graphtools-docs/docs/text/GXF_Development.html#periodicschedulingterm

Using graph composer, it is possible to drag PeriodicSchedulingTerm on the graph canvas, but I couldn’t find a way to connect it to NvDsRecordAction. The graph’s .yaml then contains:

---
components:
- name: periodic_scheduling_term5
  parameters:
    recess_period: 10s
  type: nvidia::gxf::PeriodicSchedulingTerm
name: PeriodicSchedulingTerm
ui_property:
  position:
    x: -1360.8486328125
    y: 106.59441375732422

By coincidence, I also noticed that it is possible to drag PeriodicSchedulingTerm ‘into’ NvDsRecordAction, so that it becomes part of that component (as visually represented in graph composer). The graph’s .yaml then contains:

---
components:
- name: Record Action0
  type: nvidia::deepstream::NvDsRecordAction
- name: periodic_scheduling_term0
  parameters:
    recess_period: 10s
  type: nvidia::gxf::PeriodicSchedulingTerm
name: Record Action
ui_property:
  position:
    x: -1060.3350830078125
    y: 156.1263885498047

Questions:

  1. What is the best way to achieve contiguous chunks of all incoming video stored in files with filenames containing the frameid (or timestamps), so that frame metadata (containing frameid and timestamp), stored separately, can be associated again to these files?
  2. Where can I find documentation/guidance on how to use the nvidia::gxf components in a deepstream composer graph, including PeriodicSchedulingTerm?

Thanks in advance!

The smart recodering file name change is not supported with any of current Graph Composer extensions.
The best way to achieve your goal is to use c/c++ DeepStream APIs, you can refer to the deepstream-test5 source code and you must implement your own smart_record_callback() in /opt/nvidia/deepstream/deepstream/sources/apps/apps-common/src/deepstream_source_bin.c.
The callback API is in NVIDIA DeepStream SDK API Reference: NvDsSRInitParams Struct Reference | NVIDIA Docs
and NVIDIA DeepStream SDK API Reference: Smart Record | NVIDIA Docs

Hi Fiona,

Thanks for your reply, there are some useful pointers in there (although not all questions are answered).

I’m currently running into a few problems:

  1. Building libnvdsgst_deepstream_bins.so (‘contains’ compiled deepstream_source_bin.c) :
    gst-inspect-1.0 nvurisrcbin indicates that it is part of:
    /usr/lib/x86_64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_deepstream_bins.so

I couldn’t find any make files that could help out how to build this .so (or a copied/altered version of it). How can this be built?

  1. Building libnvdsgst_nvmultiurisrcbin.so
    There seem to be certain missing header files:
/opt/nvidia/deepstream/deepstream-6.2/sources/gst-plugins/gst-nvmultiurisrcbin# CUDA_VER=11.8 make
-fPIC -DDS_VERSION="6.2.0" -I../../includes -I/usr/local/cuda-11.8/include -pthread -I/usr/include/gstreamer-1.0 -I/usr/include/orc-0.4 -I/usr/include/gstreamer-1.0 -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include
g++ -c -o gstdsnvmultiurisrcbin.o -fPIC -DDS_VERSION=\"6.2.0\" -I../../includes -I/usr/local/cuda-11.8/include  -pthread -I/usr/include/gstreamer-1.0 -I/usr/include/orc-0.4 -I/usr/include/gstreamer-1.0 -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include gstdsnvmultiurisrcbin.cpp
In file included from gstdsnvmultiurisrcbin.cpp:26:
gstdsnvmultiurisrcbin.h:30:10: fatal error: gstdsnvurisrcbin.h: No such file or directory
   30 | #include "gstdsnvurisrcbin.h"
      |          ^~~~~~~~~~~~~~~~~~~~
compilation terminated.

This file doesn’t seem to be present on the deepstream-devel container:

/opt/nvidia/deepstream/deepstream-6.2/sources/gst-plugins/gst-nvmultiurisrcbin# find / -iname gstdsnvurisrcbin.h
/opt/nvidia/deepstream/deepstream-6.2/sources/gst-plugins/gst-nvmultiurisrcbin# find / -iname "*urisrcbin.h"
/opt/nvidia/deepstream/deepstream-6.2/sources/gst-plugins/gst-nvmultiurisrcbin/gstdsnvmultiurisrcbin.h
  1. Missing configuration settings?

From deepstream_source_bin.c, lines 1068 - 1076 (function create_rtsp_src_bin), it seems to me that it should be possible to enable ‘automatic’ recording (using a timer via g_timeout_add) by the urisrcbin by specifying rtsp (smart-record == 2), and configuring the smart-rec-interval, smart-rec-duration and smart-rec-start-time, which are used in functions smart_record_event_generator and NvDsSRStart:

  // Enable local start / stop events in addition to the one
  // received from the server.
  if (config->smart_record == 2) {
    if (bin->config->smart_rec_interval)
      g_timeout_add (bin->config->smart_rec_interval * 1000,
          smart_record_event_generator, bin);
    else
      g_timeout_add (10000, smart_record_event_generator, bin);
  }

However, NvDsSingleSrcInput does not support a few configuration options, specifically, smart-rec-duration, smart-rec-interval, and smart-rec-start-time are missing. I’ve textually added those to the .yaml file of the graph (using NvDsSingleSrcInput), but unfortunately this generates the warnings below, which confirms that these settings aren’t used:

2023-06-08 14:54:48.794 WARN  gxf/std/yaml_file_loader.cpp@957: Using unregistered parameter 'smart-rec-duration' in component 'Single Source Input0'.
2023-06-08 14:54:48.794 WARN  gxf/std/yaml_file_loader.cpp@957: Using unregistered parameter 'smart-rec-interval' in component 'Single Source Input0'.
2023-06-08 14:54:48.794 WARN  gxf/std/yaml_file_loader.cpp@957: Using unregistered parameter 'smart-rec-start-time' in component 'Single Source Input0'.

These options are also missing from gstdsnvmultiurisrcbin.cpp. I think this could be fixed by adding these configuration settings to gstdsnvmultiurisrcbin.cpp and rebuilding libnvdsgst_nvmultiurisrcbin.so (and changing NvDsSingleSrcInput to NvDsMultiSrcInput), but I’m missing the required files to do so (see 2. above).

  1. Missing sources?

The log of my graph shows this:

0:00:00.109275711 4360 0x55d2d5cf9cd0 DEBUG nvurisrcbin gstdsnvurisrcbin.cpp:1817:populate_rtsp_bin:<Single Source Input/Single Source Input0> Decode bin created. Waiting for a new pad from decodebin to link

However, gstdsnvurisrcbin.cpp is not present on the deepstream-devel container (most likely related to 2. above).

Any pointers will be much appreciated, thanks in advance!

This is Nvidia proprietary component, it is not open source. After the DeepStream SDK is installed correctly, the binary file is available.

It is a bug. You can put this header file into the build directory.
gstdsnvurisrcbin.h (3.3 KB)

Please refer to /opt/nvidia/deepstream/deepstream/reference_graphs/deepstream-test5 for the usage.

Seems your graph is not correct. Please refer to /opt/nvidia/deepstream/deepstream/reference_graphs/deepstream-test5 for the usage.

Thank you for your answers and the provided file.

After adding the missing header file, it was possible to build libnvdsgst_nvmultiurisrcbin.so. However, in order to implement our requirements, it would be more helpful to have the sources (and make files) for NvDsSingleSrcInput. Could these be made available?

The sources for gst-nvmultiurisrcbin unfortunately don’t include the code related to the smart recording. It could be possible to get something working if there is a way to retrieve the instance (or list of instances) of GstDsNvUriSrcBin (i.e. from GstDsNvMultiUriBin), but I did not see how that would be possible. Any hints to that will be appreciated.

Thanks!

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

No. Currently, it is Nvidia proprietary component, it is not open source.

You can use the smart recording interfaces directly instead of by using GstDsNvUriSrcBin or GstDsNvMultiUriBin. Smart Video Record — DeepStream 6.2 Release documentation

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.