Deepstream Smart Record Python

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) dGPU 3070 ti laptop
• DeepStream Version 7.1
• JetPack Version (valid for Jetson only)
• TensorRT Version 10.12
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

I followed the steps for compiling python bindings with smart record enabled from DeepStream SDK FAQ - #41 by fanzh[nvurisrcbin plugin to perform smart recording in]
I then tried to replicate the test1 pipeline , but it starts the recording but never stops it> I am attaching my code and output.

Code:

nvurisrcbin = None

def stop_record_callback():
    global nvurisrcbin
    nvurisrcbin.emit('stop-sr', 0)

# keep same as native for get size of native struct
class SRUserContext(ctypes.Structure):
    _fields_ = [
        ("sessionid", ctypes.c_int),
        ("name", ctypes.c_char * 32)
    ]
# GstElement * src, NvDsSRRecordingInfo* recordingInfo, void* data, void* user_data
def record_done(nvurisrcbin, recordingInfo, user_ctx, user_data):
    print('******sr done*****')
    print(f"record_done nvurisrcbin {nvurisrcbin} recordingInfo {recordingInfo}"
          f" user_ctx {user_ctx} user_data {user_data}")
    info = pyds.NvDsSRRecordingInfo.cast(hash(recordingInfo))
    sr = pyds.SRUserContext.cast(hash(user_ctx))
    print(f"filename {info.filename} -- dir {info.dirpath} {info.width} x {info.height}")
    print(f"session id {sr.sessionid} -- name {sr.name}")
global nvurisrcbin
    nvurisrcbin = Gst.ElementFactory.make("nvurisrcbin", "uri-decode-bin")
    if not nvurisrcbin:
        sys.stderr.write(" Unable to create nvurisrcbin \n")

    nvurisrcbin.set_property("smart-record", 2) # enable smart record.
    nvurisrcbin.set_property("smart-rec-dir-path", "./") # set record path.


    # Create nvstreammux instance to form batches from one or more sources.
    streammux = Gst.ElementFactory.make("nvstreammux", "Stream-muxer")
    if not streammux:
        sys.stderr.write(" Unable to create NvStreamMux \n")

    nvurisrcbin.set_property("uri",uri)
    # Connect to the "pad-added" signal of the decodebin which generates a
    # callback once a new pad for raw data has beed created by the decodebin
    nvurisrcbin.connect("pad-added", cb_newpad, streammux)
    nvurisrcbin.connect("sr-done", record_done, pipeline)
     pipeline.set_state(Gst.State.PLAYING)
    
    sessionid = pyds.get_native_ptr(pyds.alloc_buffer(4))
    print(f"sessionid {sessionid}")

    sr_user_context_size = ctypes.sizeof(SRUserContext)
    sr_user_context_buf = pyds.get_native_ptr(pyds.alloc_buffer(sr_user_context_size))
    sr_user_context = pyds.SRUserContext.cast(sr_user_context_buf)
    sr_user_context.sessionid = 42
    sr_user_context.name = "sr-demo"
    print(f"sr_user_context_buf {sr_user_context_buf}")

    # signal_info = GObject.signal_query('start-sr', nvurisrcbin)
    # print(f"signal_info {signal_info.param_types}")

    # stop smart record manually, optional
    # GLib.timeout_add(5000, stop_record_callback)
    def start_sr_function():
        nvurisrcbin.emit('start-sr', sessionid, 2, 5, sr_user_context_buf)
        print('******start sr*****')
        return True
    timer_id = GLib.timeout_add_seconds(10, start_sr_function)
    try:
         loop.run()
    except:
         pass
     # cleanup
    pipeline.set_state(Gst.State.NULL)
    # release native memory
    pyds.free_gbuffer(sessionid)
    pyds.free_gbuffer(sr_user_context_buf)


Output:

❯ uv run test
Creating Pipeline

Is it Integrated GPU? : 0
Creating EGLSink

Playing file{clip}
Adding elements to Pipeline

Linking elements in the Pipeline

Starting pipeline

0:00:00.274042625 479665     0x210a9210 INFO                 nvinfer gstnvinfer.cpp:684:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:2092> [UID = 1]: deserialized trt engine from :/home/jagpreet/weapon_detection/model/resnet18_trafficcamnet_pruned.onnx_b1_gpu0_int8.engine
Implicit layer support has been deprecated
INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:327 [Implicit Engine Info]: layers num: 0

0:00:00.274084395 479665     0x210a9210 INFO                 nvinfer gstnvinfer.cpp:684:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2195> [UID = 1]: Use deserialized engine model: /home/jagpreet/weapon_detection/model/resnet18_trafficcamnet_pruned.onnx_b1_gpu0_int8.engine
0:00:00.281561824 479665     0x210a9210 INFO                 nvinfer gstnvinfer_impl.cpp:343:notifyLoadModelStatus:<primary-inference> [UID 1]: Load new model:test1_pgie.txt sucessfully
sessionid <capsule object NULL at 0x7a66477fd680>
sr_user_context_buf <capsule object NULL at 0x7a66477fd6b0>
Failed to query video capabilities: Invalid argument
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7a66477cd610 (GstCapsFeatures at 0x7a65a4351050)>
******start sr*****
******start sr*****
******start sr*****
******start sr*****
******start sr*****
nvstreammux: Successfully handled EOS for source_id=0
End-of-stream

out.txt (1.7 KB)
test.txt (12.4 KB)

In your code, “stop-sr” signal is not sent. you can send “stop-sr” to stop the recording, Or the recording will stop automatically after the recording duration is bigger than the SMART_REC_DEFAULT_DURATION. Please refer to the C version sample deepstream-testsr.

I tried adding it like this:

    def start_sr_function():
        nvurisrcbin.emit('start-sr', sessionid, 1, 5, sr_user_context_buf)
        print('******start sr*****')

        GLib.timeout_add_seconds(15, stop_record_callback)
        return False

Also in output I get

sessionid <capsule object NULL at 0x7772d6bf94d0>
sr_user_context_buf <capsule object NULL at 0x7772d6bf9620>
Failed to query video capabilities: Invalid argument
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7772d6bc9670 (GstCapsFeatures at 0x777228350bd0)>
******start sr*****

are session_id and sr_user_context_buf supposed to be NULL??

  1. No, get_native_ptr is opensource. it only prints the “native_ptr”.
  2. Please add log in stop_record_callback. do you mean “sr done” is not printed after stop_record_callback is called? if so, was the recording generated? is the same issue with ths topic?

I added log to the stop callback:

def stop_record_callback():
    global nvurisrcbin
    nvurisrcbin.emit('stop-sr', 0)
    print("Sr stop cb called")

Output:

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x78215c351670 (GstCapsFeatures at 0x7820ac350f70)>
******start sr*****
******start sr*****
Sr stop cb called
******start sr*****
Sr stop cb called
******start sr*****
Sr stop cb called
nvstreammux: Successfully handled EOS for source_id=0
End-of-stream

No recording is generated. and record_done is not called either.
I also ran gst-launch-1.0 nvurisrcbin uri=file:///xxx.mp4 ! fakesink
from the linked topic, output:

Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
Redistribute latency...
New clock: GstSystemClock
Redistribute latency...
Got EOS from element "pipeline0".
Execution ended after 0:00:02.038409674
Setting pipeline to NULL ...
Freeing pipeline ...

I am also attaching my bindfuctions.cpp and utils.cpp
bindfunc.txt (35.9 KB)
utils.txt (9.1 KB)

I made the changes you made in Use nvurisrcbin plugin to do smart record in Python

But I also saw in another post “cpp patch (gpointer_bind.dff )” which adds alloc_buffer1 on bindfunctions.cpp and functiondoc.h. Do I need to apply that patch as well? and use allocbuffer1 instead of allocbuffer?

  1. As the doc shown, “Sources must be of type source-type-rtsp” , meaning nvurisrcbin smart recording only support RTSP source.
  2. Please only refer to the FAQ Use nvurisrcbin plugin to do smart record in Python. some topics are too old.

Thank you so much, that solved the smart record problem.
However I get a warning when running with the rtsp:

There may be a timestamping problem, or this computer is too slow.
Warning: gst-core-error-quark: A lot of buffers are being dropped. (13): ../libs/gst/base/gstbasesink.c(3147): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstEglGlesSink:nvvideo-renderer:
There may be a timestamping problem, or this computer is too slow.
******start sr*****
** WARN: <queue_src_pad_probe:276>: Invalid PTS found in stream. Stream recorded by Smart Record might have issues
Warning: gst-core-error-quark: A lot of buffers are being dropped. (13): ../libs/gst/base/gstbasesink.c(3147): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstEglGlesSink:nvvideo-renderer:
There may be a timestamping problem, or this computer is too slow.
Warning: gst-core-error-quark: A lot of buffers are being dropped. (13): ../libs/gst/base/gstbasesink.c(3147): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstEglGlesSink:nvvideo-renderer:

But recording works fine

if you are using eglsink, please set the sync property to false, meaning playing as fast as possible. Or, you can use fakesink instead.

that solved it,

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x75d2a19c15b0 (GstCapsFeatures at 0x75d194060b40)>
Failed to query video capabilities: Invalid argument
******start sr*****
** WARN: <queue_src_pad_probe:276>: Invalid PTS found in stream. Stream recorded by Smart Record might have issues
Failed to query video capabilities: Invalid argument
******sr done*****
record_done nvurisrcbin <__gi__.GstDsNvUriSrcBin object at 0x75d2a1933780 (GstDsNvUriSrcBin at 0x10584600)> recordingInfo <gpointer at 0x129544870718400> user_ctx <gpointer at 0x316998448> user_data <Gst.Pipeline object at 0x75d2a19e9180 (GstPipeline at 0x10579cc0)>
filename 129540978380160 -- dir 297277760 2560 x 1920
session id 42 -- name sr-demo
******start sr*****
** ERROR: <RunUserCallback:208>: No video stream found
******start sr*****
** ERROR: <RunUserCallback:208>: No video stream found
******start sr*****

One Last Question Can I trigger the sr start using a probe after nvinfer?? and What if Want to save the video with overlayed bboxes in case of using a detection model or label when using a classification model?

yes, you can send the ‘start’ signal in probe function. if you want to record the video with bboxes, you need to do encoding after nvosd plugin. please refer to the ready-made sample deepstream-testsr in DeepStream SDK. nvurisrcbin only supports recording the RTSP stream.