How can I set nvstreammux's options for multiple rtsp sources sinking?

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)
GPU
• DeepStream Version
6.0.1
• JetPack Version (valid for Jetson only)
• TensorRT Version
8.2.1.8
• NVIDIA GPU Driver Version (valid for GPU only)
470.57.02
• Issue Type( questions, new requirements, bugs)
questions
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

gsttestplugin.py

from datetime import datetime

import gi
gi.require_version("Gst", "1.0")
gi.require_version("GstBase", "1.0")
from gi.repository import GObject, Gst, GstBase

import pyds

class GstTestPlugin(GstBase.BaseTransform):
    GST_PLUGIN_NAME = "gsttestplugin"

    __gstmetadata__ = (
        "TestPlugin",
        "Plugin",
        "Test Plugin",
        "Test",
    )

    __gsttemplates__ = (
        Gst.PadTemplate.new(
            "src",
            Gst.PadDirection.SRC,
            Gst.PadPresence.ALWAYS,
            Gst.Caps.new_any(),
        ),
        Gst.PadTemplate.new(
            "sink",
            Gst.PadDirection.SINK,
            Gst.PadPresence.ALWAYS,
            Gst.Caps.new_any(),
        ),
    )

    def __init__(self):
        super(GstTestPlugin, self).__init__()
        self.test = "sample"

    def do_transform_ip(self, buffer: Gst.Buffer) -> Gst.FlowReturn:
        print("do_transform_ip", datetime.now())

        batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(buffer))
        l_frame = batch_meta.frame_meta_list
        while l_frame is not None:
            try:
                frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
            except StopIteration:
                break

            print(frame_meta.batch_id, datetime.fromtimestamp(frame_meta.ntp_timestamp / 1e9), frame_meta.frame_num, frame_meta.source_id)

            l_frame = l_frame.next

        return Gst.FlowReturn.OK

GObject.type_register(GstTestPlugin)
__gstelementfactory__ = (
    GstTestPlugin.GST_PLUGIN_NAME,
    Gst.Rank.NONE,
    GstTestPlugin,
)

gst pipeline

gst-launch-1.0 \
    uridecodebin uri=rtsp://172.25.168.232/h264 ! nvvideoconvert nvbuf-memory-type=3 ! "video/x-raw(memory:NVMM),format=RGBA" ! m.sink_0 \
    uridecodebin uri=rtsp://172.25.168.229/h264 ! nvvideoconvert nvbuf-memory-type=3 ! "video/x-raw(memory:NVMM),format=RGBA" ! m.sink_1 \
    uridecodebin uri=rtsp://172.25.168.224/h264 ! nvvideoconvert nvbuf-memory-type=3 ! "video/x-raw(memory:NVMM),format=RGBA" ! m.sink_2 \
    uridecodebin uri=rtsp://172.25.168.226/h264 ! nvvideoconvert nvbuf-memory-type=3 ! "video/x-raw(memory:NVMM),format=RGBA" ! m.sink_3 \
    uridecodebin uri=rtsp://172.25.168.228/h264 ! nvvideoconvert nvbuf-memory-type=3 ! "video/x-raw(memory:NVMM),format=RGBA" ! m.sink_4 \
    uridecodebin uri=rtsp://172.25.168.230/h264 ! nvvideoconvert nvbuf-memory-type=3 ! "video/x-raw(memory:NVMM),format=RGBA" ! m.sink_5 \
    uridecodebin uri=rtsp://172.25.168.231/h264 ! nvvideoconvert nvbuf-memory-type=3 ! "video/x-raw(memory:NVMM),format=RGBA" ! m.sink_6 \
    uridecodebin uri=rtsp://172.25.168.227/h264 ! nvvideoconvert nvbuf-memory-type=3 ! "video/x-raw(memory:NVMM),format=RGBA" ! m.sink_7 \
    nvstreammux name=m nvbuf-memory-type=3 width=1920 height=1080 batch-size=8 sync-inputs=1 live-source=1 batched-push-timeout=100000 ! \ 
    gsttestplugin ! fakesink sync=0

output log

do_transform_ip 2021-12-21 09:02:33.521643
0 2021-12-21 09:02:33.521614 43 2
1 2021-12-21 09:02:33.521622 43 4
do_transform_ip 2021-12-21 09:02:33.600635
0 2021-12-21 09:02:33.600557 43 0
1 2021-12-21 09:02:33.600572 43 3
2 2021-12-21 09:02:33.600576 43 5
3 2021-12-21 09:02:33.600581 42 6
do_transform_ip 2021-12-21 09:02:33.601007
0 2021-12-21 09:02:33.600986 44 1
1 2021-12-21 09:02:33.600993 44 7
do_transform_ip 2021-12-21 09:02:33.603333
0 2021-12-21 09:02:33.603269 44 2
1 2021-12-21 09:02:33.603281 44 4
do_transform_ip 2021-12-21 09:02:33.647970
0 2021-12-21 09:02:33.647897 44 0
1 2021-12-21 09:02:33.647910 44 3
2 2021-12-21 09:02:33.647915 44 5
3 2021-12-21 09:02:33.647919 43 6
do_transform_ip 2021-12-21 09:02:33.695859
0 2021-12-21 09:02:33.695786 45 1
1 2021-12-21 09:02:33.695802 45 7
do_transform_ip 2021-12-21 09:02:33.763235
0 2021-12-21 09:02:33.763182 45 2
1 2021-12-21 09:02:33.763193 45 4
do_transform_ip 2021-12-21 09:02:33.763512
0 2021-12-21 09:02:33.763485 45 0
1 2021-12-21 09:02:33.763491 45 3
2 2021-12-21 09:02:33.763494 45 5
3 2021-12-21 09:02:33.763498 44 6

I made plugin using gst-python and deepstream-python and tested multiple RTSP sources. (8 RTSP: 1920x1080 10fps, same ntp server)

I want to sync 8 RTSP sources, but it did not work.
(0, 3, 5, 6 source_id → synced
1, 7 source_id → synced
2, 4 source_id → synced)

In documentation, there are many option for nvstreammux, so I tried many time but it did not work.
(sync-inputs, live-source, batched-push-timeout, max-latency)

If I want to sync all rtsp sources well, how can I do?

3

Do you mean you want all sources has the same ntp timestamp?

4

Yes. I want all sources has same timestamp.

Even if the fps drops little bit, It’s ok. (eg 10 fps, if source has about 30fps)
Or, even if the precision of the synchronization time is slightly lowered? (eg 50ms ~ 100ms)

Is there anything I can do?
Thanks.

5

No. There is no such feature. NTP Timestamp in DeepStream — DeepStream 6.3 Release documentation

6

Thank you for reply.
How does the nvstreammux’s sync-inputs option work?
And why only some sources has same ntp timestamp?

7

The “sync-inputs” flag enables sync multiple input according to PTS to generate batch. It will not change any timestamp.

NTP Timestamp in DeepStream — DeepStream 6.0 Release documentation (nvidia.com)

9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.