Please provide complete information as applicable to your setup.
• Hardware Platform ( GPU) • DeepStream Version : 7• JetPack Version (valid for Jetson only) • TensorRT Version • NVIDIA GPU Driver Version (valid for GPU only) : 535.183.01• Issue Type( questions]) • How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing) • Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)
I have a deepstream pipeline, I want to add a delay in one of the tee branches.
gst-launch-1.0 -e filesrc location=/opt/nvidia/deepstream/deepstream-7.0/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! m.sink_0 nvstreammux batch-size=1 width=1280 height=720 name=m ! nvstreamdemux name=d d.src_0 ! tee name=t ! queue name=delay_queue ! autovideosink sync=true t. ! autovideosink sync=false
delay queue should introduce 10 seconds of delay, other part can run its own pace.
This is actually a continuation of
• x86 Machine with dGPU
• DeepStream Version : deepstream-6.3
• TensorRT Version: 8.6.1.
• NVIDIA GPU Driver Version (valid for GPU only) : Driver Version: 535.86.10 CUDA Version: 12.2
• Issue Type: questions
I have a deepstream pipeline as followiing.
export USE_NEW_NVSTREAMMUX=yes
gst-launch-1.0 filesrc location=/opt/nvidia/deepstream/deepstream-6.3/samples/streams/sample_1080p_h264.mp4 ! decodebin ! nvvideoconvert ! videorate ! "video/x-raw,framerate=25/1" ! nvvideoconvert ! "video/…
But that was on new streammux, I want it in old streammux.
Please help
Can you tell us why nvstreammux instead of the new nvstreammux is needed? Any special function with nvstreammux is needed?
we use rtspclientsink plugin, after demux, reccovering rtsp streams with new streammux is when sink pad in playing state is not working in our side.
What kind of problem did you meet with the new nvstreammux? “drop-pipeline-eos” is supported by both nvstreammux.
We have deepstream-test5 sample for dynamically add/remove sources. Can you develop your app based on deepstream-test5?
I have successfully introduced a controllable delay using a queue element with the following pipeline:
gst-launch-1.0 -e filesrc location=/opt/nvidia/deepstream/deepstream-7.0/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder num-extra-surfaces=16 ! queue ! tee name=t ! queue ! autovideosink t. ! m.sink_0 nvstreammux batch-size=1 width=1280 height=720 name=m ! nvstreamdemux name=d d.src_0 ! nvvideoconvert ! videoconvert ! capsfilter ! video/x-raw,format=RGBA ! queue max-size-buffers=60 max-size-time=2000000000 max-size-bytes= 440401920 min-threshold-time=2000000000 min-threshold-buffers=50 name=delay_queue ! autovideosink sync=false
The delay works as intended, but I noticed that the queue element results in memory copies between the CPU and GPU. This introduces overhead and affects the system’s performance.
My goals are:
Achieve the delay entirely within nvmm (NVIDIA’s memory management system) buffers to avoid unnecessary memory copies.
Simplify the pipeline while maintaining high performance.
Questions:
Is there a way to configure queue or another element to perform buffering directly in nvmm?
Are there alternative approaches or elements in GStreamer/DeepStream that can help implement this functionality more efficiently?
Any guidance, example pipelines, or suggestions would be greatly appreciated. Thank you!
ajithkumar.ak95:
I have successfully introduced a controllable delay using a queue element with the following pipeline:
gst-launch-1.0 -e filesrc location=/opt/nvidia/deepstream/deepstream-7.0/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder num-extra-surfaces=16 ! queue ! tee name=t ! queu
For your pipeline uses “tee” to render the video frames to different sink branches, the video frames are shared between the branches, I noticed that the buffering queue is added in the inferencing branch, the “nvvideoconvert” and “videoconvert” will generate new GstBuffer which will return the original frame buffers from video decoder. It is OK to use in this way. The so-called delay is implemented by the video frame buffering by “queue”, the more delay, the more frame buffers are buffered in the queue, so the memory usage will increase.
The “NVMM” frame buffers are shared between DeepStream plugins, the nvv4l2decoder use bufferpool to manage and recycle the NVMM buffers. The default buffer pool size is 16.
If you want to use “queue” to buffering NVMM buffers, you may remove videoconvert and put the queue after nvvideoconvert. The hardware buffer consumption may increase if you use the queue to delay buffers by the buffering method.
There is no such element. You may develop a new plugin which copy the GstBuffer to a new GstBuffer with delayed timestamp.
How to do it, thats the question, it doesnt work without converting to RGBA Host. Kindly help or give new pipeline.
Ajithkumar A K
This pipeline works in my device
gst-launch-1.0 -e filesrc location=/opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder num-extra-surfaces=16 ! queue ! tee name=t ! queue ! nvvideoconvert ! autovideosink t. ! m.sink_0 nvstreammux batch-size=1 width=1280 height=720 name=m ! nvstreamdemux name=d d.src_0 ! nvvideoconvert ! queue max-size-buffers=60 max-size-time=2000000000 max-size-bytes= 440401920 min-threshold-time=2000000000 min-threshold-buffers=90 name=delay_queue ! autovideosink sync=0
For me,
I am using deepstream-7.0
These are the logs
root@ajith-OMEN-by-HP-Laptop-16-c0xxx:/opt/nvidia/deepstream/deepstream-7.0/sources/src# GST_DEBUG=3 gst-launch-1.0 -e filesrc location=/opt/nvidia/
deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder num-extra-surfaces=16 ! queue ! tee name=t ! queue
! nvvideoconvert ! autovideosink t. ! m.sink_0 nvstreammux batch-size=1 width=1280 height=720 name=m ! nvstreamdemux name=d d.src_0 ! nvvideoconve
rt ! queue max-size-buffers=60 max-size-time=2000000000 max-size-bytes= 440401920 min-threshold-time=2000000000 min-threshold-buffers=90 name=dela
y_queue ! autovideosink sync=0
Setting pipeline to PAUSED ...
0:00:00.983678800 85874 0x5703874e23b0 WARN kmssink gstkmssink.c:850:gst_kms_sink_start:<autovideosink1-actual-sink-kms> error: Could not open DRM module (NULL)
0:00:00.983715609 85874 0x5703874e23b0 WARN kmssink gstkmssink.c:850:gst_kms_sink_start:<autovideosink1-actual-sink-kms> error: reason: No such file or directory (2)
0:00:00.983782655 85874 0x5703874e23b0 WARN basesink gstbasesink.c:5878:gst_base_sink_change_state:<autovideosink1-actual-sink-kms> error: Failed to start
libEGL warning: DRI2: could not open /dev/dri/card1 (Operation not permitted)
0:00:01.994043064 85874 0x5703874e23b0 WARN kmssink gstkmssink.c:850:gst_kms_sink_start:<autovideosink0-actual-sink-kms> error: Could not open DRM module (NULL)
0:00:01.994062030 85874 0x5703874e23b0 WARN kmssink gstkmssink.c:850:gst_kms_sink_start:<autovideosink0-actual-sink-kms> error: reason: No such file or directory (2)
0:00:01.994093359 85874 0x5703874e23b0 WARN basesink gstbasesink.c:5878:gst_base_sink_change_state:<autovideosink0-actual-sink-kms> error: Failed to start
0:00:02.073397855 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:3100:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:02.073419465 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:2985:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe minimum capture size for pixelformat MJPG
0:00:02.073425116 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:3100:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:02.073429795 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:2991:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe maximum capture size for pixelformat MJPG
0:00:02.073444492 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:3100:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:02.073448851 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:2985:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe minimum capture size for pixelformat AV10
0:00:02.073456425 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:3100:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:02.073460663 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:2991:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe maximum capture size for pixelformat AV10
0:00:02.073478045 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:3100:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:02.073485519 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:2985:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe minimum capture size for pixelformat DVX5
0:00:02.073492452 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:3100:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:02.073497452 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:2991:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe maximum capture size for pixelformat DVX5
0:00:02.073504605 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:3100:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:02.073508863 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:2985:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe minimum capture size for pixelformat DVX4
0:00:02.073512891 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:3100:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:02.073518181 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:2991:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe maximum capture size for pixelformat DVX4
0:00:02.073525845 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:3100:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:02.073530163 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:2985:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe minimum capture size for pixelformat MPG4
0:00:02.073533600 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:3100:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:02.073537657 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:2991:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe maximum capture size for pixelformat MPG4
0:00:02.073547656 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:3100:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:02.073552124 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:2985:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe minimum capture size for pixelformat MPG2
0:00:02.073555972 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:3100:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:02.073559899 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:2991:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe maximum capture size for pixelformat MPG2
0:00:02.073569046 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:3100:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:02.073573344 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:2985:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe minimum capture size for pixelformat H265
0:00:02.073578674 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:3100:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:02.073582972 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:2991:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe maximum capture size for pixelformat H265
0:00:02.073589655 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:3100:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:02.073593783 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:2985:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe minimum capture size for pixelformat VP90
0:00:02.073597870 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:3100:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:02.073601968 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:2991:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe maximum capture size for pixelformat VP90
0:00:02.073607749 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:3100:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:02.073612027 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:2985:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe minimum capture size for pixelformat VP80
0:00:02.073616265 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:3100:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:02.073620252 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:2991:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe maximum capture size for pixelformat VP80
0:00:02.073626674 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:3100:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:02.073631063 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:2985:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe minimum capture size for pixelformat H264
0:00:02.073635020 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:3100:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:02.073639248 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:2991:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe maximum capture size for pixelformat H264
0:00:02.073862507 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:3100:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:src> Unable to try format: Unknown error -1
0:00:02.073868118 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:2985:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:src> Could not probe minimum capture size for pixelformat Y444
0:00:02.073872196 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:3100:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:src> Unable to try format: Unknown error -1
0:00:02.073876444 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:2991:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:src> Could not probe maximum capture size for pixelformat Y444
0:00:02.073884859 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:3100:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:src> Unable to try format: Unknown error -1
0:00:02.073889177 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:2985:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:src> Could not probe minimum capture size for pixelformat P410
0:00:02.073893215 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:3100:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:src> Unable to try format: Unknown error -1
0:00:02.073897443 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:2991:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:src> Could not probe maximum capture size for pixelformat P410
0:00:02.073904166 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:3100:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:src> Unable to try format: Unknown error -1
0:00:02.073908454 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:2985:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:src> Could not probe minimum capture size for pixelformat PM10
0:00:02.073912401 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:3100:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:src> Unable to try format: Unknown error -1
0:00:02.073916529 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:2991:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:src> Could not probe maximum capture size for pixelformat PM10
0:00:02.073922580 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:3100:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:src> Unable to try format: Unknown error -1
0:00:02.073926187 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:2985:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:src> Could not probe minimum capture size for pixelformat NM12
0:00:02.073930024 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:3100:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:src> Unable to try format: Unknown error -1
0:00:02.073934503 85874 0x5703874e23b0 WARN v4l2 gstv4l2object.c:2991:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:src> Could not probe maximum capture size for pixelformat NM12
0:00:02.074798485 85874 0x5703874e23b0 WARN basesrc gstbasesrc.c:3688:gst_base_src_start_complete:<filesrc0> pad not activated yet
Pipeline is PREROLLING ...
0:00:02.074987861 85874 0x5703874e23b0 WARN structure gststructure.c:2091:priv_gst_structure_append_to_gstring: No value transform to serialize field 'display' of type 'GstEGLDisplay'
Got context from element 'autovideosink1-actual-sink-nveglgles': gst.egl.EGLDisplay=context, display=(GstEGLDisplay)NULL;
0:00:02.075051420 85874 0x7034dc0013e0 WARN qtdemux qtdemux.c:3121:qtdemux_parse_trex:<qtdemux0> failed to find fragment defaults for stream 1
0:00:02.075139746 85874 0x7034dc0013e0 WARN qtdemux qtdemux.c:3121:qtdemux_parse_trex:<qtdemux0> failed to find fragment defaults for stream 2
0:00:02.367876737 85874 0x7034dc0013e0 WARN v4l2videodec gstv4l2videodec.c:2311:gst_v4l2_video_dec_decide_allocation:<nvv4l2decoder0> Duration invalid, not setting latency
0:00:02.370223324 85874 0x7034dc0013e0 WARN v4l2bufferpool gstv4l2bufferpool.c:1116:gst_v4l2_buffer_pool_start:<nvv4l2decoder0:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:00:02.370806068 85874 0x7034dc001980 WARN v4l2bufferpool gstv4l2bufferpool.c:1567:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:src> Driver should never set v4l2_buffer.field to ANY
Redistribute latency...0.0 %)
^Chandling interrupt. (0.0 %)
Interrupt: Stopping pipeline ...
EOS on shutdown enabled -- Forcing EOS on the pipeline
Waiting for EOS...
^C00:00.0 / 0:00:48.1 (0.0 %)
Please upgrade to DeepStream 7.1 version.
