Bug Report: nvmultiurisrcbin/nvurisrcbin/nvstreammux with HLS Streaming Issues

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)
GPU
• DeepStream Version
7.1
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)
566.03
• Issue Type( questions, new requirements, bugs)
bugs
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)

Environment

  • Using GStreamer with NVIDIA DeepStream plugins
  • Pipeline involves nvmultiurisrcbin/nvurisrcbin/nvstreammux with HLS streams

Issue Description

When using nvmultiurisrcbin/nvurisrcbin/nvstreammux with HLS streams, timing and playback issues occur, including pipeline freezing, depending on the configuration of live-source, max-batch-size and sync parameters.

Working Configuration without nvmultiurisrcbin/nvurisrcbin

The following pipeline works correctly:

gst-launch-1.0 souphttpsrc location=<redacted> ! hlsdemux ! decodebin ! videoconvert ! autovideosink

Note: the test URL(m3u8) can be shared privately with NVIDIA support team.

Problem Scenarios

Failed Pipeline Example

gst-launch-1.0 nvmultiurisrcbin \
port=9000 ip-address=localhost \
batched-push-timeout=33333 max-batch-size=2 \
drop-pipeline-eos=1 live-source=1 \
uri-list=<redacted> width=720 height=480 \
! nvmultistreamtiler width=720 height=480 ! nveglglessink sync=0

Working but Restricted Pipeline


gst-launch-1.0 nvmultiurisrcbin \
port=9000 ip-address=localhost \
batched-push-timeout=33333 max-batch-size=1 \
drop-pipeline-eos=1 live-source=0 \
uri-list=<redacted> width=720 height=480 \
! nvmultistreamtiler width=720 height=480 ! nveglglessink sync=1

Scenario 1: live-source=1 with max-batch-size >= num_source

  • Configuration: nvmultiurisrcbin live-source=1 max-batch-size=3 nveglglessink sync=0 source=1

  • Result: PTS is not respected

  • Issue: Video plays ~1.5x faster and freezes at the next segment

Scenario 2: live-source=1 with max-batch-size = num_source

  • Configuration: nvmultiurisrcbin live-source=1 max-batch-size=1 nveglglessink sync=0 source=1

  • Result: PTS is not respected

  • Issue: Video plays ~3.5x faster and freezes at the next segment

Scenario 3: live-source=0 with max-batch-size > num_source

  • Configuration: nvmultiurisrcbin live-source=0 max-batch-size=5 nveglglessink sync=1 source=1

  • Result: Similar timing issues

  • Issue: Video plays ~1.5x faster and freezes

Working but Limited Scenario

  • Configuration: live-source=0 with max-batch-size=num_source and nveglglessink sync=1

  • Result: Works correctly

  • Limitation: According to documentation, we need live-source=1 for dynamic pipelines (adding/removing sources on demand)

Documentation Reference

The documentation states that live-source=1 is required for dynamic pipelines:

“live-source need to be turned ON to allow batching of available buffers when number of sources is < max-batch-size configuration.”

Impact

  • This bug prevents the proper use of nvmultiurisrcbin with HLS streams in dynamic pipeline scenarios where sources need to be added or removed on demand.

  • Users are forced to use sync=1 in the sink because live-source=0 causes PTS (Presentation Time Stamps) to be disregarded, resulting in incorrect playback timing

  • This creates a conflicting situation where:

  • Using live-source=0 requires sync=1 to maintain proper timing

  • But dynamic pipelines require live-source=1 according to documentation

  • When using live-source=1, the PTS timing issues occur regardless of sync settings

Expected Behavior

The pipeline should maintain correct playback speed and timing when using live-source=1 max-batch-size > num_sources and sync=0 with HLS streams, allowing for dynamic source management while respecting PTS timing.

2

Same as HLS framerate changed after nvstreammux

4
  1. For sink elements, sync=1 is necessary, otherwise the pipeline will run as quickly as possible, thus ignoring synchronization.
    2.I tested the following pipeline and it works fine. I set max-batch-size=10 and live-source=1 (run about 13 hours)
gst-launch-1.0 nvmultiurisrcbin port=9000 ip-address=localhost batched-push-timeout=33333 max-batch-size=10 drop-pipeline-eos=1 live-source=1 uri-list=xxxxxx.m3u8 width=720 height=480 ! nvmultistreamtiler width=720 height=480 ! nveglglessink

3.Set batch-size to 1 and let add/remove APIs update this logic cause pipeline slowdown and hence the maxBatchSize usage.This is a known issue，Use new streammux to avoid this issue

USE_NEW_NVSTREAMMUX=yes gst-launch-1.0 nvmultiurisrcbin port=9000 ip-address=localhost batched-push-timeout=33333 max-batch-size=1 drop-pipeline-eos=1 live-source=1 uri-list=http://10.19.226.168:8080/live/livestream.m3u8 width=720 height=480 ! nvmultistreamtiler width=720 height=480 ! nveglglessink
5

Thank you for your time.
I wanted to share that the following pipeline command was successful in my testing:

USE_NEW_NVSTREAMMUX=yes gst-launch-1.0 nvmultiurisrcbin port=9000 ip-address=localhost batched-push-timeout=33333 max-batch-size=1 drop-pipeline-eos=1 live-source=1 uri-list=livestream.m3u8 width=720 height=480 ! nvmultistreamtiler width=720 height=480 ! nveglglessink

This configuration worked successfully while other variations I tried were problematic. I will be implementing this in my development tomorrow and will provide further updates. I’ve shared some additional test URLs privately for further testing.

6

@junshengy I’m observing issues with HTTP live streaming:

“Redistributed latency…” warning and playback freezes occur on both VLC and GStreamer when using HTTP streams. This suggests a network latency issue rather than a player-specific problem. (Ignored)

Moreover, looks like when latency occurs in a single source, it affects all sources in the pipeline, causing cumulative delays across all sources over time. This cascading effect suggests possible resource contention or synchronization issues in the pipeline handling.

Are there any configurations available to prevent this cross-source latency propagation and maintain independent stream performance?

7

Updates…
When using YouTube HLS streams, the issue doesn’t seem to occur.
However, when using my own streams, the problem manifests itself. I will investigate internally and return with more information.

Thank you for your time.

8

Actually, I saw a similar situation, this may require reconnection.

You can refer to this FAQ, set the timeout, and then reconnect.

9

I’m having trouble tracking down an issue while debugging. Looking at gst-nvurisrcbin , I noticed the RTSP implementation is well-built while HLS relies on uridecodebin. I made some improvements to better handle HLS streams, like using uridecodebin3 instead of uridecodebin. decodebin3 comes with enhanced HLS support, though I’m still evaluating its impact on my pipeline.

If the issue lies between nvurisrcbin and nvstreammux, visibility is limited since nvstreammux is closed source. It’s concerning that the entire pipeline freezes due to issues with a single stream.

I’ll check timeout feature.

Please let me know if you discover anything new about this issue.

10

I’ve tested individual streams without issues - although they sometimes freeze for a few seconds, the pipeline self-corrects. However, when I combine streams with different latency characteristics in the same pipeline (some with latency, others without), the pipeline starts experiencing issues. In a pipeline with 12 streams where 6 freeze randomly, from the user’s perspective, the pipeline appears constantly frozen because each freeze lasts about 3 seconds.

I believe the main issue here is that latency from one stream affects the entire pipeline. It seems the freezing occurs in the streammux, which waits for the frozen stream to recover. While I can’t confirm this definitively, this is my observation of the behavior.

The core problem appears to be that individual stream latency impacts the whole pipeline, with the streammux waiting for frozen streams to recover before proceeding. This is my current understanding of the situation.

I’m having difficulty understanding exactly which component causes the entire pipeline to freeze when latency occurs. Is it the streammux that’s blocking the pipeline, or is there another component responsible for this behavior? Any insights would be appreciated.

11

The new nvstreammux is open source, you can refer to this

/opt/nvidia/deepstream/deepstream/sources/gst-plugins/gst-nvmultistream2

nvstreammux should not cause freezing. I tested nvmultiurisrcbin and added a file url and a rtsp camera. Then I disconnected the rtsp camera from the network and the pipeline did not get stuck.

1 Like