DeepStream 9.1 dGPU: nvv4l2decoder never recovers, and set_state(NULL) blocks, when the first keyframe-flagged buffer has a damaged slice

• Hardware Platform (Jetson / GPU) dGPU, NVIDIA GeForce RTX 5060 Ti 16 GB
• DeepStream Version 9.1.0 (nvcr.io/nvidia/deepstream:9.1-triton-multiarch, unmodified)
• JetPack Version (valid for Jetson only) N/A
• TensorRT Version 10.16.0.72 (not involved)
• NVIDIA GPU Driver Version (valid for GPU only) 610.57.04
• Issue Type( questions, new requirements, bugs) Bug, or a question about the intended usage
• How to reproduce the issue ? Below, with a self-contained script for the unmodified container
• Requirement details N/A

Summary

On dGPU, when the first buffer that nvv4l2decoder receives without the DELTA_UNIT flag cannot be decoded (for example, a slice damaged by packet loss on an RTSP/UDP link), the decoder:

  • never outputs a frame again, even though every following access unit is intact and there is an IDR every 12 frames;
  • never forwards EOS;
  • blocks set_state(GST_STATE_NULL) forever.

Only the first non-delta buffer matters: once the decoder has initialized, damaged buffers are handled fine.

We hit this in production with RTSP cameras. The affected source can no longer be removed, and with nvmultiurisrcbin that also hangs the REST server and freezes the other cameras (a separate topic about nvmultiurisrcbin, linked in a reply below).

Reproduction (unmodified container, about 1 minute)

Save repro_decoder_hang.py (at the end of this post) in the current directory, then:

docker run --rm -it --gpus all -e NVIDIA_DRIVER_CAPABILITIES=compute,utility,video \
    -v "$PWD":/repro nvcr.io/nvidia/deepstream:9.1-triton-multiarch bash
apt-get update && apt-get install -y python3-gi
python3 /repro/repro_decoder_hang.py

The script (below) encodes a 3 s H.264 clip with NVENC (videotestsrc, GOP 12, IDR at frames 0, 12 and 24) and feeds it through the same chain nvurisrcbin builds for an RTSP H.264 source:

appsrc (avc + codec_data, as rtph264depay outputs) ! h264parse config-interval=-1 ! nvv4l2decoder ! fakesink

Each case runs in its own process. “Damaged” keeps the NAL header byte and overwrites the rest of that one slice NAL; every other access unit is intact.

clip: /tmp/tmpuqvir5ku/clip.mkv  access units=36  keyframes at [0, 12, 24]
DeepStream 9.1.0  GStreamer 1.24.2

case A  start at the IDR, intact (control)
        pushed=36  decoded=36  expected=36  end=EOS  set_state(NULL)=returned
case B  join mid-GOP, intact (control)
        pushed=35  decoded=24  expected=24 (from the next IDR)  end=EOS  set_state(NULL)=returned
case C  join mid-GOP, first P slice damaged
        pushed=35  decoded=0  expected=24 (from the next IDR)  end=no EOS after 5 s  set_state(NULL)=BLOCKED (> 5 s)
case D  start at the IDR, IDR slice damaged, SPS/PPS only in codec_data
        pushed=36  decoded=0  expected=24 (from the next IDR)  end=no EOS after 5 s  set_state(NULL)=BLOCKED (> 5 s)
  • A and B are controls. Joining in the middle of a GOP (B), the decoder skips what it cannot use and starts at the next IDR, as expected.
  • C and D differ from the controls in a single damaged slice, in the first non-delta buffer. Expected: the same as B (24 frames from the next IDR), or an error. Actual: 0 frames, no EOS, and set_state(NULL) does not return.

How such a first buffer reaches the decoder with a real camera

  • C: joining an RTSP stream in the middle of a GOP, h264parse (config-interval=-1) inserts SPS/PPS before the first P frame and outputs that buffer without DELTA_UNIT. If that P slice is damaged, the decoder hangs.
  • D: rtph264depay carries SPS/PPS only in codec_data. h264parse inserts them before the IDR, and the buffer is non-delta even when the IDR slice is unreadable.

What we see inside the decoder

With gdb, the decoder’s streaming thread sits in usleep() inside libgstnvvideo4linux2.so (handle_frame, around offset +0x310fa), in a VIDIOC_DQEVENT polling loop that waits for the resolution-change event. For an undecodable first frame that event never comes, and the loop does not check for flushing or for the PAUSED→READY transition. set_state(NULL) then blocks in gst_pad_set_active(), waiting for the stream lock held by that thread.

Workaround we use

A pad probe on the decoder sink pad drops everything until the first non-delta buffer with SPS/PPS and an IDR whose slice header is valid. It costs up to one GOP when a source starts.

Questions

  1. Is this the expected behavior of nvv4l2decoder on dGPU?
  2. Is there a property or a recommended configuration that makes it drop the frame or post an error instead of waiting forever?
  3. Could the wait loop honor flushing and state changes, so the element can at least be stopped?
repro_decoder_hang.py
#!/usr/bin/env python3
"""nvv4l2decoder (dGPU) never recovers when its first keyframe-flagged buffer is undecodable.

Pipeline under test (same chain nvurisrcbin builds for an RTSP H.264 source):

    appsrc (H.264 avc + codec_data, as rtph264depay outputs)
      ! h264parse config-interval=-1 ! nvv4l2decoder ! fakesink

Each case runs in its own process, because a hung case leaves stuck threads behind.

Cases:
  A  start at the IDR, intact                                   (control)
  B  join in the middle of a GOP, intact                        (control)
  C  join in the middle of a GOP, first P slice damaged         (bug)
  D  start at the IDR, IDR slice damaged, SPS/PPS only in codec_data   (bug)

"Damaged" simulates what packet loss / bit errors do to a slice on an RTSP/UDP link: the NAL
header is kept and the slice payload after it is overwritten. Every following access unit is
intact, and there is a clean IDR every 12 frames, so a decoder is expected to recover at the
next IDR at the latest.

Usage:
    python3 repro_decoder_hang.py [clip.mkv]

The clip is any short H.264 Matroska file with a GOP shorter than the clip. Without an
argument, one is generated with NVENC:
    videotestsrc pattern=ball num-buffers=36 ! 1280x720 12 fps ! nvvideoconvert
      ! nvv4l2h264enc iframeinterval=12 idrinterval=12 insert-sps-pps=true profile=0
      ! h264parse ! matroskamux ! filesink
"""

import os
import struct
import subprocess
import sys
import tempfile
import threading

import gi

gi.require_version("Gst", "1.0")
from gi.repository import Gst  # noqa: E402

FPS = 12
WAIT_EOS_S = 5
WAIT_NULL_S = 5

CASES = {
    "A": "start at the IDR, intact (control)",
    "B": "join mid-GOP, intact (control)",
    "C": "join mid-GOP, first P slice damaged",
    "D": "start at the IDR, IDR slice damaged, SPS/PPS only in codec_data",
}


def generate_clip(path):
    pipeline = Gst.parse_launch(
        "videotestsrc pattern=ball num-buffers=36 "
        f"! video/x-raw,width=1280,height=720,framerate={FPS}/1 ! nvvideoconvert "
        "! nvv4l2h264enc iframeinterval=12 idrinterval=12 insert-sps-pps=true profile=0 "
        f"! h264parse ! matroskamux ! filesink location={path}")
    pipeline.set_state(Gst.State.PLAYING)
    msg = pipeline.get_bus().timed_pop_filtered(30 * Gst.SECOND, Gst.MessageType.EOS | Gst.MessageType.ERROR)
    pipeline.set_state(Gst.State.NULL)
    if msg is None or msg.type != Gst.MessageType.EOS:
        sys.exit("could not generate the clip with NVENC; pass an H.264 .mkv as argument")


def load_access_units(path):
    """Access units in avc format (4-byte lengths), with their caps (codec_data)."""
    pipeline = Gst.parse_launch(
        f"filesrc location={path} ! matroskademux "
        "! video/x-h264,stream-format=avc,alignment=au ! appsink name=sink sync=false")
    sink = pipeline.get_by_name("sink")
    pipeline.set_state(Gst.State.PLAYING)
    units, caps = [], None
    try:
        while (sample := sink.emit("try-pull-sample", Gst.SECOND)) is not None:
            caps = caps or sample.get_caps()
            buf = sample.get_buffer()
            ok, info = buf.map(Gst.MapFlags.READ)
            units.append((bytes(info.data), not buf.has_flags(Gst.BufferFlags.DELTA_UNIT)))
            buf.unmap(info)
    finally:
        pipeline.set_state(Gst.State.NULL)
    return caps, units


def split_nals(unit):
    nals, i = [], 0
    while i + 4 <= len(unit):
        size = struct.unpack(">I", unit[i:i + 4])[0]
        nals.append(unit[i + 4:i + 4 + size])
        i += 4 + size
    return nals


def join_nals(nals):
    return b"".join(struct.pack(">I", len(n)) + n for n in nals)


def nal_type(nal):
    return nal[0] & 0x1F


def damage_slice(nal):
    """Keep the NAL header byte, overwrite the slice (header and data) with a fixed pattern."""
    return nal[:1] + bytes((i * 37 + 11) & 0xFF for i in range(len(nal) - 1))


def damage_units(unit, types):
    return join_nals([damage_slice(n) if nal_type(n) in types else n for n in split_nals(unit)])


def without_parameter_sets(unit):
    return join_nals([n for n in split_nals(unit) if nal_type(n) not in (7, 8)])


def build_case(case, units):
    keyframes = [i for i, (_, key) in enumerate(units) if key]
    first_p = keyframes[0] + 1
    data = [u for u, _ in units]
    if case == "A":
        return data
    if case == "B":
        return data[first_p:]
    if case == "C":
        return [damage_units(data[first_p], {1})] + data[first_p + 1:]
    if case == "D":
        stripped = [without_parameter_sets(u) for u in data]
        return [damage_units(stripped[0], {5})] + stripped[1:]
    raise ValueError(case)


def run_case(case, clip):
    Gst.init(None)
    caps, units = load_access_units(clip)
    queue = build_case(case, units)

    pipeline = Gst.parse_launch(
        f'appsrc name=src format=time caps="{caps.to_string()}" '
        "! h264parse config-interval=-1 ! nvv4l2decoder name=dec "
        "! fakesink name=sink sync=false signal-handoffs=true")
    decoded = []
    pipeline.get_by_name("sink").connect("handoff", lambda *_: decoded.append(1))
    src = pipeline.get_by_name("src")
    pipeline.set_state(Gst.State.PLAYING)
    for i, unit in enumerate(queue):
        buf = Gst.Buffer.new_wrapped(unit)
        buf.pts = i * Gst.SECOND // FPS
        buf.duration = Gst.SECOND // FPS
        src.emit("push-buffer", buf)
    src.emit("end-of-stream")

    msg = pipeline.get_bus().timed_pop_filtered(WAIT_EOS_S * Gst.SECOND, Gst.MessageType.EOS | Gst.MessageType.ERROR)
    if msg is None:
        end = f"no EOS after {WAIT_EOS_S} s"
    elif msg.type == Gst.MessageType.EOS:
        end = "EOS"
    else:
        end = "ERROR: " + msg.parse_error()[0].message

    stopped = threading.Event()
    threading.Thread(target=lambda: (pipeline.set_state(Gst.State.NULL), stopped.set()), daemon=True).start()
    null = "returned" if stopped.wait(WAIT_NULL_S) else f"BLOCKED (> {WAIT_NULL_S} s)"

    keyframes = [i for i, (_, key) in enumerate(units) if key]
    expected = len(units) if case == "A" else f"{len(units) - keyframes[1]} (from the next IDR)"
    print(f"case {case}  {CASES[case]}\n"
          f"        pushed={len(queue)}  decoded={len(decoded)}  expected={expected}  end={end}  set_state(NULL)={null}",
          flush=True)
    os._exit(0)


def main():
    if len(sys.argv) > 2 and sys.argv[1] == "--case":
        run_case(sys.argv[2], sys.argv[3])
    Gst.init(None)
    clip = sys.argv[1] if len(sys.argv) > 1 and sys.argv[1] else None
    if clip is None:
        clip = os.path.join(tempfile.mkdtemp(), "clip.mkv")
        generate_clip(clip)
    _, units = load_access_units(clip)
    print(f"clip: {clip}  access units={len(units)}  keyframes at {[i for i, (_, k) in enumerate(units) if k]}")
    print(f"DeepStream {open('/opt/nvidia/deepstream/deepstream/version').readline().split()[-1]}  "
          f"{Gst.version_string()}\n", flush=True)
    for case in CASES:
        try:
            subprocess.run([sys.executable, __file__, "--case", case, clip], timeout=60, check=False)
        except subprocess.TimeoutExpired:
            print(f"case {case}  process did not exit in 60 s (killed)", flush=True)


if __name__ == "__main__":
    main()

Cases C and D reproduce on our side; we’re still investigating.
Note that C/D modify the stream (corrupted slice) to simulate loss. On a real RTSP source, would the hang still occur?
In theory the pipeline should not hang.

Thanks for reproducing C and D.

Yes, it happens with real RTSP sources; that is where we found it. Before we added the workaround, it happened twice with real cameras, in production and on a test bench, both reading the cameras over RTSP through MediaMTX. Each time, the source being removed had not output a single frame, stream/remove never returned, and the other cameras froze (the cascade is in the companion topic: DeepStream 9.1: one blocked stream/remove in nvmultiurisrcbin freezes all other sources (EOS probe takes creator->lock for every event)). We could not capture the decoder thread in those two cases, so we don’t know what made that first frame undecodable; the reproduction is how we found the mechanism.

What we measured:

  • The precondition is present on every connection to a real camera. On 48 connections to our cameras, the first buffer that reached nvv4l2decoder was always non-delta, with the SPS/PPS that h264parse inserts. When the connection started mid-GOP, that buffer was a P frame, which is case C. Whether a source ever starts depends on that one frame being decodable.
  • The RTSP elements don’t change the outcome. We also reproduced it end to end through nvmultiurisrcbin, with rtspsrc reading from an RTSP server that sent a clip with one damaged P slice. The decoder never output a frame, stream/remove timed out, and the other two sources dropped from 20 and 30 fps to 0. With our gate on the decoder sink pad, the removal returned and the other sources kept running. If it helps, we can turn that variant into a script for the unmodified container.

C and D modify the stream only to make the first frame undecodable at a known point. On a live link we can’t rule such a frame out, and the decoder already handles the same damage once it has started. The difference is that before the first frame, the result is a source that can never be stopped. So we agree the pipeline should not hang: dropping the frame or posting an error would be enough, and so would letting the wait loop honor flushing and state changes.

In cases C and D, the pipeline hangs on the first access unit when the coded slice in that frame is damaged (undecodable), even though later frames are fine.
Here is a workaround: repro_decoder_hang_fix.py (11.6 KB).

python3 repro_decoder_hang_fix.py --drop-damaged-slices [clip.mkv]

A pad probe on h264parse → src drops buffers that contain undecodable P/IDR slice NALs before they reach the decoder. With the flag, C/D should no longer hang (probe_dropped≥1 in the log).

Thanks for the script. It is the same kind of workaround we already run in production (the gate on the decoder sink pad described in the first post), so it does help in practice.

It cannot replace a fix in the decoder, though, for two reasons:

  • A filter in front of the decoder can lower the odds, not remove them. In the attached script, part of the detection matches the exact byte pattern that our repro writes (_is_repro_damaged_slice_payload), and the generic part only checks that first_mb, slice_type and pps_id parse within loose bounds. In our own gate we had to be stricter (an IDR slice with first_mb_in_slice == 0, I/SI slice type, its PPS and SPS in the same buffer, frame_num == 0), because the keyframe flag alone is not enough: with SPS/PPS only in codec_data, h264parse marks an unreadable IDR as non-delta. Whatever a filter misses still reaches the decoder.
  • When that happens, the element cannot be stopped. The wait loop in handle_frame (VIDIOC_DQEVENT / usleep, around +0x310fa in libgstnvvideo4linux2.so) ignores flushing and the PAUSED→READY transition, so set_state(NULL) blocks forever and the source can never be removed. With nvmultiurisrcbin that also freezes the REST server (the other topic).

So our questions from the first post still stand:

  1. Will this be tracked as an nvv4l2decoder issue?
  2. Could the decoder drop such a buffer or post an error instead of waiting forever?
  3. Could the wait loop honor flushing and state changes, so that the element can at least always be stopped?
  1. Yes, nvv4l2decoder hangs indefinitely because the lower layer fails to respond after decoding a corrupted frame.
    2&3, Since the source code for nvv4l2decoder is available in the public_sources.tbz2 package (found on the sources page at the path gst-nvvideo4linux2_src/gst-v4l2), you can modify it to implement custom functionality—such as changing an infinite wait to a finite wait. An example and the corresponding workaround code are provided here:

2&3. nvv4l2decoder code is opensource in public_sources.tbz2 of sources.. The plugin path is gst-nvvideo4linux2_src/gst-v4l2. You can modify it to implement custom functionality—such as changing an infinite wait to a finite wait. An example and the corresponding workaround code are provided here:: gstv4l2videodec.zip (20.9 KB)

Thanks, the patch fixes it in our tests.

We built gst-nvvideo4linux2 from the L4T r39.2.1 public sources for x86, inside the unmodified nvcr.io/nvidia/deepstream:9.1-triton-multiarch container (make NVDS_VERSION=9.1; the Makefile still defaults to 8.0). We built it twice, once as shipped and once with your gstv4l2videodec.c, and ran our repro on an RTX 3070 Ti, waiting up to 60 s for EOS:

case shipped binary r39 source, unmodified r39 source + your patch
C (first P slice damaged) 0 frames, no EOS in 60 s, set_state(NULL) blocked same as shipped 24/24 from the next IDR, EOS in 1.1 s, set_state(NULL) returns
D (IDR slice damaged, SPS/PPS only in codec_data) 0 frames, no EOS in 60 s, set_state(NULL) blocked same as shipped 24/24 from the next IDR, EOS in 1.1 s, set_state(NULL) returns

Cases A and B (the controls) are unchanged in all three builds.

The unmodified source behaves exactly like the shipped binary, so the difference comes from your change. The decoder drops the bad frame after the 1 s wait and resumes at the next IDR. The active check lets PAUSED→READY break the loop.

We also ran the nvmultiurisrcbin repro from the other topic, swapping only libgstnvvideo4linux2.so:

shipped decoder patched decoder
stream/remove of the bad source no answer HTTP 200
REST probe no answer HTTP 200
other two sources after the scene change 0 fps 12 fps

Two questions:

  1. Will the bounded wait and the active check go into a future DeepStream release? Otherwise every user has to rebuild this plugin and keep it in sync across upgrades.
  2. Is the L4T r39 gst-nvvideo4linux2 source the one the DeepStream 9.1 dGPU libgstnvvideo4linux2.so is built from? It behaved the same in our test, but we would like to know which source to track for future versions.
  1. Currently it is just a workaround. We will get back to you after internal discussion.
  2. yes. dGPU and Jetson shared the same gst-nvvideo4linux2 codes. As shown in the compatibility doc , DS 9.1 uses Jetpack 7.2 and L4T 39.2. Build with NVDS_VERSION=9.1 when compiling for DeepStream 9.1.

Thanks for confirming, and for the pointer to the compatibility table.

Until a release includes the fix, we ship the rebuilt plugin in our image: the L4T 39.2 gst-nvvideo4linux2 source with your patch, built with NVDS_VERSION=9.1. Before replacing it, our build checks the md5 of the shipped libgstnvvideo4linux2.so. A new DeepStream image will make the build fail, so we re-validate instead of mixing versions without noticing.

One suggestion for the internal discussion: the drop is only logged with GST_DEBUG_OBJECT (“Stream format not found, dropping the frame”). At the default log level, an application cannot tell that it happened. A GST_WARNING_OBJECT, or an element message on the bus, would let us count these drops per source.

We are happy to test a candidate build when one is available.

Would you mind sharing your company name? You can use forum private email. please click forum avatar → "personal messages’ → “view all personal messages” → “new message”.