File→RTSP (OpenCV+GStreamer→mediamtx) unstable on Jetson; direct RTSP works

I’m looking for a known-good GStreamer pipeline for file → RTSP (via mediamtx) on Jetson. Result video is attached and below is the script I run.

[Script]

#!/usr/bin/env python3

# test-rtsp-publish-from-file.py

import cv2, numpy as np, sys

# ------------------- CONFIG -------------------

input_file = sys.argv[1] if len(sys.argv) > 1 else “sample.mp4”

out_url = sys.argv[2] if len(sys.argv) > 2 else “rtsp://127.0.0.1:8554/gst-pub-test”

codec = (sys.argv[3] if len(sys.argv) > 3 else “h265”).lower() # h264|h265

if codec not in (“h264”, “h265”):

raise SystemExit(“codec must be h264 or h265”)

enc = f"nvv4l2{codec}enc"

parse = f"{codec}parse"

# ------------------- OPEN INPUT VIDEO -------------------

cap = cv2.VideoCapture(input_file)

if not cap.isOpened():

raise SystemExit(f"Cannot open input file: {input_file}")

W = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))

H = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))

FPS = cap.get(cv2.CAP_PROP_FPS) or 25 # fallback if zero

print(f"Input: {input_file} → {W}x{H} @ {FPS:.2f} fps")

# ------------------- GStreamer pipeline (same as before) -------------------

PIPE = (

f"appsrc is-live=true block=true format=time do-timestamp=true "

f"caps=video/x-raw,format=BGR,width={W},height={H},framerate={FPS:.2f}/1 ! "

"videoconvert ! video/x-raw,format=NV12 ! "

"nvvidconv ! video/x-raw(memory:NVMM),format=NV12 ! "

f"{enc} insert-sps-pps=true maxperf-enable=true iframeinterval={int(FPS)} bitrate=4000000 ! "

f"{parse} ! "

f"rtspclientsink location={out_url} protocols=tcp do-rtsp-keep-alive=true latency=0"

)

out = cv2.VideoWriter(PIPE, cv2.CAP_GSTREAMER, 0, FPS, (W, H))

print(f"[{codec.upper()}] Publishing to → {out_url}")

if not out.isOpened():

raise SystemExit(“Failed to open RTSP publish pipeline”)

# ------------------- MAIN LOOP -------------------

frame_count = 0

while True:

ret, frame = cap.read()

if not ret:

print(“End of input file or error reading frame”)

break

out.write(frame)

frame_count += 1

if frame_count % 30 == 0:

print(f"Published {frame_count} frames…")

cap.release()

out.release()

print(“Finished publishing. Play with:”)

print(f" gst-play-1.0 {out_url}")

print(f" or ffplay {out_url}")

[Env]
=== Jetson model ===
NVIDIA Jetson Orin NX Engineering Reference Developer Kit=== L4T/JetPack ===

R36 (release), REVISION: 4.3, GCID: 38968081, BOARD: generic, EABI: aarch64, DATE: Wed Jan 8 01:49:37 UTC 2025

nvidia-l4t-core 36.4.3-20250107174145
=== Kernel ===
Linux yuan-6n0cnx 5.15.148-tegra #1 SMP PREEMPT Thu May 15 09:27:14 CST 2025 aarch64 aarch64 aarch64 GNU/Linux
=== GStreamer ===
gst-inspect-1.0 version 1.20.3
Factory Details:
Factory Details:
=== OpenCV ===
OpenCV: 4.5.4
GStreamer: YES
=== MediaMTX ===
v1.15.3

Hello @karlpan,

Let me ask you a few questions just to make sure I am understanding the situation.

  1. What do you mean that the stream is unstable with mediamtx but it works with direct RTSP ?

  2. Was the video provided generated using direct RTSP?

  3. Can you provide an example of how you are trying to use MediaMTX in your app?

  4. Have you tried using filesrc in the pipeline instead of OpenCV to open the video and appsrc to grab it ?

  5. I would suggest you put some queue elements before the elements with might have heavier processing, such as videoconvert and the encoder.

Looking forward to hearing from you.

best regards,
Andrew
Embedded Software Engineer at ProventusNova

Hi,
Please try this simple python sample and see if the video preview is good:
Doesn't work nvv4l2decoder for decoding RTSP in gstreamer + opencv - #3 by DaneLLL

Hi Andrew,

Thanks for the follow-up:

  1. “Unstable with MediaMTX vs direct RTSP?”
    The stream I publish (file → my script → MediaMTX) is almost all black for ~6 s; after that, colors are relatively normal but there are still lots of glitches and broken textures. A direct RTSP stream from another device plays fine on the same viewer.

  2. Was the video from direct RTSP?
    No. It’s a screen recording from a Samsung Galaxy Tab Active 3 showing the stream produced by my script + MediaMTX running on Jetson Orin NX.

  3. Example of how I use MediaMTX
    Start MediaMTX (defaults), then publish:
    python3 /mnt/data/test-rtsp-publish-video.py --in segment_1216.mp4 --out rtsp://127.0.0.1:8554/result --codec h265
    (script attached)

  4. Tried filesrc instead of OpenCV?
    Yes—tested with a new GI/GStreamer filesrc script (attached as test-rtsp-publish-gst.py). Behavior didn’t improve. ffplay shows errors/warnings including:

    • [rtsp] max delay reached. need to consume packet

    • [rtsp] RTP: missed N packets (seen: 6, 9, 10, 11, 13, 17)

    • [hevc] Could not find ref with POC X (seen: 8, 13, 14, 15)

    • [NULL] No start code is found.

    • [hevc] No start code is found.

    • [hevc] Error splitting the input into NAL units.

    • [rtsp] max delay reached. need to consume packet

    • [rtsp] RTP: missed N packets (seen: 6, 9, 10, 11, 13, 17)

    • [hevc] Could not find ref with POC X (seen: 8, 13, 14, 15)

    • [NULL] No start code is found.

    • [hevc] No start code is found.

    • [hevc] Error splitting the input into NAL units.

  5. Queues before heavy stages?
    Yes—also tested with the new script, added queue before h264parse/nvv4l2decoder/nvvidconv/nvv4l2h265enc/rtspclientsink. Unplayable for most of the duration; only the last few seconds are partially viewable and glitchy.

    test-rtsp-publish-gst.txt (4.6 KB)

    Best regards,
    Karl

Hi @DaneLLL ,

Thanks for the link. I ran into a different problem though—mine is on the encoding/publishing side, not decoding. The sample you shared demonstrates RTSP decoding and preview (rtspsrc → nvv4l2decoder → appsink). In my case, I’m reading a local file and publishing to RTSP via MediaMTX; that published stream is largely un-decodable until the very end, while direct RTSP from another device plays fine on the same client. So the issue seems to be payloading/timestamps/encoder output, not the decoder path. If you have a recommended publish pipeline (caps/payloader/config-interval/transport), I can try that.

Best regards,
Karl

Hi,
There is a sample for setting up UDP:
Stream processed video with OpenCV on Jetson TX2 - #5 by DaneLLL

For setting up RTSP through OpenCV, would see if other users can share experience.

FYR, in C code, we can set up RTSP through test-launch:
Jetson AGX Orin FAQ

Hello @karlpan,

Thangs for the detailed update.

Quick question.
What happens if instead of a file as input you use videotestsrc ?

Do you get the same results?

best regards,
Andrew
Embedded Software Engineer at ProventusNova

Update — root cause & working pipeline

This turned out to be a packetization size issue. With rtspclientsink the RTP payloader is internal, so there’s no way to set MTU/payload size; that led to fragmentation/loss and the startup corruption.
I also tried MPEG-TS + udpsink, but payload size still isn’t controllable.

Switching to explicit RTP payloader + udpsink (so I can set mtu) fixed it. MediaMTX is configured to ingest the RTP/UDP on the target port.

**Final pipeline (H.265):
**
appsrc is-live=true block=true format=time do-timestamp=true
caps=video/x-raw,format=BGR,width=,height=,framerate=/1 !
queue max-size-buffers=30 max-size-time=1000000000 leaky=downstream !
videoconvert ! video/x-raw,format=NV12 !
nvvidconv ! video/x-raw(memory:NVMM),format=NV12 !
nvv4l2h265enc maxperf-enable=true bitrate=8000000 iframeinterval=30 idrinterval=30
insert-sps-pps=true insert-vui=true profile=0 control-rate=1 preset-level=1
num-B-Frames=0 num-Ref-Frames=1 vbv-size=8000000 !
h265parse ! video/x-h265,stream-format=byte-stream,alignment=au !
rtph265pay pt=96 mtu=1400 config-interval=1 !
udpsink host=127.0.0.1 port=1235 sync=false

Key bits:

rtph265pay mtu=1400 config-interval=1 (controllable payload size + periodic VPS/SPS/PPS)

If there’s a way to expose MTU via rtspclientsink, I’m happy to test it; for now, RTP payloader → udpsink into MediaMTX is stable on my side.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.