MP4 File Streaming via RTSP Causes Encoder to Freeze

[Alvin_Fok]

Please provide complete information as applicable to your setup.

• Hardware Platform: RTX 5090 D
• DeepStream Version: 8.0
• TensorRT Version: 10.9.0
• NVIDIA GPU Driver Version: 580.95.05
• Issue Type: questions
• How to reproduce the issue ?

stream sample_720p.mp4 via RTSP

ffmpeg -stream_loop -1 -re -fflags +genpts -i /input/sample_720p.mp4 -c:v libx264 -preset veryfast -tune zerolatency -x264-params bframes=0 -f rtsp rtsp://rtsp-server:8554/sample

Consuming RTSP stream causes pipline to freeze

gst-launch-1.0 uridecodebin uri=rtsp://192.168.50.120:8554/sample ! nvstreammux0.sink_0 nvstreammux name=nvstreammux0 batch-size=1 batched-push-timeout=40000 width=1280 height=720 live-source=1 ! nvv4l2h264enc ! fakesink
Setting pipeline to PAUSED ...
Opening in BLOCKING MODE 
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Pipeline is PREROLLED ...
Prerolled, waiting for progress to finish...
Progress: (connect) Connecting to rtsp://192.168.50.120:8554/sample
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (request) SETUP stream 1
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Redistribute latency...
Redistribute latency...
Redistribute latency...
Progress: (request) Sending PLAY request
Redistribute latency...
Progress: (request) Sent PLAY request
WARNING: from element /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstRTSPSrc:source: Could not read from resource.
Additional debug info:
../gst/rtsp/gstrtspsrc.c(6223): gst_rtspsrc_reconnect (): /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstRTSPSrc:source:
Could not receive any UDP packets for 5.0000 seconds, maybe your firewall is blocking it. Retrying using a tcp connection.
Redistribute latency...
Redistribute latency...
Opening in BLOCKING MODE 
Redistribute latency...
Redistribute latency...
Redistribute latency...
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:10.894622455
Setting pipeline to NULL ...
Freeing pipeline ...

Consum mp4 file works normally

gst-launch-1.0 uridecodebin uri=file:///opt/nvidia/deepstream/deepstream-8.0/samples/streams/sample_720p.mp4 ! nvstreammux0.sink_0 nvstreammux name=nvstreammux0 batch-size=1 batched-push-timeout=40000 width=1280 height=720 live-source=0 ! nvv4l2h264enc ! fakesink

Consum real RTSP camera works normally too

gst-launch-1.0 uridecodebin uri='rtsp://admin:varadise888@192.168.48.49:554/cam/realmonitor?channel=1&subtype=0' ! nvstreammux0.
sink_0 nvstreammux name=nvstreammux0 batch-size=1 batched-push-timeout=40000 width=1280 height=720 live-source=1 ! nvv4l2h264enc ! fakesink

[fanzh]

1. From the log, the app can’t receive udp packets, then tried tcp connection. are you testing the two cmds in docker container? To simplfy the pipeline, could you share the result of “gst-launch-1.0 uridecodebin uri=rtsp://192.168.50.120:8554/sample ! fakesink”?
2. if the issue persists in step1, to only test the stream receiving, please share the result of “gst-launch-1.0 rtspsrc location=rtsp://192.168.50.120:8554/sample ! fakesink”?

[Alvin_Fok]

1. yes, all cmd run in docker
2. log of “gst-launch-1.0 uridecodebin uri=rtsp://192.168.50.120:8554/sample ! fakesink“ and it work normally

3. gst-launch-1.0 uridecodebin uri=rtsp://192.168.50.120:8554/sample ! fakesink
   Setting pipeline to PAUSED ...
   Pipeline is live and does not need PREROLL ...
   Progress: (open) Opening Stream
   Pipeline is PREROLLED ...
   Prerolled, waiting for progress to finish...
   Progress: (connect) Connecting to rtsp://192.168.50.120:8554/sample
   Progress: (open) Retrieving server options
   Progress: (open) Retrieving media info
   Progress: (request) SETUP stream 0
   Progress: (request) SETUP stream 1
   Progress: (open) Opened Stream
   Setting pipeline to PLAYING ...
   New clock: GstSystemClock
   Progress: (request) Sending PLAY request
   Redistribute latency...
   Redistribute latency...
   Progress: (request) Sending PLAY request
   Redistribute latency...
   Redistribute latency...
   Progress: (request) Sent PLAY request
   WARNING: from element /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstRTSPSrc:source: Could not read from resource.
   Additional debug info:
   ../gst/rtsp/gstrtspsrc.c(6223): gst_rtspsrc_reconnect (): /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstRTSPSrc:source:
   Could not receive any UDP packets for 5.0000 seconds, maybe your firewall is blocking it. Retrying using a tcp connection.
   Redistribute latency...
   Redistribute latency...
   Opening in BLOCKING MODE 
   Redistribute latency...
   Redistribute latency...
   ^Chandling interrupt.
   Interrupt: Stopping pipeline ...
   Execution ended after 0:00:45.064650505
   Setting pipeline to NULL ...
   Freeing pipeline ...
   

[fanzh]

@Alvin_Fok

log of “gst-launch-1.0 uridecodebin uri=rtsp://192.168.50.120:8554/sample ! fakesink“ and it work normally

1. the log is simiar to the log in the issue descripiton. How did you observe that one freezed and other other worked well? Thanks! you can use “gst-launch-1.0 uridecodebin uri=rtsp://192.168.50.120:8554/sample ! nveglglessink” to view the output video.
2. If this cmd works well, that means receiving and decoding are fine. you can add other plugins one by one. can “gst-launch-1.0 uridecodebin uri=rtsp://192.168.50.120:8554/sample ! nvstreammux0.sink_0 nvstreammux name=nvstreammux0 batch-size=1 batched-push-timeout=40000 width=1280 height=720 live-source=1 ! fakesink” work well?

[Alvin_Fok]

1.The progress of the last line in the log freezes and never exceeds 20 seconds.

gst-launch-1.0 uridecodebin uri=rtsp://192.168.50.120:8554/sample ! nvstreammux0.sink_0 nvstreammux name=nvstreammux0 batch-size=1 batched-push-timeout=40000 width=1280 height=720 live-source=1 ! nvv4l2h264enc ! fakesink
Setting pipeline to PAUSED ...
Opening in BLOCKING MODE 
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Pipeline is PREROLLED ...
Prerolled, waiting for progress to finish...
Progress: (connect) Connecting to rtsp://192.168.50.120:8554/sample
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (request) SETUP stream 1
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Redistribute latency...
Redistribute latency...
Redistribute latency...
Progress: (request) Sending PLAY request
Redistribute latency...
Progress: (request) Sent PLAY request
WARNING: from element /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstRTSPSrc:source: Could not read from resource.
Additional debug info:
../gst/rtsp/gstrtspsrc.c(6223): gst_rtspsrc_reconnect (): /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstRTSPSrc:source:
Could not receive any UDP packets for 5.0000 seconds, maybe your firewall is blocking it. Retrying using a tcp connection.
Redistribute latency...
Redistribute latency...
Opening in BLOCKING MODE 
Redistribute latency...
Redistribute latency...
Redistribute latency...
0:00:14.5 / 99:99:99.

Here are some cmd I try and failed

gst-launch-1.0 uridecodebin uri=rtsp://192.168.50.120:8554/sample ! nveglglessink

gst-launch-1.0 uridecodebin uri=file:///opt/nvidia/deepstream/deepstream-8.0/samples/streams/sample_720p.mp4 ! nveglglessink

gst-launch-1.0 uridecodebin uri=rtsp://192.168.50.120:8554/sample ! nvstreammux0.sink_0 nvstreammux name=nvstreammux0 batch-size=1 batched-push-timeout=40000 width=1280 height=720 live-source=1 ! nveglglessink 


gst-launch-1.0 uridecodebin uri=file:///opt/nvidia/deepstream/deepstream-8.0/samples/streams/sample_720p.mp4 ! nvstreammux0.sink_0 n
vstreammux name=nvstreammux0 batch-size=1 batched-push-timeout=40000 width=1280 height=720 live-source=0 ! nveglglessink

same log of above cmd

Setting pipeline to PAUSED ...
Failed to set pipeline to PAUSED.
Setting pipeline to NULL ...
Freeing pipeline ...

2.“gst-launch-1.0 uridecodebin uri=rtsp://192.168.50.120:8554/sample ! nvstreammux0.sink_0 nvstreammux name=nvstreammux0 batch-size=1 batched-push-timeout=40000 width=1280 height=720 live-source=1 ! fakesink” work well

[fanzh]

1. Did the freeze issue occur at the beginning or after a while? can the freeze issue be reproduced every time? what is the fps of "rtsp://192.168.50.120:8554/sample”?
2. To get more logs, could you share the 1.log after running this cmd? gst-launch-1.0 --gst-debug-level=3,rtpjitterbuffer:6,h264parse:6,v4l2videoenc:6,videoencoder:6 uridecodebin uri=rtsp://192.168.50.120:8554/sample ! nvstreammux0.sink_0 nvstreammux name=nvstreammux0 batch-size=1 batched-push-timeout=40000 width=1280 height=720 live-source=1 ! nvv4l2h264enc ! fakesink >1.log 2>1.log

[Alvin_Fok]

1.Freeze issue occur at the beginning. Yes, this happen every time. The FPS of "rtsp://192.168.50.120:8554/sample” is 30.

2.This is the log from running the command for approximately 1 minute

1.log (22.8 KB)

[fanzh]

from 1.log, the cmd did not receive any packets. Are you sure gst-launch-1.0 uridecodebin uri=rtsp://192.168.50.120:8554/sample ! nvstreammux0.sink_0 nvstreammux name=nvstreammux0 batch-size=1 batched-push-timeout=40000 width=1280 height=720 live-source=1 ! fakesink works well? could you share 2.log after running this cmd?
gst-launch-1.0 --gst-debug-level=3,rtpjitterbuffer:6,h264parse:6,v4l2videodec:6,videodecoder:6 uridecodebin uri=rtsp://192.168.50.120:8554/sample ! nvstreammux0.sink_0 nvstreammux name=nvstreammux0 batch-size=1 batched-push-timeout=40000 width=1280 height=720 live-source=1 ! fakesink >2.log 2>2.log

[Alvin_Fok]

The log look similar before I restart the docker

2_before_docker_restart.log (15.5 KB)

After I restart the docker gst-launch-1.0 uridecodebin uri=rtsp://192.168.50.120:8554/sample ! nveglglessink and gst-launch-1.0 uridecodebin uri=rtsp://192.168.50.120:8554/sample ! nvstreammux0.sink_0 nvstreammux name=nvstreammux0 batch-size=1 batched-push-timeout=40000 width=1280 height=720 live-source=1 ! nveglglessink can show the video ouput on the screen.

However gst-launch-1.0 --gst-debug-level=3,rtpjitterbuffer:6,h264parse:6,v4l2videodec:6,videodecoder:6 uridecodebin uri=rtsp://192.168.50.120:8554/sample ! nvstreammux0.sink_0 nvstreammux name=nvstreammux0 batch-size=1 batched-push-timeout=40000 width=1280 height=720 live-source=1 ! nvv4l2h264enc ! fakesink still freeze

And this is the log of gst-launch-1.0 --gst-debug-level=3,rtpjitterbuffer:6,h264parse:6,v4l2videodec:6,videodecoder:6 uridecodebin uri=rtsp://192.168.50.120:8554/sample ! nvstreammux0.sink_0 nvstreammux name=nvstreammux0 batch-size=1 batched-push-timeout=40000 width=1280 height=720 live-source=1 ! fakesink >2.log 2>2.log

2_after_docker_restart.log (15.5 KB)

seem samilar with before docker restart

[fanzh]

1. from 2_after_docker_restart.log, there is no any decoding log. did gst-launch-1.0 uridecodebin uri=rtsp://192.168.50.120:8554/sample ! nvstreammux0.sink_0 nvstreammux name=nvstreammux0 batch-size=1 batched-push-timeout=40000 width=1280 height=720 live-source=1 ! fakesink still freeze? if so, the issue should be not related to nvv4l2h264enc encoding.
2. noticing gst-launch-1.0 uridecodebin uri=rtsp://192.168.50.120:8554/sample ! nveglglessink and gst-launch-1.0 uridecodebin uri=rtsp://192.168.50.120:8554/sample ! nvstreammux0.sink_0 nvstreammux name=nvstreammux0 batch-size=1 batched-push-timeout=40000 width=1280 height=720 live-source=1 ! nveglglessink can show the video, if run this cmd many times, can the cmd output video every time?

[Alvin_Fok]

1. No, it work well, it can run over 60s

gst-launch-1.0 uridecodebin uri=rtsp://192.168.50.120:8554/sample ! nvstreammux0.sink_0 nvstreammux name=nvstreammux0 batch-size=1 batched-push-timeout=40000 width=1280 height=720 live-source=1 ! fakesink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Pipeline is PREROLLED ...
Prerolled, waiting for progress to finish...
Progress: (connect) Connecting to rtsp://192.168.50.120:8554/sample
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (request) SETUP stream 1
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Redistribute latency...
Redistribute latency...
Redistribute latency...
Progress: (request) Sending PLAY request
Redistribute latency...
Progress: (request) Sent PLAY request
WARNING: from element /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstRTSPSrc:source: Could not read from resource.
Additional debug info:
../gst/rtsp/gstrtspsrc.c(6223): gst_rtspsrc_reconnect (): /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstRTSPSrc:source:
Could not receive any UDP packets for 5.0000 seconds, maybe your firewall is blocking it. Retrying using a tcp connection.
Redistribute latency...
Redistribute latency...
Opening in BLOCKING MODE 
Redistribute latency...
Redistribute latency...
0:01:10.9 / 99:99:99.

2. I ran the command 10 times, and it worked each time, displaying the video output for over 30 seconds.

[fanzh]

One correction. sometimes “–gst-debug-level=3,rtpji…” does not work. To get the receiving, decoding and encoding log, could you share the 3.log after running this cmd. you can use zip to compress the big file. Thanks! gst-launch-1.0 --gst-debug-level=3 --gst-debug=rtpjitterbuffer:6,h264parse:6,v4l2videodec:6,videodecoder:6,v4l2videoenc:6,videoencoder:6 uridecodebin uri=rtsp://192.168.50.120:8554/sample ! nvstreammux0.sink_0 nvstreammux name=nvstreammux0 batch-size=1 batched-push-timeout=40000 width=1280 height=720 live-source=1 ! nvv4l2h264enc ! fakesink >3.log 2>3.log

[Alvin_Fok]

Just wondering. How do you find out useful info in big log?Feed to AI or experience.

3.zip (3.1 MB)

[fanzh]

1. No AI. After checking 3.log, most decoded frames are dumped because decoding is slow and comming too late. you can find many “discarding ghost frame” in the log. There are only 13 seconds logs in 3.log. Noticing " 2. I ran the command 10 times, and it worked each time, displaying the video output for over 30 seconds.", How long did you wait before the first frame is displayed?
2. will test.264 be empty after running this cmd for 30 seconds? gst-launch-1.0 uridecodebin uri=rtsp://192.168.50.120:8554/sample ! nvstreammux0.sink_0 nvstreammux name=nvstreammux0 batch-size=1 batched-push-timeout=40000 width=1280 height=720 live-source=1 ! nvv4l2h264enc ! filesink location=test.264. If test.264 is empty, could you share some source streams after running this cmd gst-launch-1.0 rtspsrc location=rtsp://192.168.50.120:8554/sample ! rtph264depay ! h264parse ! mux. mpegtsmux name=mux ! filesink location=output.ts? Thanks! Did you run other decoding tasks? if so, please check if this is a decoding performance issue.

[Alvin_Fok]

Sorry for the delayed response!

1. I ran gst-launch-1.0 uridecodebin uri=rtsp://192.168.50.120:8554/sample ! nveglglessink 5 times, and it took about 12.66s–15.41s each time to start showing the video playback.
2. gst-launch-1.0 uridecodebin uri=rtsp://192.168.50.120:8554/sample ! nvstreammux0.sink_0 nvstreammux name=nvstreammux0 batch-size=1 batched-push-timeout=40000 width=1280 height=720 live-source=1 ! nvv4l2h264enc ! filesink location=test.264 still freeze and test.264 is only 100.7kB
3. 30s ouput of gst-launch-1.0 rtspsrc location=rtsp://192.168.50.120:8554/sample ! rtph264depay ! h264parse ! mux. mpegtsmux name=mux ! filesink location=output.ts

test.zip (98.4 KB)

output.zip (14.0 MB)

[fanzh]

1. Thanks for the update! To help to reproduce this issue, could you provide the steps to set up the rtsp-server and the related configuration file? Thanks! As written in my last comment, did you run other decoding tasks? If so, please stop the other tasks first, then try again.
2. if using nveglglessink, was the video smooth after showing the video? not narrow down this issue, could you provide 30 seconds of 4.log? gst-launch-1.0 --gst-debug-level=3 --gst-debug=rtpjitterbuffer:6,h264parse:6,v4l2videodec:6,videodecoder:6,v4l2videoenc:6,videoencoder:6 uridecodebin uri=rtsp://192.168.50.120:8554/sample ! nvstreammux0.sink_0 nvstreammux name=nvstreammux0 batch-size=1 batched-push-timeout=40000 width=1280 height=720 live-source=1 ! nveglglessink >4.log 2>4.log, is the video smooth after running this cmd?
3. , there is only one frame in test.264. To compare the log, could you provide 30 seconds of 5.log? gst-launch-1.0 --gst-debug-level=3 --gst-debug=rtpjitterbuffer:6,h264parse:6,v4l2videodec:6,videodecoder:6,v4l2videoenc:6,videoencoder:6 uridecodebin uri=rtsp://192.168.50.120:8554/sample ! nvstreammux0.sink_0 nvstreammux name=nvstreammux0 batch-size=1 batched-push-timeout=40000 width=1280 height=720 live-source=1 ! nvv4l2h264enc ! filesink location=test.264 >5.log 2>5.log

[Alvin_Fok]

1.Here is the docker-compose and config files of the rtsp server.

rtsp-server.zip (8.2 KB)

And the cmd to stream the sample.mp4

pm2 start --name sample --interpreter none "ffmpeg -stream_loop -1 -re -fflags +genpts -i /input/sample_720p.mp4 -c:v libx264 -preset veryfast -tune zerolatency -x264-params bframes=0 -f rtsp rtsp://rtsp-server:8554/sample"
pm2 save

2.There are no other decoding task when I ran those cmd.

3.The video is smooth when using nveglglessink

4.gst-launch-1.0 --gst-debug-level=3 --gst-debug=rtpjitterbuffer:6,h264parse:6,v4l2videodec:6,videodecoder:6,v4l2videoenc:6,videoencoder:6 uridecodebin uri=rtsp://192.168.50.120:8554/sample ! nvstreammux0.sink_0 nvstreammux name=nvstreammux0 batch-size=1 batched-push-timeout=40000 width=1280 height=720 live-source=1 ! nveglglessink >4.log 2>4.log the video ouput is also smooth

4.zip (3.8 MB)

5.zip (1.3 MB)

[fanzh]

Thanks for the sharing! Here is a workaround. We will continue to check why encoding nv12 hangs.

gst-launch-1.0 uridecodebin uri=rtsp://192.168.50.120:8554/sample  ! nvstreammux0.sink_0 nvstreammux name=nvstreammux0 batch-size=1 batched-push-timeout=40000 width=1280 height=720 live-source=1 ! nvvideoconvert ! 'video/x-raw(memory:NVMM),format=I420' ! nvv4l2h264enc ! filesink location=test.264