RTSP stream not work "Stream format not found, dropping the frame"

Hardware Jetson AGX Jetpack JP502.
GStreamer 1.16.3

Hello, i have an camera that stream the video via udp port 5600.

"#include <gst/gst.h>
#include <gst/rtsp-server/rtsp-server.h>
#include <iostream>
#include <string>
int main(int argc, char* argv[]) {
    gst_init(&argc, &argv);

    GMainLoop* main_loop = g_main_loop_new(NULL, false);

    GstRTSPServer* server = gst_rtsp_server_new();
    g_object_set(server, "service", "8553", NULL);
    
    std::string launch_string = "udpsrc port=5600 caps=application/x-rtp,encoding-name=(string)H264 ! rtph264depay ! rtph264pay name=pay0";
    std::cout << "start streaming to port 8553/live !!! \n";
    GstRTSPMediaFactory* factory = gst_rtsp_media_factory_new();
    gst_rtsp_media_factory_set_launch(factory, launch_string.c_str());
    gst_rtsp_media_factory_set_shared(factory, true);
    
    GstRTSPMountPoints* mount_points = gst_rtsp_server_get_mount_points(server);
    gst_rtsp_mount_points_add_factory(mount_points, "/live", factory);
    g_object_unref(mount_points);

    gst_rtsp_server_attach(server, NULL);

    g_main_loop_run(main_loop);
}

"
the code is working fine as i can capture the video stream in VLC with the address rtsp://127.0.0.1.:8553/live
however, i have another c++ app running on same system, that i expect to read the input rtsp address.
the gstream pipline as follow :

pipe_string = "rtspsrc location=\"";
pipe_string += address;
 pipe_string += "\" latency=0 ! rtph264depay ! h264parse ! queue ! nvv4l2decoder ! queue ! nvvideoconvert ! video/x-raw,format=BGRx";

however i got the bunch of error “Stream format not found, dropping the frame”.
my pipeline is working fine with other video source from rtsp camera. but for this rtsp stream, it’s not work. idk why.

Can you guys give me some advices?

I used this test pipeline
export GST_DEBUG=3
gst-launch-1.0 rtspsrc location=rtsp://127.0.0.1:8553/live ! application/x-rtp,media=video ! parsebin ! queue ! nvv4l2decoder ! queue ! fakesink

but i got the warning

0:00:11.317535899 3622921 0xaaaafb733d20 WARN              bufferpool gstbufferpool.c:1235:default_reset_buffer:<nvv4l2decoder0:pool:sink> Buffer 0xffff60060480 without the memory tag has maxsize (0) that is smaller than the configured buffer pool size (4194304). The buffer will be not be reused. This is most likely a bug in this GstBufferPool subclass
Stream format not found, dropping the frame
0:00:11.416085454 3622921 0xaaaafb733d20 WARN              bufferpool gstbufferpool.c:1235:default_reset_buffer:<nvv4l2decoder0:pool:sink> Buffer 0xffff60060360 without the memory tag has maxsize (0) that is smaller than the configured buffer pool size (4194304). The buffer will be not be reused. This is most likely a bug in this GstBufferPool subclass
Stream format not found, dropping the frame
0:00:11.514314164 3622921 0xaaaafb733d20 WARN              bufferpool gstbufferpool.c:1235:default_reset_buffer:<nvv4l2decoder0:pool:sink> Buffer 0xffff5003eea0 without the memory tag has maxsize (0) that is smaller than the configured buffer pool size (4194304). The buffer will be not be reused. This is most likely a bug in this GstBufferPool subclass
Stream format not found, dropping the frame

Okay i found the solution for this
because i forgot to run the sudo jetson_clocks, then the stream working fine

hmm,no i still got that problem after that. the stream is only work one time and not work anymore. the frame dropped error still there

  1. can VLC play the RTSP source?
  2. could you share the result “gst-discoverer-1.0 -v rtsp://your rtsp address”? if no gst-discoverer-1.0, you can use “ffprobe rtsp://your rtsp address” instead.

Hi @fanzh here is the result of gst-discoverer command

Analyzing rtsp://192.168.1.4:8559/live
Opening in BLOCKING MODE 
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
reference in DPB was never decoded
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
NVDEC_COMMON: NvDecGetSurfPinHandle : Surface not registered 
Done discovering rtsp://192.168.1.4:8559/live

Topology:
  unknown: application/x-rtp, media=(string)video, payload=(int)96, clock-rate=(int)90000, encoding-name=(string)H264, packetization-mode=(string)1, profile-level-id=(string)64002a, sprop-parameter-sets=(string)"Z2QAKqyyAPAET8uAtQEBAUAAAAMAQAAAHiPGDJI\=\,aOvMsiw\=", a-tool=(string)GStreamer, a-type=(string)broadcast, a-ts-refclk=(string)local, a-mediaclk=(string)sender, ssrc=(uint)654825449, clock-base=(uint)2585653043, seqnum-base=(uint)7889, npt-start=(guint64)0, play-speed=(double)1, play-scale=(double)1
    video: video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)4.2, profile=(string)high, pixel-aspect-ratio=(fraction)1/1, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true
      Tags:
        video codec: H.264 (High Profile)
      
      Codec:
        video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)4.2, profile=(string)high, pixel-aspect-ratio=(fraction)1/1, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true
      Additional info:
        None
      Stream ID: b9049c323800fa1dbf0c9c2f5d6dcf0e63b50fc2c5030d1c14e44a893d14e333/video:0:0:RTP:AVP:96
      Width: 1920
      Height: 1080
      Depth: 24
      Frame rate: 60/1
      Pixel aspect ratio: 1/1
      Interlaced: false
      Bitrate: 0
      Max bitrate: 0

Properties:
  Duration: 99:99:99.999999999
  Seekable: no
  Live: yes
  Tags: 
      video codec: H.264 (High Profile)

vlc is working fine as i said on above. but it’s not working in my app.

My system are setup like this

(Simulation UDP Stream used gstreamer) via port 5600 —> (My RTSP c++ server 8554/live) ----> (c++ application)

Thanks for the update!

  1. could you share more logs? please do the following setting first,
    export GST_DEBUG=6
    then execute
    gst-launch-1.0 rtspsrc location=rtsp://127.0.0.1:8553/live ! application/x-rtp,media=video ! parsebin ! queue ! nvv4l2decoder ! queue ! fakesink >1.log 2>1.og,
    then use “ctrl+c” to end after 10 seconds, then share 1.log. you can use zip to compress the log.
  2. could you use this command-line to dump some stream data?
gst-launch-1.0 rtspsrc location=rtsp://your rtsp address ! rtph264depay ! h264parse !   mux. mpegtsmux name=mux ! filesink location=output.ts
  1. how did you generate "(Simulation UDP Stream used gstreamer) via port 5600 "?

Hi @fanzh , Thank you for the support

  1. here is the 1.log and 1.og for above gst-launch command
    log.zip (7.6 MB)

  2. I can use the commend to save some stream data in output.ts file

gst-launch-1.0 rtspsrc location=rtsp://your rtsp address ! rtph264depay ! h264parse !   mux. mpegtsmux name=mux ! filesink location=output.ts
  1. i got the UDP simulation stream from PX4 gazebo camera plugin source , so i think the stream source is fine
    Gazebo Simulation · PX4 Developer Guide
  1. do you mean the decoding state is sometimes good and sometimes bad?
  2. could you share that output? I will have a try. You can use forum private email. please click forum avatar-> personal messages->new message.
1 Like
  1. i mean last time i get it to work somehow after activate jetson-clock. but only for that time, i tried again but not work. so i think that is not the solution so far.

  2. yes, i was shared the rtsp output to you. pls check the DM

AYK, decoder will decode successfully only after getting IDR frame. the rtspserver does not do transcoding. I suppose the camera 's h264 IDR idrinterval is too big. can you lower idrinterval to have a try?

1 Like

Thank you, i tried to reduce the frame update rate from 60fps to 30fps. but it’s seem not work.

not frame rare(fps), it is IDR frame interval.

1 Like

I tried but cannot find anywhere in the codebase of the camera to change the IDR frame. However, I have an idea that the UDP stream of the camera is working fine, and my app can capture the data from the udpsrc source.

In my application, two clients are needed to capture the camera data, but currently, the udpsrc can only supply the stream to one client. Therefore, I think creating a UDP stream to multiple UDP ports can be a solution

thanks for the update!

  1. will playing rtsp://127.0.0.1:8553/live fail if waiting one minute?
  2. could you do this test to get decoder low level log 2.log?
export GST_DEBUG=1 && touch /tmp/tegrav4l2_logs  && export TEGRA_TVMR_GENERAL_LOGS=1 && export TEGRA_VIDEO_LOGS=1 
gst-launch-1.0 rtspsrc location=rtsp://127.0.0.1:8553/live ! application/x-rtp,media=video ! parsebin ! queue ! nvv4l2decoder ! queue ! fakesink >2.log 2>2.log
mv /tmp/tegrav4l2_logs /tmp/tegrav4l2_logs_1
  1. from the output.ts you provided, there is only one IDR frame during 2 minutes. could you use this command-line to get 3 minutes h264 data? Thank you!
gst-launch-1.0 rtspsrc location=rtsp://your rtsp address ! rtph264depay ! h264parse ! video/x-h264,stream-format=byte-stream ! filesink location=test.h264

Hi fanzh, sorry for late reply.

  1. no, the RTSP stream keeps working fine VLC
  2. here is the attached 2.log and test.zip
    2.log (452.5 KB)
    test.zip (2.2 MB)

Thanks for the sharing! could provide more device information? Thanks! here are some command-line.
CUDA version nvcc -V
dpkg -l |grep TensorRT
dpkg -l | grep gstreamer
cat /etc/nv_tegra_release or jetson_release
apt-cache show nvidia-jetpack

Please provide complete information as applicable to your setup.
• Hardware Platform (Jetson / GPU)
• DeepStream Version
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)

@fanzh thank you, here is my device information

  • CUDA Version:
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_May__4_00:02:26_PDT_2022
Cuda compilation tools, release 11.4, V11.4.239
Build cuda_11.4.r11.4/compiler.31294910_0
  • Hardware Jetson AGX Xavier

  • TensorRT version: 8.4.1-1+cuda11.4

  • nvidia-jetpack: (apt-cache show nvidia-jetpack)
    Version: 5.0.2-b231
    Architecture: arm64
    Maintainer: NVIDIA Corporation
    Installed-Size: 194
    Depends: nvidia-jetpack-runtime (= 5.0.2-b231), nvidia-jetpack-dev (= 5.0.2-b231)

  • Deepstream Version 6.1

I can decode the test.h264 by this command-line
gst-launch-1.0 filesrc location=test.h264 ! h264parse config-interval=-1 ! nvv4l2decoder ! fakesink

please the following command-line to play the rtsp source.
no rendering:

gst-launch-1.0 rtspsrc location=rtsp://10.19.227.166/media/video1! rtph264depay ! h264parse config-interval=-1 ! nvv4l2decoder ! fakesink

with rendering:

gst-launch-1.0 rtspsrc location=rtsp://10.19.227.166/media/video1! rtph264depay ! h264parse config-interval=-1 ! nvv4l2decoder ! nv3dsink