Garbled frames are coming in RTSP streams

• Hardware Platform (Jetson / GPU) Jetson
• DeepStream Version 6.1
• JetPack Version (valid for Jetson only) 5.0.2
• TensorRT Version 8.4.1
• Issue Type( questions, new requirements, bugs) Bugs

Getting garbled frames while running RTSP streams in deepstream. We have another pipeline using FFMPEG to run the same RTSP streams, there the issue is not happening. The RTSP streams are stable.

Currently I’m using “nvurisrcbin” to decode the streams. I’ve change the property "“select-rtp-protocol” to 4 (TCP). I got slightly improved frames but the issue is still there.

Please help me to solve this issue.

can you share sample code and config file ?

provide a complete log when executing export GST_DEBUG=3

update to the latest version if possible

thanks.

@vino_mm I’ve got the same issue and was told “It was my network”, good luck getting any actual help on these forums. They typically reply with short sentences and are quick to close your topic. I have 4 Jetson Orins with the latest Deepstream and we see this issue on all 4 Jetson’s. We have tried every network configuration possible, even direct connecting into the Jetsons and we are still facing this issue. We have tried changing absolutely every setting in the deepstream_app_config.txt file to no avail.

We also notice that the RTSP stream that is generated from Nvidia Deepstream has issues. It’s very slow to start and it’s constantly dropping frames. You can run the following example:

#include <gst/gst.h>
#include <gst/rtsp-server/rtsp-server.h>

int main(int argc, char *argv[]) {
          GMainLoop *loop;
            GstRTSPServer *server;
              GstRTSPMountPoints *mounts;
                GstRTSPMediaFactory *factory;

                  gst_init(&argc, &argv);

                    loop = g_main_loop_new(NULL, FALSE);

                      server = gst_rtsp_server_new();
                        mounts = gst_rtsp_server_get_mount_points(server);

                          factory = gst_rtsp_media_factory_new();
                            // gst_rtsp_media_factory_set_launch(factory, "( "
                            //                     "videotestsrc ! "
                            //                             "x264enc speed-preset=ultrafast tune=zerolatency key-int-max=1 intra-refresh=true insert-vui=1 insert-sps-pps=1  !  "
                            //                                 "rtph264pay name=pay0 pt=96 config-interval=1 )");

                           gst_rtsp_media_factory_set_launch(factory, "( videotestsrc ! nvvidconv ! nvv4l2h264enc ! rtph264pay name=pay0 pt=96 )");
                           gst_rtsp_media_factory_set_protocols(factory, GST_RTSP_LOWER_TRANS_UDP | GST_RTSP_LOWER_TRANS_TCP);
                            gst_rtsp_mount_points_add_factory(mounts, "/test", factory);
                                g_object_unref(mounts);

                                  gst_rtsp_server_attach(server, NULL);

                                    g_print("Stream ready at rtsp://127.0.0.1:8554/test\n");
                                      g_main_loop_run(loop);

                                        return 0;
}

Copy the above code to a file named test-rtsp-server.c on your Jetson and build it like so:

gcc -o test-rtsp-server test-rtsp-server.c $(pkg-config --cflags --libs gstreamer-1.0 gstreamer-rtsp-server-1.0)

This will start the videotestsrc on an RTSP Stream on your Jetson. We can play this stream on VLC without any issues.

But try inputting it into your Nvidia Deepstream app, even with PGIE turned off and we lose packets constantly. Something is just flat out wrong with Nvidia Deepstream, I have seen too many people with this same exact issue(just search the forums). They have a bug and refuse to take a look at it. Here is my test configuration file for reference:
deepstream_app_config_test.txt (2.4 KB)

If you play the above test-rtsp-server RTSP feed on VLC it pops up instantly without any errors. However, if you run the same exact RTSP stream through Nvidia Deepstream and try to play that output you see packet loss and it’s very slow to start(You also see the ghosting/pixilation that you are seeing in the above image!). Here are the warnings VLC give when trying to play the Nvidia Deepstream RTSP Stream:
VLCDeepstreamLog5-15-23.txt (5.1 KB)

@junshengy When are you guys going to elevate this issue and come out with a fix? You can search the forums and many others have this same exact issue. Why are we seeing the ghosting/pixilation? Why does the RTSP feed generated by Nvidia Deepstream lose packets(even when directly connected to the Jetson Orin)?

Here’s how you can recreate the issue:

1.Run the test RTSP Server with the code I have above(test-rtsp-server.c).
2. This will create an RTSP Stream running on rtsp://localhost:8554/test.
3. Input this stream into the base Nvidia Deepstream App and run with the config file I have posted above:

deepstream-app -c deepstream_app_config_test.txt
  1. This will host a stream on rstp://localhost:8556/ds-test. Play this with gst-launch, VLC, or ffmpeg with debug logs and you will see that it is slow to start and has packet loss.

I’ve kind of made some lead way as far as the RTSP Stream output from Nvidia Deepstream Starting up slowly. I have edited the following file apps-common/deepstream_sink_bin.c to add idframeinterval ever 5 frames and it starts up much faster!

In this method:

static gboolean
create_udpsink_bin (NvDsSinkEncoderConfig * config, NvDsSinkBinSubBin * bin)

I edited the following line and it starts much quicker!

if (prop.integrated) {
    if (config->enc_type == NV_DS_ENCODER_TYPE_HW) {
      g_object_set (G_OBJECT (bin->encoder), "preset-level", 1, NULL);
      g_object_set (G_OBJECT (bin->encoder), "insert-sps-pps", 1, NULL);

      #ifdef IS_TEGRA
        g_print("System is TEGRA! Setting idrinterval! \n");
        g_object_set (G_OBJECT (bin->encoder), "idrinterval", 5, NULL);
      
      #endif

    
    }
  } else {
    g_object_set (G_OBJECT (bin->transform), "gpu-id", config->gpu_id, NULL);
  }

The part I added was:

#ifdef IS_TEGRA
        g_print("System is TEGRA! Setting idrinterval! \n");
        g_object_set (G_OBJECT (bin->encoder), "idrinterval", 5, NULL);
      
      #endif

Which basically says if the systems architecture is Tegra then add the idrinterval at an interval of 5! I got this idea from the following forum post:
https://forums.developer.nvidia.com/t/deepstream-6-2version-seems-not-produce-idr-periodically/251958/5

@tjwhitten25 Thanks for your time.
Currently I’m using fakesink because I only need the output predictions. Also I just want to run the pipeline at 1-2 FPS. So I’m trying to tweak some parameters in the src plugin like latency, buffer-size, etc. to make the frames stable.

thanks for sharing @tjwhitten25

I think your problem is different from @vino_mm.

First because rtp is transmitted by udp, udp will drop packets.

Packet loss will cause the decoder to be abnormal.

dump the input stream can verify this problem.

dump rtsp stream to local file when meet garbled frames

then play use ffplay/vlc,ffplay/vlc use software decoder by default

If not ok,packet loss has already occurred, nvdecoder can’t work fine.

If ok, then use gst-launch-1.0 playbin uri=xxxx to play it.

If it plays normally, it can prove that nvdecoder is fine.

If not,Let’s follow up the difference between software decoder and nvdecoder.

and GST_DEBUG=3 rtph264depay will be print log about packets loss

@tjwhitten25

From your description, the problem seems to be related to the nvencoder.

I’m trying to reproduce the problem in your code.

If there is progress, I will sync to here.

Thank you so much for sharing.

@junshengy is there a way to minimise the packet loss?? I’m willing to trade FPS because I just want to run the pipeline at 1-2 FPS…

I’m afraid it cannot be done with gsteamer or deepstream alone.

It depends on your rtsp server, if your rtsp server can configured as rtp over tcp.

There are no packets loss.because tcp is reliable transmission.

thanks

@junshengy Most of our streams are TCP only.

If it is not a network problem,Can you dump some garbled streams for me?

I want to try it with local file.

Because I don’t have this problem with my rtsp camera.

thanks for sharing.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.