Issue with RTSP Streams on Jetson Nano and Xavier NX/Orin Nano

[simopalacino]

Defect:
RTSP streams in h.264/h.265 exhibit artifacts, seemingly stemming from incorrect decoding of h.264 frames. Particularly noticeable are artifacts during object movement, with characteristics from previous frames appearing repeatedly. This creates a temporal frame exchange impression. An evident example is the timestamp in the camera OSD, where digits appear to jump backward.

Previous Situation:
Jetson Nano (JetPack 4.6.1) with DeepStream 6.0.1 installed. RTSP streams without issues.

Intermediate Situation:
Same Jetson, same JetPack, DS v6.0.1 with BSP library update —specifically, a patch for the lib libgstvideo.so of GStreamer. This patch is present in GStreamer 1.16 but missing in the 1.14 version, which is the one currently used in L4T 32.7.1 (corresponding to JetPack 4.6.1).
The issue arises.

Current Situation:
Tested only on Xavier NX and Orin Nano with Jetpack versions later than the one mentioned above (which includes DS with the updated BSP library). In particular, I mention for both boards the JetPack version 5.1.2 corresponding to L4T v35.4.1, with DeepStream v6.3.
The issue persists.

Reproduce:
A simply way to reproduce this issue is using the deepstream-test3 with 1 or more RTSP streams.
I observed that the issue is not immediately apparent in certain RTSP streams, especially if it is the sole stream. However, when there are multiple streams, it generally manifests in at least one of them.
I have already experimented with various pipelines, and I’ve also attempted to use DeepStream with C GStreamer APIs, including using a fakesink and extracting frames through a probe after osd element after infer/analytics.

link original video:

202312121808_video01.mp4

[fanzh]

1. are you testing deepstream_test3_app.c? did you modify the code?
2. what is the rtsp media information? you can use “gst-discoverer-1.0 -v rtsp://xxx” or “ffprobe rtsp://xxx” to check.
3. wondering if it is network issue or decoding issue, could you share more logs? Thanks! please do the following setting first,
   export GST_DEBUG=3,rtpjitterbuffer:6,v4l2videodec:6
   then execute
   “your app command” >1.log 2>1.log,
   then use “ctrl+c” to end when there is artifact, then share 1.log. you can use zip to compress the log.

[simopalacino]

Yes, I try with small changes. In this case I just changed the batch-size of the streammux and infer by setting it to 1, with multiple RTSP streams, and the defect is much more noticeable (by default test3 has the batch-size equal to the number of streams)
I put in the zip file the 1.log with the GST_DEBUG that you mentioned, and two files cam_data.txt cam_data2.txt with the info of two streams used with both gst-discover and ffprobe. In particular the test3 is executed with 4 RTSP streams (2 streams repeated twice) with batch-size set to 1.
Thank you.
logs.zip (3.5 MB)

[fanzh]

gstrtpjitterbuffer.c:3211:gst_rtp_jitter_buffer_chain:<rtpjitterbuffer5>e[00m Queue full, dropping old packet 0xfffe4c00e200
from the log, many packets were dropped by rtpjitterbuffer plugin. you can modify
g_object_set(G_OBJECT(object),“drop-on-latency”,true,NULL); to g_object_set(G_OBJECT(object),“drop-on-latency”,false,NULL);

[simopalacino]

@fanzh

GST_DEBUG=3,rtpjitterbuffer:6,v4l2videodec:6

I’ve tried to set drop-on-latency to false and now deepstream-test3 seems to decode correctly but it is extremely slow and accumulates a lot of delay.
I’ve also tried this pipeline using gst-launch with nvurisrcbin (which it should have default drop-on-latency value set to false), and the problem persists, like in the attached video. Log of this last try is also attached.

export GST_DEBUG=3,rtpjitterbuffer:6,v4l2videodec:6
gst-launch-1.0 \
    nvurisrcbin uri="rtsp://admin:12345@192.168.81.111:555/media/video1"                   ! queue ! m.sink_0 \
    nvurisrcbin uri="rtsp://admin:12345@192.168.81.111:555/media/video1"                   ! queue ! m.sink_1 \
    nvurisrcbin uri="rtsp://admin:Fgs145836.@192.168.81.79:554/0/onvif/profile2/media.smp" ! queue ! m.sink_2 \
    nvurisrcbin uri="rtsp://admin:Fgs145836.@192.168.81.79:554/0/onvif/profile2/media.smp" ! queue ! m.sink_3 \
        nvstreammux name=m width=1920 height=1080 batch_size=4 live-source=true batched-push-timeout=40000 ! nvmultistreamtiler ! nv3dsink sync=false

1.zip (2.4 MB)

The same pipeline on Jetson Nano (without this patch: Update BSP library), works perfectly (with the patch it breaks like the other boards, but only with 25fps or more).

[fanzh]

please refer to topic for performance improvement.
please refer to topic for fps checking.
please refer to topic for latency checking.

[fanzh]

Sorry for the late reply, Is this still an DeepStream issue to support?

[zgjja]

I encountered your problem, i solved it by adding “select-rtp-protocol=4” option to “nvurisrcbin”, maybe the default UDP transmission is unstable, so a lot of buffers are lost during transmission.

（i am using deepstream 6.4, with “USE_NEW_NVSTREAMMUX=yes”)

[fanzh]

Sorry for the late reply, Is this still an DeepStream issue to support? Thanks!

[simopalacino]

I’m still trying to do some other tests based on the link that you mentioned in your previous messages. But still I have the problem. With the select-rtp-protocol=4 it gets better sometimes but the issue is still there.
Thank you

[fanzh]

There is no update from you for a period, assuming this is not an issue any more. Hence we are closing this topic. If need further support, please open a new one. Thanks.
can you provide a new log by the method on Dec 13?