High latency in RTSP output for deepstream default app

[emadomari40]

**• Hardware Platform: Jetson Orin Nano
**• DeepStream Version: 6.3
**• JetPack Version: 5.1

I am facing a huge latency in the RTSP output from the deepstream default app.
When I run the EglSink, I don’t see any delay or high latency, but when I run the RTSP sink, I get a huge delay (3-5 seconds), I am using ffplay with no buffer and low delay to stream the output.

My configuration:

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=1
gie-kitti-output-dir=output

[tiled-display]
enable=0
rows=1
columns=1
width=1280
height=720
gpu-id=0
nvbuf-memory-type=0
#compute-hw=2

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=4
num-sources=1
#select-rtp-protocol=0
latency=30
uri=rtsp://192.168.20.10:8554/live0
#uri=udp://192.168.20.3:11024
#uri=file://IR.mp4
#drop-frame-interval=5
num-sources=1
gpu-id=0
cudadec-memtype=0
#camera-width=640
#camera-height=480
#camera-fps-n=30
#camera-fps-d=1
#camera-v4l2-dev-node=0
gpu-id=0
#num-extra-surfaces=1

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File 4=RTSPStreaming 5=Overlay
type=4
enc-type=1
sw-preset=40000000
nvbuf-memory-type=0
source-id=0
#1=h264 2=h265
codec=1
sync=0
bitrate=4000000
profile=0
rtsp-port=8554
udp-port=5400


[osd]
enable=1
gpu-id=0
border-width=5
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[streammux]
gpu-id=0
live-source=1
batch-size=1
batched-push-timeout=0
width=640
height=480
enable-padding=0
nvbuf-memory-type=0
#sync-inputs=1
#max-latency=1

[primary-gie]
enable=1
gpu-id=0
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary_yoloV5.txt

[tests]
file-loop=0

Please note that I dont face any delay with EglSink sink (type=2).
I have tried everything in the sink source.

[fanzh]

please refer to this topic.

[emadomari40]

The output rtsp is too slow to play through the tool

you can modify the encoder 's idrinterval to improve. you can get the explanation by gst-inspect-1.0 nvv4l2h264enc. encoder.set_property(‘idrinterval’, 60)

How to implement the solution to the deepstream-app?

[fanzh]

deepstream-app code is opensource. you can set that property in create_udpsink_bin of \opt\nvidia\deepstream\deepstream\sources\apps\apps-common\src\deepstream_sink_bin.c. then rebuild deepstream-app. the path is /opt/nvidia/deepstream/deepstream/sources/apps/sample_apps/deepstream-app.

[emadomari40]

I got this error:

…/…/apps-common/src/deepstream_sink_bin.c: In function ‘create_udpsink_bin’:
…/…/apps-common/src/deepstream_sink_bin.c:577:3: error: ‘encoder’ undeclared (first use in this function)
577 | encoder.set_property(‘idrinterval’, 60);

[fanzh]

the actual plugin name is bin->encoder.

[emadomari40]

So how the actual code should be?
What should I write in deepstream_sink_bin.c?

[emadomari40]

I got this error when I added ( g_object_set (G_OBJECT (bin->encoder), “idrinterval”, 60, NULL); ):

(deepstream-app:4232): GLib-GObject-WARNING **: 13:53:20.109: g_object_set_is_valid_property: object class ‘GstX264Enc’ has no property named ‘idrinterval’

When I changed:

g_object_set (G_OBJECT (bin->codecparse), “config-interval”, -1, NULL);

to

  g_object_set (G_OBJECT (bin->codecparse), "config-interval", 60, NULL);

It became slower.
Please advice.

[fanzh]

please refer to this code:
else
bin->encoder =
gst_element_factory_make (NVDS_ELEM_ENC_H265_HW, encode_name);
g_object_set (G_OBJECT (bin->encoder), “idrinterval”, 60, NULL);

[emadomari40]

still the same error:

(deepstream-app:6284): GLib-GObject-WARNING **: 14:19:01.480: g_object_set_is_valid_property: object class ‘GstX264Enc’ has no property named ‘idrinterval’

[emadomari40]

I tried everywhere:

  switch (config->codec) {
    case NV_DS_ENCODER_H264:
      bin->codecparse = gst_element_factory_make ("h264parse", "h264-parser");
      g_object_set (G_OBJECT (bin->codecparse), "config-interval", -1 , NULL);
      bin->rtppay = gst_element_factory_make ("rtph264pay", rtppay_name);
      if (config->enc_type == NV_DS_ENCODER_TYPE_SW)
        {bin->encoder =
            gst_element_factory_make (NVDS_ELEM_ENC_H264_SW, encode_name);
            g_object_set (G_OBJECT (bin->encoder), "idrinterval", 60, NULL);
            }
      else{
        bin->encoder =
            gst_element_factory_make (NVDS_ELEM_ENC_H264_HW, encode_name);
            g_object_set (G_OBJECT (bin->encoder), "idrinterval", 60, NULL);}
      break;
    case NV_DS_ENCODER_H265:
      bin->codecparse = gst_element_factory_make ("h265parse", "h265-parser");
      g_object_set (G_OBJECT (bin->codecparse), "config-interval", -1, NULL);
      bin->rtppay = gst_element_factory_make ("rtph265pay", rtppay_name);
      if (config->enc_type == NV_DS_ENCODER_TYPE_SW){
        bin->encoder =
            gst_element_factory_make (NVDS_ELEM_ENC_H265_SW, encode_name);
            g_object_set (G_OBJECT (bin->encoder), "idrinterval", 60, NULL);}
      else{
        bin->encoder =
            gst_element_factory_make (NVDS_ELEM_ENC_H265_HW, encode_name);
            g_object_set (G_OBJECT (bin->encoder), "idrinterval", 60, NULL);}
      break;
    default:
      goto done;
  }

[fanzh]

@emadomari40

(deepstream-app:6284): GLib-GObject-WARNING **: 14:19:01.480: g_object_set_is_valid_property: object class ‘GstX264Enc’ has no property named ‘idrinterval’

you are using x264enc to do software encoding, which has no idrinterval property . x264enc plugin is opensource. please refer to this code.
g_object_set (G_OBJECT (bin->encoder), “key-int-max”, 60, NULL);

[emadomari40]

Thank you so much, your answer was so helpful to overcome the problem.

In addition to that I have changed the GstX264EncPreset to 1 (“speed-preset” [GstX264EncPreset]).