Compositor with RTSP sources, and RTP streaming

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

Hello all,

I have an issue with the following script:

gst-launch-1.0 nvcompositor name=comp operator=source
sink_0::alpha=1 sink_0::width=1920 sink_0::height=840
sink_1::alpha=1 sink_1::xpos=0 sink_1::ypos=728 sink_1::width=480 sink_1::height=240
sink_2::alpha=1 sink_2::xpos=480 sink_2::ypos=728 sink_2::width=480 sink_2::height=240
sink_3::alpha=1 sink_3::xpos=960 sink_3::ypos=728 sink_3::width=480 sink_3::height=240
sink_4::alpha=1 sink_4::xpos=1440 sink_4::ypos=728 sink_4::width=480 sink_4::height=240

! nvvidconv ! omxh264enc bitrate=3000000 control-rate=4 qos=true !
rtph264pay config-interval=1 pt=96 ! udpsink host=192.168.0.123 port=5001 async=false sync=false

rtspsrc location=“rtsp://@192.168.0.21/onvif-media/media.amp?camera=1" latency=10 !
rtph264depay ! nvv4l2decoder enable-max-performance=1 ! queue2 ! comp.sink_0

rtspsrc location="rtsp://@192.168.0.21/onvif-media/media.amp?camera=1” latency=10 !
rtph264depay ! nvv4l2decoder enable-max-performance=1 ! queue2 ! comp.sink_1

rtspsrc location=“rtsp://@192.168.0.21/onvif-media/media.amp?camera=2" latency=10 !
rtph264depay ! nvv4l2decoder enable-max-performance=1 ! queue2 ! comp.sink_2

rtspsrc location="rtsp://@192.168.0.21/onvif-media/media.amp?camera=3” latency=10 !
rtph264depay ! nvv4l2decoder enable-max-performance=1 ! queue2 ! comp.sink_3

rtspsrc location=“rtsp://****@192.168.0.21/onvif-media/media.amp?camera=4” latency=10 !
rtph264depay ! nvv4l2decoder enable-max-performance=1 ! queue2 ! comp.sink_4

Basically, it can work sometimes (1 on 10 times), but most of time I just got a freezed picture with only one or two sub-stream displayed. This is the NVIDIA version but I have exactly the same issue with standard avdec_h264 and v4l2h264enc plugins.
I suspect a sync issue, but not sure. If I try this script with 5 videotestsrc, it works perfectly. Same thing with a direct display of the compositor result (no encoding / streaming), with nvoverlaysink. It works also all the time.

I’m lost right now, no idea of what I can do / try to fix it.
If anyone has any idea, it would be great !

Thanks a lot,

Mael

3

Hi,
Does it work with nvoverlaysink sync=false? And we are deprecating omx plugins. Please try nvv4l2h264enc.

4

Hi,

Thanks for answering. Actually I also tried nvv4l2*enc, with h264, vp8 and vp9. Same issue. I tried also with nvoverlaysink synk=false and there it worked.
I add GST_DEBUG=3 to get more information, and saw that, when the pipe get stuck, I had the error:
gst_encoder_setcaps: rejected caps video/x-raw(memory:NVMM), width=(int)1920, height=(int)968, framerate=(fraction)25/1, format=(string)I420
It seems to be my first sink, even if the size is different, or maybe the overall compositor output ? Don’t know, but the mistake should be hidden there…

5

Hi,
Could you try with test-mp4 lunched RTSP server?

$ ./test-mp4 sample_1080p_h264.mp4

gst-rtsp-server/test-mp4.c at master · GStreamer/gst-rtsp-server · GitHub

Probably it is specific to your RTSP sources. Would like to clarify this.

6

Hi,

I tried but I get the same issue, 9 times on 10 I get stuck stream with one or two of the pictures, and black screen for the rest. If you want to reproduce, my receiver script:

gst-launch-1.0 udpsrc port=5001 ! application/x-rtp, encoding-name=H264, payload=96 ! rtph264depay ! avdec_h264 ! videoconvert ! xvimagesink

Thanks

7

Not sure, but I’ve seen cases where omxh264enc was failing with rtsp input using excessively low latency.
You may try :

  • adding h264parse between omxh264enc and rtph264pay
  • increasing rtsp latency (say up to 500 ms)
  • use nvv4l2h264enc instead of omxh264enc
1 Like
8

Thank you Honey_Patouceul (j’adore le pseudo ;-) ), that was the latency the issue. I raised to 100 ms (no change) then 500 ms and now it works at all times !!
The best is that I don’t see the 500 ms in the overall latency, it seems to be even more quick…
It’s weird, but it’s ok

9

Nice to hear you’ve catched it ;-) For fun.

Maybe nvvv4l2h264enc could provide better performance and better behave with low latency specified stalling.
Good luck !