Increasing GStreamer stream framerate

Hey! I’ve been posting often this past few weeks trying to develop this program and I’m finally close to making everything work as intended, but the framerate it’s still a bit lower than I need it to be so I would like some advice improving my pipelines.

The program consists of two pipelines, the first one gets an rtsp h264 stream from a camera, sends it to opencv (the opencv processing is done with pad probe buffer in “myconv”), it get processed there and then sends it through udp-vp8. This pipeline is the following:

rtspsrc location="<<streamip<<" ! queue ! rtph264depay ! video/x-h264, stream-format=byte-stream ! h264parse ! nvv4l2decoder ! nvvidconv name=myconv !  video/x-raw(memory:NVMM), format=RGBA ! nvvidconv name=myconv2 ! video/x-raw(memory:NVMM),width="<< w <<",height="<< h <<", formatt=I420 !  nvv4l2vp8enc maxperf-enable=1 ! video/x-vp8 ! rtpvp8pay ! udpsink host=224.1.1.1 port=5000 sync=false

This part works at 30 stable fps with a 1080p video so I’m happy with it (however, if it could be improved even further I’d be happy with it and might even help the following element.

The next pipeline is where mi headache comes from. This one decodes the udp stream and sends it to a webrtc client upon request. What I need to achieve is keeping this 30 fps (or at least +20) for 2 or 3 webrtcbin clients. The pipeline is the following:

udpsrc multicast-group=224.1.1.1 auto-multicast=true port=5000 ! queue ! application/x-rtp,media=video,clock-rate=90000,encoding-name=VP8,payload=96 ! webrtcbin name=sendrecv

This one works ok-ish (I haven’t managed to measure fps in the clients yet) for 1 client but with 2 or 3 starts to lose a lot of fps (I don’t think it reaches 10 fps for any client). I don’t know if this pipeline or webrtcbin performance can be improved, maybe the best option I have is to increase performance of the first pipeline.

All of this was done in the latest jetpack release, with clocks enabled and with gstreamer 1.16.2 (I know it isn’t officially supported but it made a big improvement in performance when updated so we stuck with it).

Does anyone have any advice for improving my pipelines? Thank you for your help with everything!

Hi,
Please check if it help by tuning buffer-size in udpsrc:

  buffer-size         : Size of the kernel receive buffer in bytes, 0=default
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0

And please run sudo jetson_clocks to get max performance, and sudo tegrastats to show system loading. See if we can catch the bottleneck from the system loading.

I finally managed to make it work properly, apparently the bottleneck was in simulating the camera stream using vlc and a video, when I had a chance to test with the camera we were able to stream stable 4k with that pipeline!

However, the weird error I mentioned in my past post its still there, is there a chance you could help me with that?