Hey! I’ve been posting often this past few weeks trying to develop this program and I’m finally close to making everything work as intended, but the framerate it’s still a bit lower than I need it to be so I would like some advice improving my pipelines.
The program consists of two pipelines, the first one gets an rtsp h264 stream from a camera, sends it to opencv (the opencv processing is done with pad probe buffer in “myconv”), it get processed there and then sends it through udp-vp8. This pipeline is the following:
rtspsrc location="<<streamip<<" ! queue ! rtph264depay ! video/x-h264, stream-format=byte-stream ! h264parse ! nvv4l2decoder ! nvvidconv name=myconv ! video/x-raw(memory:NVMM), format=RGBA ! nvvidconv name=myconv2 ! video/x-raw(memory:NVMM),width="<< w <<",height="<< h <<", formatt=I420 ! nvv4l2vp8enc maxperf-enable=1 ! video/x-vp8 ! rtpvp8pay ! udpsink host=22.214.171.124 port=5000 sync=false
This part works at 30 stable fps with a 1080p video so I’m happy with it (however, if it could be improved even further I’d be happy with it and might even help the following element.
The next pipeline is where mi headache comes from. This one decodes the udp stream and sends it to a webrtc client upon request. What I need to achieve is keeping this 30 fps (or at least +20) for 2 or 3 webrtcbin clients. The pipeline is the following:
udpsrc multicast-group=126.96.36.199 auto-multicast=true port=5000 ! queue ! application/x-rtp,media=video,clock-rate=90000,encoding-name=VP8,payload=96 ! webrtcbin name=sendrecv
This one works ok-ish (I haven’t managed to measure fps in the clients yet) for 1 client but with 2 or 3 starts to lose a lot of fps (I don’t think it reaches 10 fps for any client). I don’t know if this pipeline or webrtcbin performance can be improved, maybe the best option I have is to increase performance of the first pipeline.
All of this was done in the latest jetpack release, with clocks enabled and with gstreamer 1.16.2 (I know it isn’t officially supported but it made a big improvement in performance when updated so we stuck with it).
Does anyone have any advice for improving my pipelines? Thank you for your help with everything!