I am taking a udpsrc, processing it with a custom gstreamer element called qtoverlay, and then sending it to a udpsink to stream out via rtsp.
Pipeline A (udpsrc -->qtoverlay -->nvvidconv (NVMM) → nvvidconv (raw) → omxh264enc–>udpsink) works
gst-launch-1.0 udpsrc port=5007 buffer-size=60000000 ! application/x-rtp, media=video, clock-rate=90000, encoding-name=H265,playload=96 ! queue ! rtph265depay ! h265parse ! video/x-h265,alignment=au ! nvv4l2decoder disable-dpb=true enable-max-performance=1 ! queue max-size-bytes=0 ! videorate max-rate=30 ! queue ! nvvidconv ! video/x-raw,width=1920,height=1080,framerate=30/1 ! qtoverlay qml=main.qml ! tee name=ovl ovl. ! nvvidconv ! ‘video/x-raw, format=I420,width=1920,height=1080’ ! omxh264enc ! video/x-h264,stream-format=byte-stream ! h264parse ! rtph264pay pt=96 ! tee name=ovl1 ovl1. ! udpsink host=127.0.0.1 port=10006 sync=false async=false
Pipeline B (udpsrc -->qtoverlay -->nvvidconv (raw) → omxh264enc–>udpsink) runs without throwing an error, but I see nothing on the vlc player. In fact, when I run ‘top’ on linux, I don’t even see the process running.
gst-launch-1.0 udpsrc port=5007 buffer-size=60000000 ! application/x-rtp, media=video, clock-rate=90000, encoding-name=H265,playload=96 ! queue ! rtph265depay ! h265parse ! video/x-h265,alignment=au ! nvv4l2decoder disable-dpb=true enable-max-performance=1 ! queue max-size-bytes=0 ! videorate max-rate=30 ! queue ! nvvidconv ! video/x-raw,width=1920,height=1080,framerate=30/1 ! qtoverlay qml=main.qml ! tee name=ovl ovl. ! nvvidconv ! ‘video/x-raw(memory:NVMM), format=I420,width=1920,height=1080’ ! nvvidconv ! ‘video/x-raw’ ! omxh264enc ! video/x-h264,stream-format=byte-stream ! h264parse ! rtph264pay pt=96 ! tee name=ovl1 ovl1. ! udpsink host=127.0.0.1 port=10006 sync=false async=false
In pipeline B, I could replace nvvidconv with videoconvert and then the videostream will work. But this uses up a lot of cpu. Is there something wrong with nvvidconv?
Also, I convert to video/x-raw even though omxh264enc support NVMM because if I run more than 2 instances of pipeline B without converting it to video/x-raw, the video on vlc begins to tear. I increment udscr and udpsink port for each pipeline, so it’s not a problem of the src and sink stomping on each other. This is another issue that I was going to ask for help on in a seperate thread.
Note that although this is not explicit looking at SRC and SINK caps, nvvidconv expects at least one of its input or output to be in NVMM memory (i.e. x-raw to x-raw is not supported, AFAIK).
Furthermore, there an issue with h264 parsing with nvv4l2decoder, where you may have to remove h264parse before decoder.
The answer to my streaming corruption problem was nvv4l2h264enc.
When I initially replaced omxh264enc with the nvv4l2h264enc element, I saw no stream at all. The answer to that problem was found in the post I’m linking below. I had to enable insert-sps-pps. AND I had to remove the h264parse element, just as Honey_Patouceul had suggested.
Even though I can now handle 4 streams without video tearing, the video is a bit choppy. As if it is dropping frames. This choppiness goes away if I replace nvv4l2h264enc with omxh264enc. (But if I use omxh264enc, I can’t run more than 2 pipelines before experiencing tearing )
Here is my bare-bones pipeline. Is there an additional setting i could use to smooth out the video?