I’m quite new to gstreamer. Currently, I’m working on using gstreamer to encode H264 video with additional processings to the frames programmatically. My pipeline almost likes the one used in the gstVideoEncode sample:
My question is how I can add a callback func in the middle to modify and update the frames passing by? Which gst element should I use? If I can achieve this, how can I cast the gst type into NvBufSurface for further CUDA processing, and then wrap to gst type and pass to the down stream?
Hi,
You can insert nvvidconv between nveglstreamsrc and nvv4l2h264enc. And customize nvvidconv plugin for the use-case. The source code of plugins is in
Thanks for your suggestion. I was thinking whether there is any other method that I don’t have to modify the gst source code, and simply use the existing gst element with customized callback function to achieve this?
You can add it in sink pad of nvv4l2h264enc plugin. However, certain users mention that when it takes some time in prob function, the whole pipeline may get slow down. That is why we would suggest customize nvvidconv plugin.
Thanks Dane, I’m now able to probe the frame and do some processing on it. Now I encounter another problem while im doing the decoding.
The video I was recording is in 4K60FPS, my decode pipeline is: gst-launch-1.0 filesrc location=/home/jnx42/testing.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! videorate ! 'video/x-raw(memory:NVMM),framerate=60/1' ! nv3dsink
However, when I translate it into c++ code, the behavior is not as what I expected. The decoded frames seem too fast and is over 60FPS, even if I put the videorate element behind the decoder. Please find the attached example code for your reference to reproduce. Thanks if you could give any further suggestion on how can I make sure the decoded video streaming in accurate speed.