Gstreamer with GPU implementation of bayer2rgb

Currently, I’m using a Blackfly S USB camera from FLIR (genicam). Instead of using the Spinnaker SDK with the manufacturer, I have instead install Aravis. It allows for the use of genicam cameras with Gstreamer as a plugin.

My usecase is pretty straight forward. I just want to record at 60 FPS at 3072 x 2048 color images onto an Nvidia Jetson AGX over a period of 2 hours. My gstreamer pipeline so far is:

gst-launch-1.0 aravissrc ! video/x-bayer,width=3072,height=2048, framerate=60/1 ! bayer2rgb ! nvvidconv ! nvv4l2h265enc ! video/x-h265, streamformat=byte-stream ! h265parse ! qtmux ! filesink location=vid.mp4 -e

However, I can’t get 60 FPS with that pipeline. The output is around ~23 FPS. The biggest bottle neck is bayer2rgb in the pipeline since it’s performed using CPU. What would be the suggested route of action for improving performance? Does there exist a CUDA implementation of bayer2rgb as a gstreamer plugin?

I initially tried to use the Spinnaker SDK of the manufacturer to acquire images, process with OpenCV, and then encode the output to file (using the Jetson Multimedia API as an example). However, implementation was very challenging, and I encountered a lot of bugs in memory management to take advantage of Jetson Unified Memory. Took a step back and found that using gstreamer was much easier.

There is no existing implementation for bayer2rgb in CUDA. You would need to refer to code of bayer2rgb plugin and implement it through CUDA. Or may see if other users have the code and can share.

I ended up finding a gstreamer plugin that does debayer better by The Imaging Resource.

The following gstreamer pipline is working at 60 fps now.

gst-launch-1.0 aravissrc ! video/x-bayer,width=3072,height=2048,framerate=60/1 ! tcamconvert ! nvvidconv ! nvv4l2h265enc ! video/x-h265, streamformat=byte-stream ! h265parse ! qtmux ! filesink location=testvid.mp4 -e

They also have a closed source gstreamer plugin that explicitly uses cuda for bayer (tcamdutils-cuda), but I didn’t need it for 60 fps.

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.