Currently, I’m using a Blackfly S USB camera from FLIR (genicam). Instead of using the Spinnaker SDK with the manufacturer, I have instead install Aravis. It allows for the use of genicam cameras with Gstreamer as a plugin.
My usecase is pretty straight forward. I just want to record at 60 FPS at 3072 x 2048 color images onto an Nvidia Jetson AGX over a period of 2 hours. My gstreamer pipeline so far is:
gst-launch-1.0 aravissrc ! video/x-bayer,width=3072,height=2048, framerate=60/1 ! bayer2rgb ! nvvidconv ! nvv4l2h265enc ! video/x-h265, streamformat=byte-stream ! h265parse ! qtmux ! filesink location=vid.mp4 -e
However, I can’t get 60 FPS with that pipeline. The output is around ~23 FPS. The biggest bottle neck is bayer2rgb in the pipeline since it’s performed using CPU. What would be the suggested route of action for improving performance? Does there exist a CUDA implementation of bayer2rgb as a gstreamer plugin?
I initially tried to use the Spinnaker SDK of the manufacturer to acquire images, process with OpenCV, and then encode the output to file (using the Jetson Multimedia API as an example). However, implementation was very challenging, and I encountered a lot of bugs in memory management to take advantage of Jetson Unified Memory. Took a step back and found that using gstreamer was much easier.