OpenCV VideoCapture of RAW data

Using nvarguscamerasrc can use hardware ISP engine. If you don’t go this route, would need to implement debayering function.

A user has shared a solution. You can check and see if it can be applied to your use-case:
Gstreamer with GPU implementation of bayer2rgb - #4 by thanhnha