Access GMSL video stream with OpenCV

Hello,

I am trying to access the video from the GMSL camera I connected to port A0 on the Drive PX2 (P2379).
A Sekonix Camera is used (120° FOV). It works via the ‘sample_camera_gmsl’ and also via ‘./nvmimg_cap -cf drive-px2-a.conf -c dvp-ar0231-rccb-raw12-1920x1208-ab -d 0 -w 1 --aggregate 1’ very well.
I also tried to access the video stream via gstreamer but had no success.

I would like to use the video stream in OpenCV with VideoCapture() but without the Driveworks Software.
Is there a possibility to do that?

On my laptop it works with VideoCapture(0) but that is a much simpler setup.

We are a team of students working on an autonomous racecar, that has to perform a cone detection, therefore we are using OpenCV and ROS.

Thank you

Dear christoph.wald,

Regarding OpenCV, please refer to https://devtalk.nvidia.com/default/topic/1019937/faq/opencv/ Thanks.

Thanks Steve,

OpenCV works fine on the Drive PX, but as soon as we want to access the GMSL Cameras we have to use the Driveworks SDK. Therefore we have to cross-compile our code.
Integrating OpenCV code into the Driveworks software and compiling on host works fine.

What is the procedure to integrate OpenCV in Driveworks and then cross-compile?

Is there any way to access the GMSL cameras without the Driveworks Software?
Driveworks object detection only supports Tensor Flow models but we only need simple algorithms, therefore we would like to use OpenCV.

Thank you.

Hello Christoph,

I’m currently trying to make the OpenCV work with the GMSL cameras too, but I can’t find documentation of how to connect Driveworks and OpenCV together. Would you mind sharing some insights about where should I look for?

Thank you in advance.

Dear esfabian,

NVIDIA provide nvmedia for Multimedia programming.
So could you please try it for your topic? Thanks.

http://docs.nvidia.com/drive/nvvib_docs/index.html#page/NVIDIA%20DRIVE%20Linux%20SDK%20Development%20Guide%2FMultimedia%2Fnvmedia_sample_apps.html%23

Hi esfabian,

It is definetly possible to display the Sekonix GMSL camera in Ros.
We found that Autoware provides such a node.
But the problem is that the fps and video format are hardcoded and we don’t know how to change that.
We are also not able to take a look into the code because it’s a library.

We are using a USB camera as a backup right now, but we are looking further into accessing the GMSL cameras.

Hey guys, is there any updates? I am working on it these days. Do we have to fuse Nvidia Drive SDK APIs with OpenCV to make it work or is there any other way around?

Dear christoph.wald & infohyu7s,

Please refer to below link for your topic. Thanks.

https://devtalk.nvidia.com/default/topic/1032772/faq/cross-compiling-opencv-3-4-0-on-pdk-5-0-5-0/