Nvgstcapture vs nvarguscamerasrc

Hello!

I bought a CSI camera, IMX219, for my OpenCV project.

When I run the below command, there seems to be no delay at all in showing the frames in realtime.
$ nvgstcapture-1.0 -m 2 --prev-res 4

However, when I run my simple python code using the below pipleline, capturing is significantly slow,

pipeline = 'nvarguscamerasrc !  video/x-raw(memory:NVMM), width=1920, height=1080, format=NV12, framerate=30/1 ! nvvidconv flip-method=0 ! video/x-raw, width=1920, height=1080, format=BGRx ! videoconvert ! video/x-raw, format=BGR ! appsink'

cap = cv2.VideoCapture(pipeline, cv2.CAP_GSTREAMER)

What should I do if I need 1920x1080, 30fps VideoCapture() in Opencv?

Appreciate your help!

The issue is that, unless somebody knows something I don’t, Nvidia does not provide an accelerated way to convert to BGR format, and OpenCV videoCapture does not accept BGRA, which Nvidia does support. This means you have to use videoconvert, which is very slow. You could see if NV12/YUV format suits your needs, since I think videoCapture does support that, and nvvidconv can convert to it.

If you need frames captured low latency to OpenCV, you could use libargus directly, and map a cuda EGL frame to a GpuMat.

I’m working on a project to do just that, but it’s C++ only for now, and I wouldn’t use it in production code just yet.

https://github.com/mdegans/nvcvcam/

(See initial branch)

I probably won’t work on that for a few weeks since I have some other stuff to do, but contributions are welcome. Currently you can capture a raw bayer frame to a GpuMat and debayer it with cv::cuda::demosaicing, (or dump it) but that’s about it.

With minimal modification, debayering could be done by the ISP using libargus itself, which is much faster. If you’re interested in this, please file an issue with a feature request. I will prioritize work based on how many people are interested.

There’s also the camera within jetson utils that can get you a RGBA image pretty effeciently, and that can be used with Python and OpenCV.

https://github.com/dusty-nv/jetson-utils

That’s probably the easiest for your purposes.

1 Like

Sounds amazing! Since my project depends on image processing, not DL, I’m very interested in accelerating image arithmetic operations, such as frame difference. I’ll try NV12/YUV and your project and share if it solves the problem. Thank you so much!!!

You’re welcome! Please report any bugs, and make sure to check out jetson_utils as well, since that has more testing.