ISP access from v4l2 with CSI camera

I’ve read reports that say that capturing video from a CSI-connected camera will only provide bayer data, and that only using libargus enables the ISP.

I’d prefer to use v4l2, mostly because it has a C (vs C++) interface, which is easier to access via FFI. And, it does seem that when I use libv4l2, I get lots of libargus output.

Before I go all the way down the road of wiring up the video encoder, etc, I’d like to know whether this is the case – can I get YUV420M or NV12M or whatever out of the camera via v4l2, or do I need to use libargus?

(In case it matters, I’m using a partner-supplied camera with driver support already.)

Have a check below doc and reference to the …/jetson_multimedia_api/samples/unittest_samples/

https://docs.nvidia.com/jetson/l4t-multimedia/group__V4L2Argus.html

Thanks @ShaneCCC. I have seen that, couldn’t find it again when referencing for this question. Does that mean, then, that I can use v4l2 entirely without directly using libargus? I do see some ISP/Argus controls in there, so if that’s enough, that would be good news.

If you get frames through CSI from a bayer sensor, then v4l API will give you bayer frames. You would have to debayer by yourself.
CPU debayering would not allow more than a few fps.
You may debayer with GPU, but it would not make such sense as you have ISP HW in your jetson for that. This is controlled by Argus, and some ISP settings that are camera-specific in /var/nvidia/nvcam/settings.

Argus also has an API that can be accessed from C++. This would allow to capture debayered NV12 frames into NVMM memory (contiguous, DMA-able for GPU or other HW blocks for further processing).

@Honey_Patouceul thanks, that’s what I’ve been trying to figure out. Those settings look like they affect the ISP, and they have an ARGUS prefix in the name, which led me to believe it may be possible to get debayered output even via v4l.

I’m familiar with the Argus C++ API; was trying to avoid C++ to make FFI easier (ideally I’d write this in Rust), but if that’s really the only way to get hardware debayering, I can do it if necessary. My original question is whether Argus (directly, via C++) is the only way to achieve that.

My knowledge is limited, and I don’t know any other way than Argus for using ISP.
[Additional note: when using Argus, it sets bypass_mode to 1 in V4L controls so that frames go thru ISP. If playing with V4L API for getting bayer frames, be sure to set control bypass_mode to 0, otherwise this might fool argus].

@kevin.r
Current the v4l2 API support Argus so you can use v4l2 API like reference code to open Argus camera.

@ShaneCCC thanks, that’s what I was wondering. Now I’m getting output that needs debayering; I’ll ask about that in another thread.