Cropping image sensor frames at the VI/ISP stage


I’m trying to optimize performance in a media pipeline of mine. Using 2x IMX334 sensors which natively output 3864x2180 frames. Ideally I want to use 3840x2160 in my Gstreamer pipeline, because that’s what the datasheet of the sensor recommends.

The driver I’m using gets the full frame from the sensor, which I believe is the right thing to do, for color tuning at the ISP stage, however for the output frame, I’m supposed to crop 12px away from left/right and 12px/8px from top/bottom, according to the datasheet.

I know I can do that with nvvidconv and friends, but that puts additional load on the VIC, which I want to avoid.

I’ve tried:

  • modifying the V4L camera_common_frmfmt definition to reflect the 3840x2160 format
  • modifying the active_w and active_h properties in the mode definitions in the device tree
  • a combination thereof

That all results in frames the right size, but with visible artifacts (no wonder!)

The documentation mentions device tree properties named such as num_of_left_margin_pixels which sounds exactly like what I need! Defining the number of pixels to disregard on either edge of the frame, however that seems to be only related for DOL captures, which I’m not using.

Would I be able to use the DOL things, even though I’m not doing HDR captures at present?

That only for DOL HDR mode.
Suppose sensor should be able to configure the size after crop? If didn’t nvvidconv are the right solution for it.

No, the idea was to get the full 3864x2180 frame from the sensor and use all that for auto exposure etc, but then crop the frame afterwards so any downstream software using LibArgus would get a 3840x2160 frame.

Just configure the output stream as 3840x2160 then Argus will help to crop/scale it.

I specifically want to avoid scaling because the VIC is already fully loaded. I want to configure cropping, ideally specifying the exact window to crop before even creating the BufSurface, if possible.

Not sure if current Argus crop is using VIC or GPU. Maybe need to profile it to confirm it.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.