Configuring VI/CSI to capture video from a custom camera.

Autonomous Machines Jetson & Embedded Systems Jetson TX1
1

I would like to work with a CSI camera that outputs raw bayer data and use the ISP to convert it to YUV or RGB. I went to the webinar describing libargus and after reviewing it, looking through libargus docs as well as the TRM I still don’t believe I have enough information to interface with my camera.

Inside the TRM in Chapter 31 ‘Video Input (VI)’ there is a paragraph:

This section also describes the VI registers; however, it is not intended to be a programming guide to VI, as it is expected that NVIDIA supplied drivers will always be used with it. However, the register interface is documented to aid with understanding those drivers.

Is there a reference driver I can use to help me understand this? The ov5693.c in ‘/drivers/media/i2c/’ directory doesn’t configure the ISP.

It looks like I’ll need to configure the ISP by setting the 97 some odd registers to correctly route the CSI camera to an ISP and then configure the ISP to convert the image. Is this correct?

Thanks for any help.

Dave

1 Like
2

Hi Cospan
The ISP have IP license to public to general customers. For the CSI/VI you can reference to the vi driver that locate …/kernel/drivers/media/platform/tegra/xxxx .

3

Thanks a lot, I’ll look through it.

It seems as though there are two aspects when interfacing with the VI/ISP.

  1. Device specific configuration: configurations that are camera/platform dependent. Like which CSI lanes a camera is attached to and the number of lanes.

  2. Device independent configuration: configurations that are specific to a capture like the resolution of the image and where to route the images within the VI (ISP or Memory).

It looks like the DTSI is where camera/platform dependent settings should be configured and the camera/platform independent configurations is what libargus is for. Is this true?

Camera/Platform Dependent Configuration

Looking at the DTSI file for the E3326 camera expansion board

/arch/arm64/boot/dts/tegra210-platform/tegra210-camera-e3326-camera-a00.dtsi

which declares the configuration values for the OV5693 camera at a resolution of 2552 X 1944:

...
        mode0 { // OV5693_MODE_2592X1944
          mclk_khz = "24000";
          num_lanes = "2";
          tegra_sinterface = "serial_c";
          discontinuous_clk = "no";
          dpcm_enable = "false";
          cil_settletime = "0";

          active_w = "2592";
          active_h = "1944";
          pixel_t = "bayer_bggr";
          readout_orientation = "180";
          line_length = "2688";
          inherent_gain = "1";
          mclk_multiplier = "17.0";
          pix_clk_hz = "160000000";

          min_gain_val = "1.0";
          max_gain_val = "16";
          min_hdr_ratio = "1";
          max_hdr_ratio = "64";
          min_framerate = "1.816577";
          max_framerate = "30";
          min_exp_time = "34";
          max_exp_time = "550385";
          embedded_metadata_height = "0";
        };
...

It looks like the ‘modes’ within the DTSI file correspond to the sensor settings within:

/drivers/media/i2c/ov5693_mode_tbls.h

Specifically the configuration values found here:

...
static const struct camera_common_frmfmt ov5693_frmfmt[] = {
  {{2592, 1944},  ov5693_30fps, 1, 0, OV5693_MODE_2592X1944},
  {{2592, 1458},  ov5693_30fps, 1, 0, OV5693_MODE_2592X1458},
  {{1920, 1080},  ov5693_30fps, 1, 0, OV5693_MODE_1920X1080},
  {{1280, 720}, ov5693_120fps,  1, 0, OV5693_MODE_1280X720_120FPS},
  {{2592, 1944},  ov5693_24fps, 1, 1, OV5693_MODE_2592X1944_HDR},
  {{1920, 1080},  ov5693_30fps, 1, 1, OV5693_MODE_1920X1080_HDR},
};
...

Where ‘mode0’ within the DTSI file is the ‘mode0’ within this structure. I know the mode table structure is used to configure the actual sensor over I2C and it seems as though the information in a DTSI mode is supposed to be used to configure the VI/ISP for a specific mode. Is this correct?

If the purpose of ‘modeX’ configuration is supposed to be used to configure the camera/platform dependent configurations within the VI/ISP where is it actually executed? If my understand is correct I can start configuring the ISP to read raw bayer data from my camera. I’ve grepped around the kernel and can’t find anywhere that attempts to read things like ‘mclk_khz’ from the dts. Is there a specific file, such as:

/arch/arm64/mach-tegra/board-t210ref-camera.c

or

/driver/media/i2c/CAMERA_NAME.c

Where the DTSI is read from and then used to configure the VI/ISP?

Thanks again for the feedback.

Dave

4

You can reference to the “Sensor Driver Programming Guide” chapter for the detail information.

http://developer.nvidia.com/embedded/dlc/l4t-documentation-24-2

5

I looked at this documentation in the previous version of L4T and it wasn’t as helpful as it is in this version. Thanks for putting work into this. I didn’t think to check it.

Thanks also for your feedback.

6

Hello,

image
image1315×805 136 KB

where can I find this programming guide ?

Maybe this?
image

https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%2520Linux%2520Driver%2520Package%2520Development%2520Guide%2Fcamera_sensor_prog.html

Thank you.