Porting max9288(SER/DES) and CMOS Camera[gray-scale sensor] driver to TX2 R28.1

Hello,
We have developed cmos gray-scale sensor driver for TX2, based on the NVidia’s file imx274.c
The driver works is in progress in L4T 28.1
Now we have ported the driver to L4T 28.1, observing the recommendations in the documentation.
(Jetson TX2 Camera BringUp - eLinux.org)

firstly, I need to explain my camera system to get the exact information.
My system is as below.
CMOS Sesor Camera --(LVDS)—> MAX9288 --(CSI2)–> Tegra CSI port
CMSO Sensor Camera is LVDS camera and gray-scale camera. so I used lattice MAX9288 to convert LVDS to CSI.

Second,
Due to camera driver changes, the imx274 kernel module driver only raises the module, and the actual driver initialization is handled in the user area.

  1. New camera and csi setting dtsi script : platform/t18x/quill/kernel-dts/guill-modules/tegra186-camera-imx274.dtsi
    i2c@3180000 {
    imx274_a@1a {
    compatible = “nvidia,imx274”;
    /* I2C device address */
    reg = <0x1a>;

     	/* V4L2 device node location */
     	devnode = "video0";
    
     	/* Physical dimensions of sensor */
     	physical_w = "3.674";
     	physical_h = "2.738";
    
     	sensor_model ="imx274";
                  
                     mode0 {
     		mclk_khz = "24000";
     		num_lanes = "4";
     		tegra_sinterface = "serial_a";
     		discontinuous_clk = "yes";
     		dpcm_enable = "false";
     		cil_settletime = "0";
    
     		active_w = "3840";
     		active_h = "2160";
     		[b]pixel_t = "raw_uyvy"; # gray format[/b]
     		readout_orientation = "90";
     		line_length = "4208";
     		inherent_gain = "1";
     		mclk_multiplier = "24";
     		pix_clk_hz = "576000000";
    
     		min_gain_val = "1.0";
     		max_gain_val = "16.0";
     		min_hdr_ratio = "1";
     		max_hdr_ratio = "1";
     		min_framerate = "1.5";
     		max_framerate = "60";
     		min_exp_time = "16.165";
     		max_exp_time = "666637";
     		embedded_metadata_height = "1";
     	};
    

our cmos camera is a gray-scale sensor (1280x800).
We have added support for 8-bit and 10-bit gray formats “raw_uyvy” in the following kernel files:

  1. New media bus format(s) in camera_common.c:
    static const struct camera_common_colorfmt camera_common_color_fmts = {
    [b]// add
    {
    MEDIA_BUS_FMT_UYVY8_2X8,
    V4L2_COLORSPACE_JPEG,
    V4L2_PIX_FMT_UYVY,
    },
    {
    MEDIA_BUS_FMT_VYUY8_2X8,
    V4L2_COLORSPACE_JPEG,
    V4L2_PIX_FMT_VYUY,
    },

    {
    MEDIA_BUS_FMT_YUYV8_2X8,
    V4L2_COLORSPACE_JPEG,
    V4L2_PIX_FMT_YUYV,
    },

    {
    MEDIA_BUS_FMT_YVYU8_2X8,
    V4L2_COLORSPACE_JPEG,
    V4L2_PIX_FMT_YVYU,
    },

    {
    MEDIA_BUS_FMT_UYVY8_1X16,
    V4L2_COLORSPACE_JPEG,
    V4L2_PIX_FMT_UYVY,
    },

    {
    MEDIA_BUS_FMT_VYUY8_1X16,
    V4L2_COLORSPACE_JPEG,
    V4L2_PIX_FMT_VYUY,
    },

    {
    MEDIA_BUS_FMT_YUYV8_1X16,
    V4L2_COLORSPACE_JPEG,
    V4L2_PIX_FMT_YUYV,
    },

    {
    MEDIA_BUS_FMT_YVYU8_1X16,
    V4L2_COLORSPACE_JPEG,
    V4L2_PIX_FMT_YVYU,
    }, [/b]
    }

  2. New device-tree pixel formats in sensor_common.c:
    static int extract_pixel_format(
    const char *pixel_t, u32 *format)
    {
    size_t size = strnlen(pixel_t, OF_MAX_STR_LEN);

    if (strncmp(pixel_t, “bayer_bggr10”, size) == 0)
    *format = V4L2_PIX_FMT_SBGGR10;
    else if (strncmp(pixel_t, “bayer_rggb10”, size) == 0)
    *format = V4L2_PIX_FMT_SRGGB10;
    else if (strncmp(pixel_t, “bayer_bggr12”, size) == 0)
    *format = V4L2_PIX_FMT_SBGGR12;
    else if (strncmp(pixel_t, “bayer_rggb12”, size) == 0)
    *format = V4L2_PIX_FMT_SRGGB12;
    else if (strncmp(pixel_t, “bayer_wdr_pwl_rggb12”, size) == 0)
    *format = V4L2_PIX_FMT_SRGGB12;
    else if (strncmp(pixel_t, “bayer_xbggr10p”, size) == 0)
    *format = V4L2_PIX_FMT_XBGGR10P;
    else if (strncmp(pixel_t, “bayer_xrggb10p”, size) == 0)
    *format = V4L2_PIX_FMT_XRGGB10P;
    else if (strncmp(pixel_t, “raw_uyvy”, size) == 0)// add
    *format = V4L2_PIX_FMT_UYVY;

    else {
    pr_err(“%s: Need to extend format%s\n”, func, pixel_t);
    return -EINVAL;
    }

    return 0;
    }

int camera_common_try_fmt(struct v4l2_subdev *sd, struct v4l2_mbus_framefmt mf)
{
if (i == s_data->numfmts) {
// org, disabled by shchoi
/

mf->width = s_data->fmt_width;
mf->height = s_data->fmt_height;
dev_dbg(&client->dev,
“%s: invalid resolution supplied to set mode %d %d\n”,
func, mf->width, mf->height);
*/
// add
s_data->fmt_width = mf->width;
s_data->fmt_height = mf->height;

		goto verify_code;
	}

}

Attachments - RAW8 Gray Capture image
Screenshot from 2019-08-07 17-21-54.png

hello scs2.dxavier,

  1. you might check the Jetson TX2 Series Software Features, and virtual channel support should be include in l4t-r32.x release.
    however, it seems you already port the driver.

  2. may I know your commands to launch camera sensors?
    since l4t-r28.x won’t support monochrome sensors. you could only access it with v4l2 standard controls as alternative way.

Thanks for JerryChang,

When recording raw8 image(gray), i use the v4l2 open source called yavta and play it with mplayer.

I will test and capture image…

It feels like pixels are missing. Not supported by hump r28.x, these phenomena come out?

1.record of the 8 bit
./yavta /dev/video0 -c600 -n32 -s1280x799 -fSRG8 “-F/media/nvme/video_$(date ‘+%y%m%d_%H%M%S’)1280x799${BITS}b_10sec.raw”

  1. play of the file
    mplayer -demuxer rawvideo -rawvideo w=1280:h=799:format=$y8:fps=60 “$1”

  2. video0 of the info
    nvidia@tegra-ubuntu:~$ v4l2-ctl --all -d 0
    Driver Info (not using libv4l2):
    Driver name : tegra-video
    Card type : vi-output, imx274 2-001a
    Bus info : platform:15700000.vi:0
    Driver version: 4.4.38
    Capabilities : 0x84200001
    Video Capture
    Streaming
    Extended Pix Format
    Device Capabilities
    Device Caps : 0x04200001
    Video Capture
    Streaming
    Extended Pix Format
    Priority: 2
    Video input : 0 (Camera 0: no power)
    Format Video Capture:
    Width/Height : 1280/799
    Pixel Format : ‘RGGB’
    Field : None
    Bytes per Line : 2560
    Size Image : 2045440
    Colorspace : sRGB
    Transfer Function : Default
    YCbCr Encoding : Default
    Quantization : Default
    Flags

  3. capture image

hello scs2.dxavier,

here’s another FYI,
you should modify the resolution settings since hardware engine didn’t support odd values.

Width/Height : 1280/799