Hi Team,
We have a Xavier NX Devkit connected to a board generating CSI data. The imx219 sensor driver is being modified to capture the data from the csi data generator board. We would like to capture RAW8, RAW12, RAW14, RAW16 data formats. At present we could capture only RAW8 data format and we are unable to capture RAW12 and RAW16 data formats. Kindly guide us to make necessary changes in the driver code to support the RAW12, RAW14 and RAW16 formats.
Regards,
Shiva Shankar K.
Do you use v4l2-ctl to capture RAW? If you want to support multi pixel format for single driver you need to modify your sensor driver for it.
Thank you @ShaneCCC.
To support raw12 I have modified as below :
In Linux_for_Tegra/sources/kernel/nvidia/drivers/media/platform/tegra/camera/camera_common.c added
{
MEDIA_BUS_FMT_SBGGR12_1X12,
V4L2_COLORSPACE_SRGB,
V4L2_PIX_FMT_SBGGR12,
},
Changed the device tree file in Linux_for_Tegra/sources/hardware/nvidia/platform/t19x/jakku/kernel-dts/common/tegra194-camera-rbpcv2-imx219.dtsi
active_w = “2048”;
active_h = “2048”;
pixel_t = “bayer_bggr12”;
line_length = “3300”;
pix_clk_hz = “155000000”;
min_framerate = “1000000”; /* 1.0 fps /
max_framerate = “15000000”; / 15.0 fps /
step_framerate = “1”;
default_framerate = “15000000”; / 15.0 fps */
tcp: tegra-camera-platform {
num_csi_lanes = <4>;
max_lane_speed = <1860000>;
min_bits_per_pixel = <12>;
Since the support for raw12 exists in static const struct tegra_video_format vi2_video_formats structure in
nvidia/drivers/media/platform/tegra/camera/vi/vi2_formats.h and
nvidia/drivers/media/platform/tegra/camera/vi/vi5_formats.h, I didn’t add any thing in these files.
By the way we are using both v4l2-ctl and the gst-launch-1.0 commands
a) v4l2-ctl -d /dev/video0 --set-fmt-video=width=2048,height=2048,pixelformat=BA81 --stream-count=1 --stream-mmap --stream-to=csi.raw --verbose
b) gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=1 ! ‘video/x-bayer,format=bggr,width=2048,height=2048’ ! filesink location=image.raw
c) gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=1 ! ‘video/x-bayer,format=bggr,width=2048,height=2048’ ! bayer2rgb ! jpegenc ! filesink location=image.jpg
gst-launch-1.0 gives error message as below
"ERROR: from element /GstPipeline: pipeline0/GstV4l2Src:v4l2src0: Internal data stream error"
Suppose v4l2src don’t support Bayer RAW format.
gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=1 ! ‘video/x-bayer,format=bggr,width=2048,height=2048’ ! filesink location=image.raw
AFAIK, gstreamer may not be able to handle not byte-aligned bayer formats, so bayer8 or bayer16 might be your only choices here.
Not sure if more recent gstreamer versions than 1.14.5 may provide more for this case.
Thank you. Could you please tell me the places I need to make changes to add bayer16 support in the driver, pls refer the changes I made to add bayer 8 at start of this conversation. The thing is bayer8 support is already present in static const struct tegra_video_format vi2_video_formats structure in
nvidia/drivers/media/platform/tegra/camera/vi/vi2_formats.h and
nvidia/drivers/media/platform/tegra/camera/vi/vi5_formats.h files. The below is the code snippet from the vi5_formats.h file
static const struct tegra_video_format vi5_video_formats = {
/* RAW 6: TODO */
/* RAW 7: TODO */
/* RAW 8 */
TEGRA_VIDEO_FORMAT(RAW8, 8, SRGGB8_1X8, 1, 1, T_R8,
RAW8, SRGGB8, "RGRG.. GBGB.."),
TEGRA_VIDEO_FORMAT(RAW8, 8, SGRBG8_1X8, 1, 1, T_R8,
RAW8, SGRBG8, "GRGR.. BGBG.."),
TEGRA_VIDEO_FORMAT(RAW8, 8, SGBRG8_1X8, 1, 1, T_R8,
RAW8, SGBRG8, "GBGB.. RGRG.."),
TEGRA_VIDEO_FORMAT(RAW8, 8, SBGGR8_1X8, 1, 1, T_R8,
RAW8, SBGGR8, "BGBG.. GRGR.."),
/* RAW 10 */
TEGRA_VIDEO_FORMAT(RAW10, 10, SRGGB10_1X10, 2, 1, T_R16,
RAW10, SRGGB10, "RGRG.. GBGB.."),
TEGRA_VIDEO_FORMAT(RAW10, 10, SGRBG10_1X10, 2, 1, T_R16,
RAW10, SGRBG10, "GRGR.. BGBG.."),
TEGRA_VIDEO_FORMAT(RAW10, 10, SGBRG10_1X10, 2, 1, T_R16,
RAW10, SGBRG10, "GBGB.. RGRG.."),
TEGRA_VIDEO_FORMAT(RAW10, 10, SBGGR10_1X10, 2, 1, T_R16,
RAW10, SBGGR10, "BGBG.. GRGR.."),
/* RAW 12 */
TEGRA_VIDEO_FORMAT(RAW12, 12, SRGGB12_1X12, 2, 1, T_R16,
RAW12, SRGGB12, "RGRG.. GBGB.."),
TEGRA_VIDEO_FORMAT(RAW12, 12, SGRBG12_1X12, 2, 1, T_R16,
RAW12, SGRBG12, "GRGR.. BGBG.."),
TEGRA_VIDEO_FORMAT(RAW12, 12, SGBRG12_1X12, 2, 1, T_R16,
RAW12, SGBRG12, "GBGB.. RGRG.."),
TEGRA_VIDEO_FORMAT(RAW12, 12, SBGGR12_1X12, 2, 1, T_R16,
RAW12, SBGGR12, "BGBG.. GRGR.."),
/* RGB888 */
TEGRA_VIDEO_FORMAT(RGB888, 24, RGB888_1X24, 4, 1, T_A8R8G8B8,
RGB888, ABGR32, "BGRA-8-8-8-8"),
TEGRA_VIDEO_FORMAT(RGB888, 24, RGB888_1X32_PADHI, 4, 1, T_A8B8G8R8,
RGB888, RGB32, "RGB-8-8-8-8"),
The macro expansion for TEGRA_VIDEO_FORMAT is
#define TEGRA_VIDEO_FORMAT (VF_CODE, BPP, MBUS_CODE, FRAC_BPP_NUM, FRAC_BPP_DEN, FORMAT, DATA_TYPE, FOURCC, DESCRIPTION)
In order to add support RAW16 a new line has to be added and please verify if the below args can be used :
VF_CODE = RAW16
BPP = 16
MBUS_CODE = SBGGR16_1X16
FRAC_BPP_NUM = ?? ( probably it is 2)
FRAC_BPP_DEN = ?? ( probably it is 1)
FORMAT = ?? (T_R16 or T_R16_I )
DATA_TYPE = RAW16
FOURCC = SBGGR16
Regards,
Shiva Shankar K.
You can check with v4l2-ctl --list-formats-ext for your modification.
Also using v4l2-ctl --steam-mmap -c bypas_mode=0 to check if can capture successfully for it.
Thank you @ShaneCCC and @Honey_Patouceul for the clarifications.
I used the below command to capture the raw12 and it is succesfull:
v4l2-ctl -d /dev/video0 --set-fmt-video=width=2048,height=2048,pixelformat=BG12 --stream-count=1 --stream-mmap --stream-to=csi.raw --verbose
Now, we would like to capture raw16, so I have added the below macro in sources/nvidia/drivers/media/platform/tegra/camera/vi/vi5_formats.h
TEGRA_VIDEO_FORMAT(RAW16, 16, SBGGR16_1X16, 2, 1, T_R16,
RAW16, SBGGR16, “BGBG… GRGR…”)
If you see the 1st argument in the macro will expand to TEGRA_VF_RAW16, but if we see the file nvidia/include/media/tegra_camera_core.h, the below enums do not have support for RAW16.
enum tegra_image_dt {
TEGRA_IMAGE_DT_YUV420_8 = 24,
TEGRA_IMAGE_DT_YUV420_10,
TEGRA_IMAGE_DT_YUV420CSPS_8 = 28,
TEGRA_IMAGE_DT_YUV420CSPS_10,
TEGRA_IMAGE_DT_YUV422_8,
TEGRA_IMAGE_DT_YUV422_10,
TEGRA_IMAGE_DT_RGB444,
TEGRA_IMAGE_DT_RGB555,
TEGRA_IMAGE_DT_RGB565,
TEGRA_IMAGE_DT_RGB666,
TEGRA_IMAGE_DT_RGB888,
TEGRA_IMAGE_DT_RAW6 = 40,
TEGRA_IMAGE_DT_RAW7,
TEGRA_IMAGE_DT_RAW8,
TEGRA_IMAGE_DT_RAW10,
TEGRA_IMAGE_DT_RAW12,
TEGRA_IMAGE_DT_RAW14,
};
/* Supported CSI to VI Data Formats */
enum tegra_vf_code {
TEGRA_VF_RAW6 = 0,
TEGRA_VF_RAW7,
TEGRA_VF_RAW8,
TEGRA_VF_RAW10,
TEGRA_VF_RAW12,
TEGRA_VF_RAW14,
TEGRA_VF_EMBEDDED8,
TEGRA_VF_RGB565,
TEGRA_VF_RGB555,
TEGRA_VF_RGB888,
TEGRA_VF_RGB444,
TEGRA_VF_RGB666,
TEGRA_VF_YUV422,
TEGRA_VF_YUV420,
TEGRA_VF_YUV420_CSPS,
};
Although, we can use TEGRA_IMAGE_DT_RAW16 = 45 from the reserved [ referred CSI2 MIPI specification document] but I have no idea about the value to be added in the enum tegra_vf_code for RAW16. Kindly let me know your suggestion.
Regards,
Shiva Shankar K.
You can add TEGRA_IMAGE_DT_RAW16 map to 46 like below.
DTYPE:
NvCsiDataType_Unspecified = 0,
NvCsiDataType_YUV420_10 = 25,
NvCsiDataType_LEG_YUV420_8 = 26,
NvCsiDataType_YUV420_8 = 24,
NvCsiDataType_YUV420CSPS_8 = 28,
NvCsiDataType_YUV420CSPS_10 = 29,
NvCsiDataType_YUV422_8 = 30,
NvCsiDataType_YUV422_10 = 31,
NvCsiDataType_RGB444 = 32,
NvCsiDataType_RGB555 = 33,
NvCsiDataType_RGB565 = 34,
NvCsiDataType_RGB666 = 35,
NvCsiDataType_RGB888 = 36,
NvCsiDataType_RAW6 = 40,
NvCsiDataType_RAW7 = 41,
NvCsiDataType_RAW8 = 42,
NvCsiDataType_RAW10 = 43,
NvCsiDataType_RAW12 = 44,
NvCsiDataType_RAW14 = 45,
NvCsiDataType_RAW16 = 46,
NvCsiDataType_RAW20 = 47,
NvCsiDataType_User_1 = 48,
sorry, I misspelled this, yes it is TEGRA_IMAGE_DT_RAW16 =46; Importantly, I want to know the TEGRA_VF_RAW16 value, i.e., should I append the enum tegra_vf_code or insert right after TEGRA_VF_RAW14???
Regards,
Shiva Shankar K.
Any code need those TEGRA_VF_RAW* define?
I just saw the header files have it.
I dont think any part of the code uses that macro, as you said, it is seen only in the header file. So, do you mean I can simply assign any value for the TEGRA_VF_RAW16??
Regards,
Shiva Shankar K.
Any compiler error if don’t define TEGRA_VF_RAW16?
Here is the compilation error :
In file included from /home/shivashankar.k/projects/r12/sources/kernel/nvidia/include/media/mc_common.h:32:0,
from /home/shivashankar.k/projects/r12/sources/kernel/nvidia/drivers/media/platform/tegra/camera/vi/vi5_fops.c:19:*
/home/shivashankar.k/projects/r12/sources/kernel/nvidia/include/media/tegra_camera_core.h:115:2: error: ‘TEGRA_VF_RAW16’ undeclared here (not in a function); did you mean ‘TEGRA_VF_RAW14’?
Regards,
Shiva Shankar K.
Looks like need add TEGRA_VF_RAW16 for it, I am not sure if it’s working if just add right after raw14.
Could you help to try it.
yeah… I did the same, its working.
Thanks for your inputs.
Regards,
Shiva Shankar K.
After successful capture of raw16 using v4l2-ctl the below command (1) , I tried to use the below gst command (2), but it gives this error:
“ERROR: from element /GstPipeline: pipeline0/GstV4l2Src:v4l2src0: Internal data stream error”
Regards,
Shiva Shankar K.
It could be v4l2src didn’t support the format.
If you want to capture in 16 bits, this looks suspect !
Appologies, but looking at gstreamer bayer2rgb caps, now I only see 8 bits bayer formats supported. Not sure why I was thinking that 16 bits were supported… sorry for that.
For raw16 debayering, you may try this instead, it would be faster than CPU debayering anyway.