Hello all,
I have an FPGA, that outputs 1920x1080@30 2-lane CSI stream in RGB888 format, I am able to recieve this stream, however, the system misinterprets the settings I gave and treats this signal as BGRA (32-bit). Moreover, the output has only 15 fps instead of 30 and has 200ms delay (this is the shortest delay time I was able to achieve by tuning the cil_settletime). My case is similar to this one: https://devtalk.nvidia.com/default/topic/1051967/jetson-tx1/v4l-reports-ar24-instead-of-rgb888/
I browsed and read all relevant forum threads I could find with no success. More details below.
The FPGA device functionality was verified on raspberry Pi, (with slightly modified raspiraw program - i2c operations were bypassed , that’s all AFAIK) since pinout and lane configuration matches raspicam 2.1. Notably, on the RPI the latency was measured at ~130ms, the same monitor was used.
This is the tracing I get on my TX2, note the LOAD_FRAMED it isnt right, is it?:
# tracer: nop
#
# entries-in-buffer/entries-written: 253/253 #P:6
#
# _-----=> irqs-off
# / _----=> need-resched
# | / _---=> hardirq/softirq
# || / _--=> preempt-depth
# ||| / delay
# TASK-PID CPU# |||| TIMESTAMP FUNCTION
# | | | |||| | |
kworker/0:1-58 [000] ...1 60.156235: rtos_queue_peek_from_isr_failed: tstamp:2232766306 queue:0x0b4a3c58
kworker/0:1-58 [000] ...1 60.156242: rtcpu_start: tstamp:2232803798
kworker/0:1-58 [000] ...1 60.156246: rtcpu_vinotify_handle_msg: tstamp:2232911295 tag:CSIMUX_STREAM channel:0xff frame:0 vi_tstamp:2232910875 data:0x00000001
kworker/0:1-58 [000] ...1 60.205837: rtcpu_vinotify_handle_msg: tstamp:2234243891 tag:CHANSEL_PXL_SOF channel:0x00 frame:0 vi_tstamp:2234243290 data:0x00000001
kworker/0:1-58 [000] ...1 60.205841: rtcpu_vinotify_handle_msg: tstamp:2234244111 tag:ATOMP_FS channel:0x00 frame:0 vi_tstamp:2234243297 data:0x00000000
kworker/0:1-58 [000] ...1 60.205842: rtcpu_vinotify_handle_msg: tstamp:2234246393 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:0 vi_tstamp:2234245968 data:0x08000000
kworker/0:1-58 [000] ...1 60.257827: rtcpu_vinotify_handle_msg: tstamp:2235243471 tag:CHANSEL_PXL_EOF channel:0x00 frame:0 vi_tstamp:2235243061 data:0x04370002
kworker/0:1-58 [000] ...1 60.257831: rtcpu_vinotify_handle_msg: tstamp:2235247237 tag:ATOMP_FE channel:0x00 frame:0 vi_tstamp:2235246793 data:0x00000000
kworker/0:1-58 [000] ...1 60.257832: rtcpu_vinotify_handle_msg: tstamp:2236327172 tag:CHANSEL_PXL_SOF channel:0x00 frame:0 vi_tstamp:2236326636 data:0x00000001
kworker/0:1-58 [000] ...1 60.257833: rtcpu_vinotify_handle_msg: tstamp:2236327321 tag:ATOMP_FS channel:0x00 frame:0 vi_tstamp:2236326643 data:0x00000000
kworker/0:1-58 [000] ...1 60.257833: rtcpu_vinotify_handle_msg: tstamp:2236329384 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:0 vi_tstamp:2236328972 data:0x08000000
kworker/0:1-58 [000] ...1 60.310345: rtcpu_vinotify_handle_msg: tstamp:2237326876 tag:CHANSEL_PXL_EOF channel:0x00 frame:0 vi_tstamp:2237326407 data:0x04370002
kworker/0:1-58 [000] ...1 60.310348: rtcpu_vinotify_handle_msg: tstamp:2237330611 tag:ATOMP_FE channel:0x00 frame:0 vi_tstamp:2237330139 data:0x00000000
kworker/0:1-58 [000] ...1 60.310350: rtos_queue_peek_from_isr_failed: tstamp:2237767178 queue:0x0b4a3c58
This is my relevant device tree setting (you can disregard the comments):
mode0 {
mclk_khz = "24000";
mclk_multiplier = "25";//JG 10 was ok
//pix_clk_hz = "182400000";//
pix_clk_hz = "74250000"; //ok
//pix_clk_hz = "37125000";
//pix_clk_hz = "148500000";
num_lanes = "2"; //ok
tegra_sinterface = "serial_a";
discontinuous_clk = "no"; //ok, clk and lanes run at all times
cil_settletime = "150"; //not sure, "1" starts @ 60ns and ends @ ~300ns
//according to my calculations must be:
// 127ns < cil_settletime + 5 < 215ns
pixel_t = "rgb3"; //ok
//pixel_t = "bayer_rggb";
//------------------------------
//mode_type = "Rgb";//jg
//csi_pixel_bit_depth = "24";//jg
//pixel_phase = "rgb888";//jg rggb
//------------------------------
readout_orientation = "90"; //not important
inherent_gain = "1"; //not important
active_w = "1920"; //ok
active_h = "1080"; //ok
line_length = "2200"; //teoretycznie ok =2200
dpcm_enable = "false";
min_gain_val = "1.0";
max_gain_val = "16";
min_hdr_ratio = "1";
max_hdr_ratio = "64";
min_framerate = "30";//1
max_framerate = "30";
//min_exp_time = "33";
min_exp_time = "11";//11
max_exp_time = "683709";
embedded_metadata_height = "0";
};
The drivers, as well as the device tree settings are based on notes made by Cospandesign (Raspberry Pi Camera on TX2) modified in order to match the R28.2.1 version (with driver update guide) and to match the output color format.
My sensor_common.c modifications (based on what I found on the NVIDIA forums):
static int extract_pixel_format(
const char *pixel_t, u32 *format)
{
size_t size = strnlen(pixel_t, OF_MAX_STR_LEN);
if (strncmp(pixel_t, "bayer_bggr10", size) == 0)
*format = V4L2_PIX_FMT_SBGGR10;
else if (strncmp(pixel_t, "bayer_rggb10", size) == 0)
*format = V4L2_PIX_FMT_SRGGB10;
else if (strncmp(pixel_t, "bayer_bggr12", size) == 0)
*format = V4L2_PIX_FMT_SBGGR12;
else if (strncmp(pixel_t, "bayer_rggb12", size) == 0)
*format = V4L2_PIX_FMT_SRGGB12;
else if (strncmp(pixel_t, "bayer_wdr_pwl_rggb12", size) == 0)
*format = V4L2_PIX_FMT_SRGGB12;
else if (strncmp(pixel_t, "bayer_wdr_dol_rggb10", size) == 0)
*format = V4L2_PIX_FMT_SRGGB10;
else if (strncmp(pixel_t, "bayer_xbggr10p", size) == 0)
*format = V4L2_PIX_FMT_XBGGR10P;
else if (strncmp(pixel_t, "bayer_xrggb10p", size) == 0)
*format = V4L2_PIX_FMT_XRGGB10P;
else if (strncmp(pixel_t, "rgb3", size) == 0)//jg dodano
*format = V4L2_PIX_FMT_RGB24;
else {
pr_err("%s: Need to extend format%s\n", __func__, pixel_t);
return -EINVAL;
}
return 0;
}
My camera_common.c (based on what I found on the NVIDIA forums):
static const struct camera_common_colorfmt camera_common_color_fmts[] = {
{
MEDIA_BUS_FMT_SRGGB12_1X12,
V4L2_COLORSPACE_SRGB,
V4L2_PIX_FMT_SRGGB12,
},
{
MEDIA_BUS_FMT_SRGGB10_1X10,
V4L2_COLORSPACE_SRGB,
V4L2_PIX_FMT_SRGGB10,
},
{
MEDIA_BUS_FMT_SBGGR10_1X10,
V4L2_COLORSPACE_SRGB,
V4L2_PIX_FMT_SBGGR10,
},
{
MEDIA_BUS_FMT_SRGGB8_1X8,
V4L2_COLORSPACE_SRGB,
V4L2_PIX_FMT_SRGGB8,
},
{//JG, DODANO obsluge rgb24
MEDIA_BUS_FMT_RGB888_1X24,
V4L2_COLORSPACE_SRGB,
V4L2_PIX_FMT_RGB24,
},
/*
* The below two formats are not supported by VI4,
* keep them at the last to ensure they get discarded
*/
{
MEDIA_BUS_FMT_XRGGB10P_3X10,
V4L2_COLORSPACE_SRGB,
V4L2_PIX_FMT_XRGGB10P,
},
{
MEDIA_BUS_FMT_XBGGR10P_3X10,
V4L2_COLORSPACE_SRGB,
V4L2_PIX_FMT_XRGGB10P,
},
};
Sample hexdump of a frame captured with v4l2-ctl (note all the the 0xFF as in abovementioned forum thread):
07e8f60 6955 ff92 6854 ff91 6a51 ff92 684f ff90
07e8f70 684f ff92 674e ff91 684f ff90 674e ff8f
07e8f80 674e ff8f 684f ff90 684e ff90 674d ff8f
07e8f90 684b ff8f 684b ff8f 664c ff8e 664c ff8e
07e8fa0 664d ff90 654c ff8f 664c ff8e 664c ff8e
07e8fb0 664d ff8e 654c ff8d 634d ff8c 644e ff8d
07e8fc0 634d ff8c 644e ff8d 634d ff8c 634d ff8c
07e8fd0 644e ff8d 634d ff8c 634d ff8c 624c ff8b
07e8fe0 634a ff8b 634a ff8b 6349 ff8b 6349 ff8b
07e8ff0 634a ff8b 634a ff8b 6249 ff8a 6047 ff88
Here are my questions:
- Did anyone manage to get rid of this problem or at least pinpoint the cause of it?
- Will this problem disappear if I transfer to newest Jetpack/L4T?
Best regards,
JG