Kernel can not boot after enable camera

JerryChang:
After some changes, VIDIOC_STREAMON do not appear.
When the command is executed:
v4l2-ctl --set-fmt-video=width=1920,height=1080,pixelformat=RG10 --stream-mmap --stream-count=1 -d /dev/video0 ,
there is still no camera image that can be previewed on the screen.

The 1.txt is the log.The last a few lines is related to the v4l2-ctl.1.txt (79.8 KB)
Please help.

hello ycl,

you may enable v4l2-ctl to parse the sensor mode properties for checking its sensor pixel formats and also mode resolutions.
for example,

nvidia@tegra-ubuntu:~$ v4l2-ctl -d /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
        Index       : 0
        Type        : Video Capture
        Pixel Format: 'BG10'
        Name        : 10-bit Bayer BGBG/GRGR
                Size: Discrete 2592x1944
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 2592x1458
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1280x720
                        Interval: Discrete 0.008s (120.000 fps)

please do specify the exactly supported sensor formats and width/height settings to the v4l2-ctl pipeline.
it’ll show > for each success capture frame instead of camera preview frames with v4l2 standard controls. it’ll also report frames-per-second as below if you request for more frame captures.
for example,

$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=2592,height=1944,pixelformat=BG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=100
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.09 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.03 fps

according to your kernel messages, there’re errors to report timeout failures for waiting sensor frames.

[  174.423856] tegra194-vi5 15c10000.vi: no reply from camera processor
[  174.424053] tegra194-vi5 15c10000.vi: uncorr_err: request timed out after 2500 ms
[  174.424209] tegra194-vi5 15c10000.vi: err_rec: attempting to reset the capture channel

is it possible to arrange hardware resources to probe the MIPI signaling? you may check from the CSI bridge to ensure you had correct regulator settings.
there’re other settings from DT for VI engine, please adding set_mode_delay_ms into your device tree for maximum the wait time for the first frame after capture starts.
thanks

hello ycl,

you may also refer to https://elinux.org/Jetson_TX2_Camera_BringUp#Steps_to_enable_more_debug_messages,
please enable VI tracing log to gather more details for reference,
thanks

JerryChang:
Yestoday, I measured mipi output from MAX96712 des sensor, and could not get any signal. After debugging, mipi signal could be measured on oscilloscope . And “request timed out after 2500 ms” log disappeared.
Then I got the following loop log:

[ 93.458943] tegra194-vi5 15c10000.vi: corr_err: discarding frame 0, flags: 224, err_data 4194400
[ 93.460281] tegra194-vi5 15c10000.vi: corr_err: discarding frame 0, flags: 0, err_data 16384
[ 93.492274] tegra194-vi5 15c10000.vi: corr_err: discarding frame 0, flags: 224, err_data 4194400
[ 93.525667] tegra194-vi5 15c10000.vi: corr_err: discarding frame 0, flags: 224, err_data 4194400
[ 93.526929] tegra194-vi5 15c10000.vi: corr_err: discarding frame 0, flags: 0, err_data 16384
[ 93.558944] tegra194-vi5 15c10000.vi: corr_err: discarding frame 0, flags: 224, err_data 4194400
[ 93.560258] tegra194-vi5 15c10000.vi: corr_err: discarding frame 0, flags: 0, err_data 16384
[ 93.592250] tegra194-vi5 15c10000.vi: corr_err: discarding frame 0, flags: 224, err_data 4194400
[ 93.625585] tegra194-vi5 15c10000.vi: corr_err: discarding frame 0, flags: 224, err_data 4194400
[ 93.626917] tegra194-vi5 15c10000.vi: corr_err: discarding frame 0, flags: 0, err_data 16384
[ 93.658927] tegra194-vi5 15c10000.vi: corr_err: discarding frame 0, flags: 224, err_data 4194400

The logs come from the following code:
static void vi5_capture_dequeue(struct tegra_channel *chan,
struct tegra_channel_buffer *buf)
{
int err = 0;
unsigned long flags;
struct tegra_mc_vi *vi = chan->vi;
struct vb2_v4l2_buffer *vb = &buf->buf;
struct timespec ts;
struct capture_descriptor *descr =
&chan->request[buf->capture_descr_index];

if (buf->vb2_state != VB2_BUF_STATE_ACTIVE)
	goto rel_buf;

/* Dequeue a frame and check its capture status */
err = vi_capture_status(chan->tegra_vi_channel, CAPTURE_TIMEOUT_MS);
if (err) {
	if (err == -ETIMEDOUT) {
		dev_err(vi->dev,
			"uncorr_err: request timed out after %d ms  ***byd ***\n",
			CAPTURE_TIMEOUT_MS);
	} else {
		dev_err(vi->dev, "uncorr_err: request err %d\n", err);
	}
	goto uncorr_err;
} else if (descr->status.status != CAPTURE_STATUS_SUCCESS) {
	if ((descr->status.flags
			& CAPTURE_STATUS_FLAG_CHANNEL_IN_ERROR) != 0) {
		chan->queue_error = true;
		dev_err(vi->dev, "uncorr_err: flags %d, err_data %d\n",
			descr->status.flags, descr->status.err_data);
	} else {
		dev_warn(vi->dev,
			"corr_err: discarding frame %d, flags: %d, "
			"err_data %d\n",
			descr->status.frame_id, descr->status.flags,
			descr->status.err_data);
		buf->vb2_state = VB2_BUF_STATE_REQUEUEING;
		goto done;
	}
}

… … … …

Why did these loop logs appear?

hello ycl,

please enable VI tracing log to gather more details for reference, thanks

How to enable VI tracing log?
Are these commands?
echo 1 > /sys/kernel/debug/tracing/tracing_on
echo 30720 > /sys/kernel/debug/tracing/buffer_size_kb
echo 1 > /sys/kernel/debug/tracing/events/tegra_rtcpu/enable
echo 1 > /sys/kernel/debug/tracing/events/freertos/enable
echo 2 > /sys/kernel/debug/camrtc/log-level
echo 1 > /sys/kernel/debug/tracing/events/camera_common/enable
echo > /sys/kernel/debug/tracing/trace
cat /sys/kernel/debug/tracing/trace

hello ycl,

yes, that’s commands to enable VI tracing log.
besides using cat to dump all messages, you may use this commands to check tracing logs in real-time,
$ cat /sys/kernel/debug/tracing/trace_pipe

you may gather logs as below while issue reproduced for better tracking.
thanks

After enable VI tracing log, I got the following log:log.zip (640.0 KB)

hello ycl,

it seems there’re lots of CORRECTABLE_ERR, could you please review Port Binding for correct definition.
also, there’s no start-of-frame and end-of-frame signaling from the messages, VI is using that to determine one complete frame.
the frame-number did not increasing. this also looks not correct.

     kworker/0:1-8679  [000] ....  4854.865244: rtcpu_vinotify_error: tstamp:152059686763 tag:CSIMUX_STREAM channel:0x00 frame:0 vi_tstamp:152059684213 data:0x00000001
     kworker/0:1-8679  [000] ....  4854.865246: rtcpu_nvcsi_intr: tstamp:152059817861 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
     kworker/0:1-8679  [000] ....  4854.865247: rtcpu_nvcsi_intr: tstamp:152059817861 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
...
     kworker/0:1-8679  [000] ....  4854.977158: rtcpu_vinotify_error: tstamp:152064473761 tag:CSIMUX_FRAME channel:0x00 frame:0 vi_tstamp:152064469360 data:0x00400060
     kworker/0:1-8679  [000] ....  4854.977159: rtcpu_vinotify_event: tstamp:152064478622 tag:CSIMUX_FRAME channel:0x00 frame:0 vi_tstamp:152064469360 data:0x00400060
     kworker/0:1-8679  [000] ....  4854.977159: rtcpu_vinotify_event: tstamp:152064478800 tag:CHANSEL_SHORT_FRAME channel:0x01 frame:0 vi_tstamp:152064469360 data:0x00000000
     kworker/0:1-8679  [000] ....  4854.977160: rtcpu_vinotify_event: tstamp:152064478949 tag:RESERVED_19 channel:0x23 frame:0 vi_tstamp:150188951232 data:0x01020004
     kworker/0:1-8679  [000] ....  4854.977160: rtcpu_vinotify_event: tstamp:152064479121 tag:CHANSEL_SHORT_FRAME channel:0x01 frame:0 vi_tstamp:152064469360 data:0x01000000
     kworker/0:1-8679  [000] ....  4854.977160: rtcpu_vinotify_event: tstamp:152064479270 tag:RESERVED_19 channel:0x23 frame:0 vi_tstamp:150188966272 data:0x01020004
     kworker/0:1-8679  [000] ....  4854.977160: rtcpu_vinotify_event: tstamp:152064479443 tag:ATOMP_FE channel:0x00 frame:0 vi_tstamp:152064469363 data:0x00000000
     kworker/0:1-8679  [000] ....  4854.977160: rtcpu_vinotify_event: tstamp:152064479588 tag:RESERVED_19 channel:0x23 frame:0 vi_tstamp:150188980032 data:0x07020005
     kworker/0:1-8679  [000] ....  4854.977161: rtcpu_vinotify_event: tstamp:152064479758 tag:FS channel:0x00 frame:0 vi_tstamp:152064472435 data:0x00000010
     kworker/0:1-8679  [000] ....  4854.977161: rtcpu_vinotify_event: tstamp:152064479911 tag:ATOMP_FS channel:0x00 frame:0 vi_tstamp:152064472438 data:0x00000000
     kworker/0:1-8679  [000] ....  4854.977161: rtcpu_vinotify_event: tstamp:152064480083 tag:RESERVED_18 channel:0x23 frame:0 vi_tstamp:150189122048 data:0x10000000
     kworker/0:1-8679  [000] ....  4854.977161: rtcpu_vinotify_event: tstamp:152064480229 tag:RESERVED_18 channel:0x23 frame:0 vi_tstamp:150189155872 data:0x31000009
     kworker/0:1-8679  [000] ....  4854.977162: rtos_queue_send_from_isr_failed: tstamp:152064482917 queue:0x0bcb41f8
     kworker/0:1-8679  [000] ....  4854.977162: rtos_queue_send_from_isr_failed: tstamp:152064483080 queue:0x0bcb8a60
     kworker/0:1-8679  [000] ....  4854.977162: rtos_queue_send_from_isr_failed: tstamp:152064483245 queue:0x0bcba5e0
     kworker/0:1-8679  [000] ....  4854.977162: rtos_queue_send_from_isr_failed: tstamp:152064483403 queue:0x0bcbb3a0
     kworker/0:1-8679  [000] ....  4854.977162: rtos_queue_send_from_isr_failed: tstamp:152064483560 queue:0x0bcbc160
     kworker/0:1-8679  [000] ....  4855.033032: rtcpu_vinotify_error: tstamp:152064515658 tag:CHANSEL_NOMATCH channel:0x01 frame:0 vi_tstamp:152064511695 data:0x000003c9

BTW,
are you able to enable test-pattern-generator on your Ser/Des chip?
thanks

After the command “sudo media-ctl -p -d /dev/media0”, port binding result is the belowing. Where is the problem?

Media controller API version 0.1.0

Media device information

driver tegra194-vi5
model NVIDIA Tegra Video Input Device
serial
bus info
hw revision 0x3
driver version 0.0.0

Device topology

  • entity 1: imx390 2-001b (1 pad, 1 link)
    type V4L2 subdev subtype Sensor flags 0
    device node name /dev/v4l-subdev0
    pad0: Source
    [fmt:SRGGB12_1X12/1920x1080 field:none colorspace:srgb]
    -> “15a00000.nvcsi–2”:0 [ENABLED]

  • entity 3: 15a00000.nvcsi–2 (2 pads, 2 links)
    type V4L2 subdev subtype Unknown flags 0
    device node name /dev/v4l-subdev1
    pad0: Sink
    <- “imx390 2-001b”:0 [ENABLED]
    pad1: Source
    -> “vi-output, imx390 2-001b”:0 [ENABLED]

  • entity 6: vi-output, imx390 2-001b (1 pad, 1 link)
    type Node subtype V4L flags 0
    device node name /dev/video0
    pad0: Sink
    <- “15a00000.nvcsi–2”:1 [ENABLED]

  • entity 18: imx390 2-001c (1 pad, 1 link)
    type V4L2 subdev subtype Sensor flags 0
    device node name /dev/v4l-subdev2
    pad0: Source
    [fmt:SRGGB12_1X12/1920x1080 field:none colorspace:srgb]
    -> “15a00000.nvcsi–1”:0 [ENABLED]

  • entity 20: 15a00000.nvcsi–1 (2 pads, 2 links)
    type V4L2 subdev subtype Unknown flags 0
    device node name /dev/v4l-subdev3
    pad0: Sink
    <- “imx390 2-001c”:0 [ENABLED]
    pad1: Source
    -> “vi-output, imx390 2-001c”:0 [ENABLED]

  • entity 23: vi-output, imx390 2-001c (1 pad, 1 link)
    type Node subtype V4L flags 0
    device node name /dev/video1
    pad0: Sink
    <- “15a00000.nvcsi–1”:1 [ENABLED]

hello ycl,

the port binding results looks correct,
so, you’re having two video nodes, i.e. /dev/video0 and /dev/video1, right?

could you please also check why there without no start-of-frame and end-of-frame signaling, VI is using that to determine one complete frame.
thanks

Jerry,
How to check start-of-frame and end-of-frame signaling? Is that hardware pin connected to external sensor? Can it be measured by oscilloscope?

hello ycl,

you’ll need a high-speed oscilloscope, there’ll be LP-11,LP-01, LP-00 before the high-speed signaling, then following another LP-11,LP-01, LP-00. these are start-of-frame and end-of-frame signaling.
if you google it around, you’ll see the reference diagrams of MIPI DPHY signaling.
for example, https://www.circuitvalley.com/2020/02/imx219-camera-mipi-csi-receiver-fpga-lattice-raspberry-pi-camera.html

JerryChang:
The attached pictures are start-of-frame and end-of-frame signalings.
Is there any problems?

hello ycl,

these MIPI signaling results looks correct as normal workable sensors.
since you’re having the signaling; please check cil_settletime, by default it’s setting to 0 for the driver attempts to auto-calibrate. you may assign THS settle time of the MIPI lane, unit is in nanoseconds.

BTW,
are you able to enable test-pattern-generator on your Ser/Des chip?
thanks

According to the picture uploaded, can I set THS to 400ns?
Why can not use cil_settletime auto-calibrate?

Jerry:
Our sensor output YUV format data.
Can XAVIER receive YUV image?

hello ycl,

you may refer to TRM for the formula of THS settle time,
for example,

85ns + 6 * UI < (Ths-settle-programmed + 6) * lp clock periods < 145ns + 10 * UI

CSI brick is able to support YUV formats. please enable v4l2 standard control or v4l2src plugin to access that.
please refer to Camera Architecture Stack, libargus works with Jetson ISP so it only support with bayer raw formats.
thanks

There are no start-of-frame and end-of-frame signal in our logs. Is it because our config is rawdata?

How to change rgb to yuv?
In device tree,
mode0 {/mode IMX390_MODE_1920X1080_CROP_30FPS/
mclk_khz = “24000”;
num_lanes = “2”;
tegra_sinterface = “serial_a”;
vc_id = “0”;
discontinuous_clk = “no”;
dpcm_enable = “false”;
cil_settletime = “0”;
dynamic_pixel_bit_depth = “12”;
csi_pixel_bit_depth = “12”;
mode_type = “bayer”;
pixel_phase = “rggb”;
… …

we change

csi_pixel_bit_depth = “12”;
mode_type = “bayer”;
pixel_phase = “rggb”;
to
csi_pixel_bit_depth = “16”;
mode_type = “yuv”;
pixel_phase = “uyuv”;
Is the changes above OK?

But in code

static int extract_pixel_format(
const char *pixel_t, u32 *format)
{
size_t size = strnlen(pixel_t, OF_MAX_STR_LEN);

if (strncmp(pixel_t, "bayer_bggr10", size) == 0)
	*format = V4L2_PIX_FMT_SBGGR10;
else if (strncmp(pixel_t, "bayer_rggb10", size) == 0)
	*format = V4L2_PIX_FMT_SRGGB10;

… …
else if (strncmp(pixel_t, “yuv_yuyv16”, size) == 0)
*format = V4L2_PIX_FMT_YUYV;
else if (strncmp(pixel_t, “yuv_yvyu16”, size) == 0)
*format = V4L2_PIX_FMT_YVYU;
else if (strncmp(pixel_t, “yuv_uyvy16”, size) == 0)
*format = V4L2_PIX_FMT_UYVY;
else if (strncmp(pixel_t, “yuv_vyuy16”, size) == 0)
*format = V4L2_PIX_FMT_VYUY;

There are only yuv 16bit config. OUr camera output yuv 8bit. Could I select yuv_uyvy16?