Recovery after "General error queue is out of sync with frame queue" does not work

[daniel.kammerer]

Hello,

it seems that I have some EMC problems with my setup, sometimes it happens during startup, sometimes during runtime and the gstreamer stops working and I have to close it and restart.
Is there any way to automatically resume the video stream (to recover it)?

clocks are boosted

my command to start GStreamer: gst-launch-1.0 v4l2src device=/dev/video0 ! “video/x-raw,format=(string)UYVY,width=1100,height=884,framerate=30/1,interlace-mode=progressive” ! xvimagesink

attached the trace and dmesg log and some pictures how it looks like after the crash

Dmesg.txt (1.7 KB)
Trace.txt (1.3 MB)

[DaneLLL]

Hi,
Please run v4l2-ctl command to capture frame data and check if the saved frame data is good. Generally if the sensor driver is good, it should be no issue in v4l2-ctl and gstreamer commands. It is more like an issue in sensor driver or sensor does not steadily generate frame data.

[daniel.kammerer]

Hi,
I assume you meant this command.
What I don’t understand is why the programme only restores automatically the fourth time, what is different in the top 3 cases (After it hung I cancelled the command and ran it again, I didn’t restart the sensor)?

[DaneLLL]

Hi,
Please run

$ v4l2-ctl -d /dev/video0 --list-formats-ext

to get supported sensor modes. And apply exact width, height, pixelformat to the command:

$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=uyvy --stream-mmap --stream-count=1 --set-ctrl bypass_mode=0 --stream-to=/tmp/stream.raw

And check if saved stream.raw is valid.

[daniel.kammerer]

Hi,

contiorin@contiorin-desktop:~$ v4l2-ctl -d /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture

[0]: 'UYVY' (UYVY 4:2:2)
	Size: Discrete 1100x884
		Interval: Discrete 0.033s (30.000 fps)
[1]: 'NV16' (Y/CbCr 4:2:2)
	Size: Discrete 1100x884
		Interval: Discrete 0.033s (30.000 fps)
[2]: 'UYVY' (UYVY 4:2:2)
	Size: Discrete 1100x884
		Interval: Discrete 0.033s (30.000 fps)

contiorin@contiorin-desktop:~$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=1100,height=884,pixelformat=UYVY --stream-mmap --stream-count=1 --set-ctrl bypass_mode=0 --stream-to=/tmp/stream.raw
<

first image is with a preferred stride of 2200 (default)
second image is with a preferred stride of 2240

[DaneLLL]

Hi,
Somehow the raw data becomes gray. Seems not expected. You may capture 15 frames and check if the color is correct:

$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=1100,height=884,pixelformat=UYVY --stream-mmap --stream-count=15 --set-ctrl bypass_mode=0 --stream-to=/tmp/stream.raw

[daniel.kammerer]

Hi,
sorry, I’m not that familiar with the raw file format.
I opened the image with ImageJ and I think it doesn’t add colour information by default.
For the coloured image I have now used https://rawpixels.net/.

[DaneLLL]

Hi,
The raw data seems not correct since there is green bar at right side. There might be something wrong in sensor driver. We would suggest check this first.

[daniel.kammerer]

doesn’t the green block come from this preferred stride adjustment?
the camera has a line length of 1100, multiplied by 2 bytes I get 2200, but 2200 is not a multiple of 64, so I increased it to 2240 (and this green bar is exactly these 40 bytes → 20 pixels)
Without changing the preferred stride it looks like the first picture in my third post.

Could you please tell me which parameters I should change?

[DaneLLL]

Hi,
Please check if the camera sensor can generate width=1120 to fit data alignment.

[daniel.kammerer]

Unfortunately I cannot change the resolution of the camera and only changing the values in the device tree and/or in the tbls.h file does not work.

[DaneLLL]

Hi,
Please change resolution to 1120x884 in device tree. So that you see 1120x884 is shown in $ v4l2-ctl -d /dev/video0 --list-formats-ext

And can capture good frame date by running:

$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=1120,height=884,pixelformat=UYVY --stream-mmap --stream-count=1 --set-ctrl bypass_mode=0 --stream-to=/tmp/stream.raw

If above command works, please run the gstreamer command and check if camera preview is good:

gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY,width=1120,height=884,framerate=30/1" ! videoconvert ! xvimagesink

[daniel.kammerer]

If I only change the resolution in the device tree (.dtsi)

active_w = "1120";
active_h = "884";
readout_orientation = "0";
line_length = "1120";
inherent_gain = "1";
/*mclk_multiplier = "2";*/
pix_clk_hz = "29702400";
serdes_pix_clk_hz = "100000000";

contiorin@contiorin-desktop:~$ v4l2-ctl -d /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture

  Size: Discrete 1100x884
  	Interval: Discrete 0.033s (30.000 fps)

  Size: Discrete 1100x884
  	Interval: Discrete 0.033s (30.000 fps)

  Size: Discrete 1100x884
  	Interval: Discrete 0.033s (30.000 fps)

contiorin@contiorin-desktop:~$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=1120,height=884,pixelformat=UYVY --stream-mmap --stream-count=1 --set-ctrl bypass_mode=0 --stream-to=/tmp/stream.raw

then it depends on how I “convert” the raw image.
with 1100

with 1120

contiorin@contiorin-desktop:~$ gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY,width=1120,height=884,framerate=30/1" ! videoconvert ! xvimagesink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3072): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.000088928
Setting pipeline to NULL ...
Freeing pipeline ...

if I also change the resolution in …mode_tbls.h

static const struct camera_common_frmfmt ovx1f_frmfmt[] = {
	{{1120, 884}, ovx1f_30fps, 1, 0, OVX1F_RVC_MODE_1120X884},
	/* Add modes with no device tree support after below */
};

I can’t save a raw image, after I send the command, nothing happens.
Gstreamer shows only one image at start-up and then hangs

[  231.477719] bwmgr API not supported
[  233.937950] bwmgr API not supported
[  233.988736] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 2, flags: 0, err_data 512
[  234.088582] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 1, flags: 0, err_data 512
[  234.121902] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 2, flags: 0, err_data 512
[  234.155164] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 3, flags: 0, err_data 512

[DaneLLL]

Hi,
Would like to confirm the current status. So yo run this command:

@DaneLLL

$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=1100,height=884,pixelformat=UYVY --stream-mmap --stream-count=5 --set-ctrl bypass_mode=0 --stream-to=/tmp/stream.raw

You can see valid RAW frame data in 1120x884. Is this correct?
Please also attach stream.raw for reference.

[daniel.kammerer]

Hi,

yes, I have run this command and I can see valid frame data (but as I mentioned at the beginning, sometimes it does not start properly and I have to send the command several times until it works and I get valid data)

$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=1100,height=884,pixelformat=UYVY --stream-mmap --stream-count=5 --set-ctrl bypass_mode=0 --stream-to=/tmp/stream.raw

but it does not matter whether I change the resolution in the command from 1100 to 1120 or whether I do this in the device tree
active_w = “1120”;
line_length = “1120”;

active_w_1100_width_1100_preferredStride_2200.raw (9.3 MB)
active_w_1100_width_1100_preferredStride_2240.raw (9.4 MB)

[JerryChang]

hello daniel.kammerer,

it’s due to VI’s alignment, you should set the width alignment to 64.
there’s v4l2-ctl command to adjust the stride by --preferred_stride=<>.

furthermore,
if you look into the kernel sources,
it’s low-level driver trying to alignment the buffer with sources image. sometimes, it’s corner case cannot obtain correct stride values.
you may modify the VI sources, to revise the bytesperline as following for your use-case.
for example,
$public_sources/kernel_src/kernel/nvidia/drivers/media/platform/tegra/camera/vi/channel.c

static void tegra_channel_fmt_align(struct tegra_channel *chan,
                                const struct tegra_video_format *vfmt,
                                u32 *width, u32 *height, u32 *bytesperline)
{
...
        bpl = tegra_core_bytes_per_line(*width, align, vfmt);

        /* Align stride */
        if (chan->vi->fops->vi_stride_align)
                chan->vi->fops->vi_stride_align(&bpl);

[daniel.kammerer]

hello JerryChang
the problem still exists even if I hardcode the value bpl to 2240.