Can record using gstreamer but not with v4l2-ctl (frame start syncpt timeout) on IMX-678

I can record with no issue using nvarguscamerasrc with gst-launch-1.0 without issue.

However, when I try to record using v4l2-ctl directly it fails with:

Feb 13 14:38:44 localhost kernel: [ 216.695520] imx678 8-001a: Writing table
Feb 13 14:38:44 localhost kernel: [ 216.931552] video4linux video1: frame start syncpt timeout!0
Feb 13 14:38:44 localhost kernel: [ 216.939530] video4linux video1: TEGRA_VI_CSI_ERROR_STATUS 0x00000000
Feb 13 14:38:44 localhost kernel: [ 216.939589] vi 54080000.vi: TEGRA_CSI_PIXEL_PARSER_STATUS 0x00004000
Feb 13 14:38:44 localhost kernel: [ 216.939639] vi 54080000.vi: TEGRA_CSI_CIL_STATUS 0x00000012
Feb 13 14:38:44 localhost kernel: [ 216.939685] vi 54080000.vi: TEGRA_CSI_CILX_STATUS 0x00060061
Feb 13 14:38:44 localhost kernel: [ 216.939881] vi 54080000.vi: cil_settingtime was autocalculated
Feb 13 14:38:44 localhost kernel: [ 216.939903] vi 54080000.vi: csi clock settle time: 13, cil settle time: 10
Feb 13 14:38:44 localhost kernel: [ 217.143319] video4linux video1: frame start syncpt timeout!0
Feb 13 14:38:44 localhost kernel: [ 217.151864] video4linux video1: TEGRA_VI_CSI_ERROR_STATUS 0x00000000
Feb 13 14:38:44 localhost kernel: [ 217.151886] vi 54080000.vi: TEGRA_CSI_PIXEL_PARSER_STATUS 0x00004000
Feb 13 14:38:44 localhost kernel: [ 217.151903] vi 54080000.vi: TEGRA_CSI_CIL_STATUS 0x00000012
Feb 13 14:38:44 localhost kernel: [ 217.151918] vi 54080000.vi: TEGRA_CSI_CILX_STATUS 0x00040040
Feb 13 14:38:44 localhost kernel: [ 217.152049] vi 54080000.vi: cil_settingtime was autocalculated
Feb 13 14:38:44 localhost kernel: [ 217.152071] vi 54080000.vi: csi clock settle time: 13, cil settle time: 10
Feb 13 14:38:44 localhost kernel: [ 217.355208] video4linux video1: frame start syncpt timeout!0
Feb 13 14:38:44 localhost kernel: [ 217.363624] video4linux video1: TEGRA_VI_CSI_ERROR_STATUS 0x00000000
Feb 13 14:38:44 localhost kernel: [ 217.363646] vi 54080000.vi: TEGRA_CSI_PIXEL_PARSER_STATUS 0x00004000
Feb 13 14:38:44 localhost kernel: [ 217.363662] vi 54080000.vi: TEGRA_CSI_CIL_STATUS 0x00000012
Feb 13 14:38:44 localhost kernel: [ 217.363678] vi 54080000.vi: TEGRA_CSI_CILX_STATUS 0x00040040
Feb 13 14:38:44 localhost kernel: [ 217.363800] vi 54080000.vi: cil_settingtime was autocalculated
Feb 13 14:38:44 localhost kernel: [ 217.363821] vi 54080000.vi: csi clock settle time: 13, cil settle time: 10
Feb 13 14:38:45 localhost kernel: [ 217.567112] video4linux video1: frame start syncpt timeout!0
Feb 13 14:38:45 localhost kernel: [ 217.575547] video4linux video1: TEGRA_VI_CSI_ERROR_STATUS 0x00000000
Feb 13 14:38:45 localhost kernel: [ 217.575567] vi 54080000.vi: TEGRA_CSI_PIXEL_PARSER_STATUS 0x00004000
Feb 13 14:38:45 localhost kernel: [ 217.575583] vi 54080000.vi: TEGRA_CSI_CIL_STATUS 0x00000002
Feb 13 14:38:45 localhost kernel: [ 217.575599] vi 54080000.vi: TEGRA_CSI_CILX_STATUS 0x00000000
Feb 13 14:38:45 localhost kernel: [ 217.575700] vi 54080000.vi: cil_settingtime was autocalculated
Feb 13 14:38:45 localhost kernel: [ 217.575720] vi 54080000.vi: csi clock settle time: 13, cil settle time: 10
Feb 13 14:38:45 localhost kernel: [ 217.779319] video4linux video1: frame start syncpt timeout!0
Feb 13 14:38:45 localhost kernel: [ 217.787405] video4linux video1: TEGRA_VI_CSI_ERROR_STATUS 0x00000000
Feb 13 14:38:45 localhost kernel: [ 217.787427] vi 54080000.vi: TEGRA_CSI_PIXEL_PARSER_STATUS 0x00004000
Feb 13 14:38:45 localhost kernel: [ 217.787443] vi 54080000.vi: TEGRA_CSI_CIL_STATUS 0x00000012
Feb 13 14:38:45 localhost kernel: [ 217.787458] vi 54080000.vi: TEGRA_CSI_CILX_STATUS 0x00040040
Feb 13 14:38:45 localhost kernel: [ 217.787574] vi 54080000.vi: cil_settingtime was autocalculated
Feb 13 14:38:45 localhost kernel: [ 217.787595] vi 54080000.vi: csi clock settle time: 13, cil settle time: 10

This thread gave me a small clue: Nano 2gb + 2 max9296 + 4 max9295 + gw5300isp + 4 imx490 - #14 by user128497

But what I don’t understand is why nvarguscamerasrc can start recording while v4l2-ctl has issues given they go through the same start/stop sequence in the driver. Because one works and the other doesn’t, I feel this maybe a device tree issue (yes, I double checked the pixel clock hz and it seems right (at least now) based on the formula).

This is JP 32.5.1.

I did see this thread: 1080p60 capture failed

And tried to play with the cil_settletime to see if there is some clocking issue with timeouts (set it to 0). Still see syncpt timeouts.

EDIT: I just realized that the TEGRA_CSI_PIXEL_PARSER_STATUS:

HPA_UNC_HDR_ERR: Uncorrectable Header Error. Set when header parser A parses a header with a multi bit error. This error will be detected by the headers ECC, but can't be corrected. The packet will be discarded.

Seems like a signaling/clocking problem then?

Hi,
Please share your v4l2-ctl command for reference.

@DaneLLL v4l2-ctl -d /dev/video1 --set-fmt-video=width=3856,height=2180,pixelformat=RG12 --set-ctrl bypass_mode=0 --set-ctrl frame_rate=30000000 --stream-mmap --stream-count=60 --stream-to=file.raw

Again, what’s strange is nvarguscamerasrc seems to work without issue. I would expect both to fail if it was purely clocking.

Hi,
Please run $ v4l2-ctl -d /dev/video1 --list-formats-ext and share the prints for reference.

	Index       : 0
	Type        : Video Capture
	Pixel Format: 'RG12'
	Name        : 12-bit Bayer RGRG/GBGB
		Size: Discrete 3856x2180
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 1920x1080
			Interval: Discrete 0.033s (30.000 fps)

hello alex466,

please try adding --stream-skip options to bypass several frames (in the beginning) for testing.
for instance, $ v4l2-ctl -d /dev/video1 --set-fmt-video=width=3856,height=2180,pixelformat=RG12 --set-ctrl bypass_mode=0 --set-ctrl frame_rate=30000000 --stream-skip=9 --stream-mmap --stream-count=60 --stream-to=file.raw

Same thing. I tried 3, 10, 50, and 100.

hello alex466,

is it your 3856x2180 sensor mode had longer initial time for start sensor streaming?
please give it a try for launching 1080p sensor mode,
for instance,
$ v4l2-ctl -d /dev/video1 --set-fmt-video=width=1920,height=1080,pixelformat=RG12 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=100

Same issue.

Again, any idea why gstreamer/nvarguscamerasrc works just fine but v4l2-ctl doesn’t? What are the TEGRA error registers really saying?

hello alex466,

according to Camera Architecture Stack, v4l2 and Argus they’re using different pipelines and going through different software stacks.
since you’re using JP 32.5.1, which is quite old release version. is it possible for moving to latest JP-4 version for verification?

BTW,
you may also refer to Nano’s CSI driver for CSI interrupts.
for instance, $public_sources/kernel_src/kernel/nvidia/drivers/media/platform/tegra/camera/csi/csi2_fops.c

int tegra_csi_error(struct tegra_csi_channel *chan, int port_idx)
{
...
        val = pp_read(port, TEGRA_CSI_PIXEL_PARSER_STATUS);
        err |= val & 0x4000;
        pp_write(port, TEGRA_CSI_PIXEL_PARSER_STATUS, val);

        val = cil_read(port, TEGRA_CSI_CIL_STATUS);
        err |= val & 0x02;
        cil_write(port, TEGRA_CSI_CIL_STATUS, val);

        val = cil_read(port, TEGRA_CSI_CILX_STATUS);
        err |= val & 0x00020020;
        cil_write(port, TEGRA_CSI_CILX_STATUS, val);

please refer to TRM, you may see-also register description, such as… CSI_CILA_STATUS_0 for analysis.

Yes, but what does it mean? The doc doesn’t really explain outside of it claiming to not receive frames.

Another idea: Has there been significant changes in this code path in newer JP releases?

hello alex466,

ya… it’s representing VI side did not receive frames.
may I also know what’s the actual use-case for running with v4l2?

Mainly debugging but we thought this might be an indication of some clocking issue or deeper hardware one. We’d still like to understand if this is a driver issue or a Tegra one.

hello alex466,

it should be the issue of driver side.

there’s similar issue with IMX477, which has resolved by updating init register settings.
please see-also Camera error - #13 by ShaneCCC for reference.

I saw it. This was upping the sleep time or issuing a reset after register write when the driver starts. (not that patch but a very similar one when the problem was first reported by ArduCam).

But explain to me why the current register settings works with nvarguscamerasrc but not straight through tegra? I did play with the IMX-678 registers a bit and reduce and add some time before frame starts but to no avail.

The spec sheet claims that the frames will be sent over the wire at about 24 ms or more. Currently the driver waits around that time after putting the sensor in operating mode from standby.

Also the NVIDIA driver conformance tests also pass @JerryChang

if there’s no strong request to stay on r32.5.1 for validation.
please moving forward to latest rel-32 (i.e. l4t-r32.7.4) for confirmation

We are in the midst of doing that now. Stay tuned.

Is this still an issue to support? Any result can be shared?

It is taking time. Will know soon. Still have not a clue what Tegra is saying with respect to why v4l2-tl is not working.