Intermittent Failure in Capturing 4K Camera Frames with v4l2-ctl

Autonomous Machines Jetson & Embedded Systems Jetson Orin NX
,
1

I hope this message finds you well. We are currently facing an intermittent issue while capturing frames from a 4K camera using the v4l2-ctl tool on our Jetson device. The peculiar behavior we observe is as follows:

Upon the first attempt, we can successfully capture frames from the camera using v4l2-ctl. However, subsequent attempts to capture frames fail. After waiting for a certain period, retrying the operation often results in a successful capture.

It’s essential to note that our camera setup operates in a unique manner: we configure the camera at the application layer. Specifically, when the driver performs streaming on/off operations, it doesn’t directly enable or disable streaming. Instead, the camera consistently outputs MIPI data, even during these operations. This setup is due to our application layer’s involvement in configuring the camera via I2C communication.

Considering this distinctive configuration, we are curious if there might be any internal processes within the driver that could affect the consistent retrieval of frames using v4l2-ctl. We suspect that the intermittent failures might be related to how the driver handles MIPI data when streaming operations are initiated at the driver level.

To assist you in diagnosing this issue comprehensively, we will also provide a detailed log of the camera in it get error.

Could you kindly assist us in understanding the internal processes related to streaming operations in the driver? Specifically, how does the driver handle MIPI data output when streaming is turned on or off at the driver level? Understanding this mechanism could provide valuable insights into the intermittent frame capture issue we are experiencing.

Additionally, if there are any specific settings, configurations, or synchronization mechanisms that we should consider at the application layer to ensure stable frame captures, we would greatly appreciate your guidance.

Thank you very much for your time and assistance. We look forward to your insights into this matter.

Warm regards,

kworker/2:2-88 [002] .... 674.804815: rtcpu_string: tstamp:21799961075 id:0x04010000 str:"VM0 deactivating."
v4l2-ctl-14060 [004] .... 680.661910: tegra_channel_open: vi-output, max96724 2-0004
v4l2-ctl-14060 [004] .... 680.675351: tegra_channel_set_power: max96724 2-0004 : 0x1
v4l2-ctl-14060 [004] .... 680.675368: camera_common_s_power: status : 0x1
v4l2-ctl-14060 [004] .... 680.675377: tegra_channel_set_power: 13e40000.host1x:nvcsi@15a00000- : 0x1
v4l2-ctl-14060 [004] .... 680.675379: csi_s_power: enable : 0x1
v4l2-ctl-14060 [004] .... 680.675951: tegra_channel_capture_setup: vnc_id 0 W 3840 H 2160 fmt 13
v4l2-ctl-14060 [004] .... 680.681906: tegra_channel_set_stream: enable : 0x1
v4l2-ctl-14060 [004] .... 680.687417: tegra_channel_set_stream: 13e40000.host1x:nvcsi@15a00000- : 0x1
v4l2-ctl-14060 [004] .... 680.687419: csi_s_stream: enable : 0x1
v4l2-ctl-14060 [004] .... 680.687734: tegra_channel_set_stream: max96724 2-0004 : 0x1
kworker/2:2-88 [002] .... 680.732904: rtcpu_string: tstamp:21984128764 id:0x04010000 str:"VM0 activating."
kworker/2:2-88 [002] .... 680.732908: rtcpu_vinotify_event: tstamp:21984573781 cch:0 vi:0 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:703495867456 data:0x719d580010000000
kworker/2:2-88 [002] .... 680.732908: rtcpu_vinotify_event: tstamp:21984573943 cch:0 vi:0 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:703495873888 data:0x0000000031000001
kworker/2:2-88 [002] .... 680.732909: rtcpu_vinotify_event: tstamp:21984574099 cch:0 vi:0 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:703495914304 data:0x719d550010000000
kworker/2:2-88 [002] .... 680.732909: rtcpu_vinotify_event: tstamp:21984574232 cch:0 vi:0 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:703495920864 data:0x0000000031000002
kworker/2:2-88 [002] .... 680.848714: rtcpu_nvcsi_intr: tstamp:21987808414 class:GLOBAL type:PHY_INTR0 phy:0 cil:0 st:0 vc:0 status:0x0e000000
kworker/2:2-88 [002] .... 680.848716: rtcpu_nvcsi_intr: tstamp:21987808414 class:GLOBAL type:PHY_INTR0 phy:0 cil:1 st:0 vc:0 status:0x06000000
kworker/2:2-88 [002] .... 680.960589: rtcpu_nvcsi_intr: tstamp:21991973855 class:GLOBAL type:PHY_INTR0 phy:0 cil:0 st:0 vc:0 status:0x0e000000
kworker/2:2-88 [002] .... 680.960591: rtcpu_nvcsi_intr: tstamp:21991973855 class:GLOBAL type:PHY_INTR0 phy:0 cil:1 st:0 vc:0 status:0x06000000
kworker/2:2-88 [002] .... 681.072767: rtcpu_nvcsi_intr: tstamp:21996139297 class:GLOBAL type:PHY_INTR0 phy:0 cil:0 st:0 vc:0 status:0x0e000000
kworker/2:2-88 [002] .... 681.072770: rtcpu_nvcsi_intr: tstamp:21996139297 class:GLOBAL type:PHY_INTR0 phy:0 cil:1 st:0 vc:0 status:0x06000000
kworker/2:2-88 [002] .... 681.240601: rtcpu_nvcsi_intr: tstamp:22000304734 class:GLOBAL type:PHY_INTR0 phy:0 cil:0 st:0 vc:0 status:0x0e000000
kworker/2:2-88 [002] .... 681.240605: rtcpu_nvcsi_intr: tstamp:22000304734 class:GLOBAL type:PHY_INTR0 phy:0 cil:1 st:0 vc:0 status:0x06000000
kworker/2:2-88 [002] .... 681.356569: rtcpu_nvcsi_intr: tstamp:22004470175 class:GLOBAL type:PHY_INTR0 phy:0 cil:0 st:0 vc:0 status:0x0e000000
kworker/2:2-88 [002] .... 681.356572: rtcpu_nvcsi_intr: tstamp:22004470175 class:GLOBAL type:PHY_INTR0 phy:0 cil:1 st:0 vc:0 status:0x06000000
kworker/2:2-88 [002] .... 681.468593: rtcpu_nvcsi_intr: tstamp:22008635615 class:GLOBAL type:PHY_INTR0 phy:0 cil:0 st:0 vc:0 status:0x0e000000
kworker/2:2-88 [002] .... 681.468596: rtcpu_nvcsi_intr: tstamp:22008635615 class:GLOBAL type:PHY_INTR0 phy:0 cil:1 st:0 vc:0 status:0x06000000
kworker/2:2-88 [002] .... 681.636730: rtcpu_nvcsi_intr: tstamp:22012801054 class:GLOBAL type:PHY_INTR0 phy:0 cil:0 st:0 vc:0 status:0x0e000000
kworker/2:2-88 [002] .... 681.636732: rtcpu_nvcsi_intr: tstamp:22012801054 class:GLOBAL type:PHY_INTR0 phy:0 cil:1 st:0 vc:0 status:0x06000000
kworker/2:2-88 [002] .... 681.748730: rtcpu_nvcsi_intr: tstamp:22016966495 class:GLOBAL type:PHY_INTR0 phy:0 cil:0 st:0 vc:0 status:0x0e000000
kworker/2:2-88 [002] .... 681.748733: rtcpu_nvcsi_intr: tstamp:22016966495 class:GLOBAL type:PHY_INTR0 phy:0 cil:1 st:0 vc:0 status:0x06000000
kworker/2:2-88 [002] .... 681.864622: rtcpu_nvcsi_intr: tstamp:22021131936 class:GLOBAL type:PHY_INTR0 phy:0 cil:0 st:0 vc:0 status:0x0e000000
kworker/2:2-88 [002] .... 681.864626: rtcpu_nvcsi_intr: tstamp:22021131936 class:GLOBAL type:PHY_INTR0 phy:0 cil:1 st:0 vc:0 status:0x06000000
kworker/2:2-88 [002] .... 682.032719: rtcpu_nvcsi_intr: tstamp:22025297374 class:GLOBAL type:PHY_INTR0 phy:0 cil:0 st:0 vc:0 status:0x0e000000
kworker/2:2-88 [002] .... 682.032722: rtcpu_nvcsi_intr: tstamp:22025297374 class:GLOBAL type:PHY_INTR0 phy:0 cil:1 st:0 vc:0 status:0x06000000
kworker/2:2-88 [002] .... 682.144716: rtcpu_nvcsi_intr: tstamp:22029462815 class:GLOBAL type:PHY_INTR0 phy:0 cil:0 st:0 vc:0 status:0x0e000000
kworker/2:2-88 [002] .... 682.144719: rtcpu_nvcsi_intr: tstamp:22029462815 class:GLOBAL type:PHY_INTR0 phy:0 cil:1 st:0 vc:0 status:0x06000000
kworker/2:2-88 [002] .... 682.312712: rtcpu_nvcsi_intr: tstamp:22033628256 class:GLOBAL type:PHY_INTR0 phy:0 cil:0 st:0 vc:0 status:0x0e000000
kworker/2:2-88 [002] .... 682.312715: rtcpu_nvcsi_intr: tstamp:22033628256 class:GLOBAL type:PHY_INTR0 phy:0 cil:1 st:0 vc:0 status:0x06000000
kworker/2:2-88 [002] .... 682.424655: rtcpu_nvcsi_intr: tstamp:22037793694 class:GLOBAL type:PHY_INTR0 phy:0 cil:0 st:0 vc:0 status:0x0e000000
kworker/2:2-88 [002] .... 682.424658: rtcpu_nvcsi_intr: tstamp:22037793694 class:GLOBAL type:PHY_INTR0 phy:0 cil:1 st:0 vc:0 status:0x06000000
kworker/2:2-88 [002] .... 682.536527: rtcpu_nvcsi_intr: tstamp:22041959135 class:GLOBAL type:PHY_INTR0 phy:0 cil:0 st:0 vc:0 status:0x0e000000
kworker/2:2-88 [002] .... 682.536528: rtcpu_nvcsi_intr: tstamp:22041959135 class:GLOBAL type:PHY_INTR0 phy:0 cil:1 st:0 vc:0 status:0x06000000
kworker/2:2-88 [002] .... 682.704550: rtcpu_nvcsi_intr: tstamp:22046124573 class:GLOBAL type:PHY_INTR0 phy:0 cil:0 st:0 vc:0 status:0x0e000000
kworker/2:2-88 [002] .... 682.704554: rtcpu_nvcsi_intr: tstamp:22046124573 class:GLOBAL type:PHY_INTR0 phy:0 cil:1 st:0 vc:0 status:0x06000000
kworker/2:2-88 [002] .... 682.816691: rtcpu_nvcsi_intr: tstamp:22050290014 class:GLOBAL type:PHY_INTR0 phy:0 cil:0 st:0 vc:0 status:0x0e000000
kworker/2:2-88 [002] .... 682.816694: rtcpu_nvcsi_intr: tstamp:22050290014 class:GLOBAL type:PHY_INTR0 phy:0 cil:1 st:0 vc:0 status:0x06000000
kworker/2:2-88 [002] .... 682.932527: rtcpu_nvcsi_intr: tstamp:22054455455 class:GLOBAL type:PHY_INTR0 phy:0 cil:0 st:0 vc:0 status:0x0e000000
kworker/2:2-88 [002] .... 682.932530: rtcpu_nvcsi_intr: tstamp:22054455455 class:GLOBAL type:PHY_INTR0 phy:0 cil:1 st:0 vc:0 status:0x06000000
kworker/2:2-88 [002] .... 683.100680: rtcpu_nvcsi_intr: tstamp:22058620893 class:GLOBAL type:PHY_INTR0 phy:0 cil:0 st:0 vc:0 status:0x0e000000
kworker/2:2-88 [002] .... 683.100682: rtcpu_nvcsi_intr: tstamp:22058620893 class:GLOBAL type:PHY_INTR0 phy:0 cil:1 st:0 vc:0 status:0x06000000
kworker/2:2-88 [002] .... 683.212669: rtcpu_nvcsi_intr: tstamp:22062786334 class:GLOBAL type:PHY_INTR0 phy:0 cil:0 st:0 vc:0 status:0x0e000000
kworker/2:2-88 [002] .... 683.212672: rtcpu_nvcsi_intr: tstamp:22062786334 class:GLOBAL type:PHY_INTR0 phy:0 cil:1 st:0 vc:0 status:0x06000000
kworker/2:2-88 [002] .... 683.380530: rtcpu_nvcsi_intr: tstamp:22066951775 class:GLOBAL type:PHY_INTR0 phy:0 cil:0 st:0 vc:0 status:0x0e000000
kworker/2:2-88 [002] .... 683.380533: rtcpu_nvcsi_intr: tstamp:22066951775 class:GLOBAL type:PHY_INTR0 phy:0 cil:1 st:0 vc:0 status:0x06000000
v4l2-ctl-14060 [002] .... 683.465212: tegra_channel_close: vi-output, max96724 2-0004
vi-output, max9-14062 [005] .... 683.492656: tegra_channel_capture_setup: vnc_id 0 W 3840 H 2160 fmt 13
kworker/2:2-88 [002] .... 683.496500: rtcpu_nvcsi_intr: tstamp:22071117213 class:GLOBAL type:PHY_INTR0 phy:0 cil:0 st:0 vc:0 status:0x0e000000
kworker/2:2-88 [002] .... 683.496501: rtcpu_nvcsi_intr: tstamp:22071117213 class:GLOBAL type:PHY_INTR0 phy:0 cil:1 st:0 vc:0 status:0x06000000
v4l2-ctl-14060 [002] .... 683.502394: tegra_channel_set_stream: enable : 0x0
v4l2-ctl-14060 [002] .... 683.502396: tegra_channel_set_stream: max96724 2-0004 : 0x0
v4l2-ctl-14060 [002] .... 683.502424: tegra_channel_set_stream: 13e40000.host1x:nvcsi@15a00000- : 0x0
v4l2-ctl-14060 [002] .... 683.502426: csi_s_stream: enable : 0x0
v4l2-ctl-14060 [003] .... 683.505315: tegra_channel_set_power: max96724 2-0004 : 0x0
v4l2-ctl-14060 [003] .... 683.505344: camera_common_s_power: status : 0x0
v4l2-ctl-14060 [003] .... 683.505354: tegra_channel_set_power: 13e40000.host1x:nvcsi@15a00000- : 0x0
v4l2-ctl-14060 [003] .... 683.505357: csi_s_power: enable : 0x0
kworker/2:2-88 [002] .... 689.416299: rtcpu_string: tstamp:22257158262 id:0x04010000 str:"VM0 deactivating."
v4l2-ctl-14761 [003] .... 724.036078: tegra_channel_open: vi-output, max96724 2-0004
v4l2-ctl-14761 [003] .... 724.049573: tegra_channel_set_power: max96724 2-0004 : 0x1
v4l2-ctl-14761 [003] .... 724.049589: camera_common_s_power: status : 0x1
v4l2-ctl-14761 [003] .... 724.049598: tegra_channel_set_power: 13e40000.host1x:nvcsi@15a00000- : 0x1
v4l2-ctl-14761 [003] .... 724.049600: csi_s_power: enable : 0x1
v4l2-ctl-14761 [003] .... 724.050157: tegra_channel_capture_setup: vnc_id 0 W 3840 H 2160 fmt 13
v4l2-ctl-14761 [003] .... 724.057288: tegra_channel_set_stream: enable : 0x1
v4l2-ctl-14761 [003] .... 724.062248: tegra_channel_set_stream: 13e40000.host1x:nvcsi@15a00000- : 0x1
v4l2-ctl-14761 [003] .... 724.062249: csi_s_stream: enable : 0x1
v4l2-ctl-14761 [003] .... 724.062556: tegra_channel_set_stream: max96724 2-0004 : 0x1
kworker/2:2-88 [002] .... 724.095522: rtcpu_string: tstamp:23339607575 id:0x04010000 str:"VM0 activating."
kworker/2:2-88 [002] .... 724.095525: rtcpu_vinotify_event: tstamp:23340055734 cch:0 vi:0 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:746871275552 data:0x719d580010000000
kworker/2:2-88 [002] .... 724.095526: rtcpu_vinotify_event: tstamp:23340055870 cch:0 vi:0 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:746871281984 data:0x0000000031000001
kworker/2:2-88 [002] .... 724.095526: rtcpu_vinotify_event: tstamp:23340056024 cch:0 vi:0 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:746871322784 data:0x719d550010000000
kworker/2:2-88 [002] .... 724.095527: rtcpu_vinotify_event: tstamp:23340056158 cch:0 vi:0 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:746871329344 data:0x0000000031000002
2

BTW,
My platform is 8G Orin NX and custom board.
Jetpack Version:5.1.1
BSP version:35.3.1

4

hello HuangZeng,

there’re PHY interrupt errors.
it means that DPHY deskew calibration has not complete, it usually happened when the calibration sequence length is not long enough.
please review the sensor configuration, you may also try to adjust cil_settletime property for testing.

furthermore,
since it’s related to deskew, is it possible to lower the data rate below 1.5Gbps for checking the streaming stability?
if yes… you may upgrade to JP-5.1.2/ l4t-r35.4.1 to include some important fixes.
re-cap Release Notes (r35.4.1) as below…

1.2. What’s New
● Camera
○ Enhanced error resiliency for improved stability in Argus.
○ Added support for multiple camera synchronization (added sample argus_syncstereo).
○ Added support for Deskew calibration for high data rate sensors (> 1.5 Gbps).

5

hello , JerryChang
I would like to understand the specific calculation method for cil_settletime . Could you provide some examples to help me understand?
Additionally, I am curious about the fact that if I reset the camera output, it immediately becomes able to capture image data again. Why does this happen?

6

hello HuangZeng,

please refer to Orin TRM and checking settle time for more details.

may I know what’s the device it is? is it something like a bridge driver for delivering frame packages to CSI channel?
I did see some issue with SerDes chips, it’ll need to issue a reset (on the serdes chip) before s_stream() to have good packet.

7

Hi, JerryChang,

Yes, we are using the GMSL chip as the bridge chip. We have not followed the s_stream to control the streaming, instead, we keep sending MIPI data continuously. Interestingly, it works fine in low-resolution scenarios, but in high-resolution or large data volume situations, it always takes some time to reacquire the stream.

Best regards.

9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.