MIPI video streaming on Orin Nano

Autonomous Machines Jetson & Embedded Systems Jetson Orin Nano
1

I am having an issue with streaming via MIPI on CAM0 using an Orin Nano dev kit (8GB).

I have a device tree file and driver code. Basically I get /dev/video0 when loading the driver module. However when use the following gstreamer command I get a green screen.

gst-launch-1.0 v4l2src device=/dev/video0 ! ‘video/x-raw,pixelformat=YUY2,width=720,height=720,framerate=30/1’ ! queue ! videoconvert ! queue ! xvimagesink

I am also having trouble enabling dynamic debug. If I copy over the Image file it cannot find the Partition ID of the NVME drive I am running on.

Here is the gstreamer output:

Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Pipeline is PREROLLED …
Setting pipeline to PLAYING …
WARNING: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Signal lost
Additional debug info:
…/sys/v4l2/gstv4l2src.c(556): gst_v4l2src_query_preferred_size (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
No input source was detected - video frames invalid
New clock: GstSystemClock
Redistribute latency…
ERROR: from element /GstPipeline:pipeline0/GstXvImageSink:xvimagesink0: Output window was closed
Additional debug info:
…/sys/xvimage/xvimagesink.c(568): gst_xv_image_sink_handle_xevents (): /GstPipeline:pipeline0/GstXvImageSink:xvimagesink0
Execution ended after 0:00:10.774681097
Setting pipeline to NULL …
Freeing pipeline …

What is the best way to “see” the MIPI packets in a trace log to ensure I am at least getting a signal through?

2

Hello @walker.newell,

Have you tried capturing with v4l2-ctl ?

That should at leat remove tegracam from the equation and simplify a bit the system.

Regards,
Andrew

3

Yes, I have attempted to a raw output file like this:

v4l2-ctl --verbose --device=/dev/video0 --set-fmt-video=width=720,height=720,pixelformat=YUYV --stream-mmap --stream-to=output.raw --stream-count=200
VIDIOC_QUERYCAP: ok
VIDIOC_G_FMT: ok
VIDIOC_S_FMT: ok
Format Video Capture:
Width/Height : 720/720
Pixel Format : ‘YUYV’ (YUYV 4:2:2)
Field : None
Bytes per Line : 1440
Size Image : 1036800
Colorspace : sRGB
Transfer Function : Default (maps to sRGB)
YCbCr/HSV Encoding: Default (maps to ITU-R 601)
Quantization : Default (maps to Limited Range)
Flags :
VIDIOC_REQBUFS returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_STREAMON returned 0 (Success)
cap dqbuf: 0 seq: 0 bytesused: 1036800 ts: 0.000000 (error, ts-monotonic, ts-src-eof)
cap dqbuf: 0 seq: 0 bytesused: 1036800 ts: 0.000000 (error, ts-monotonic, ts-src-eof)
cap dqbuf: 1 seq: 1 bytesused: 1036800 ts: 0.000000 (error, ts-monotonic, ts-src-eof)
cap dqbuf: 2 seq: 2 bytesused: 1036800 ts: 0.000000 (error, ts-monotonic, ts-src-eof)
cap dqbuf: 3 seq: 3 bytesused: 1036800 ts: 0.000000 (error, ts-monotonic, ts-src-eof)
cap dqbuf: 0 seq: 0 bytesused: 1036800 ts: 0.000000 (error, ts-monotonic, ts-src-eof)
cap dqbuf: 0 seq: 0 bytesused: 1036800 ts: 0.000000 (error, ts-monotonic, ts-src-eof)
cap dqbuf: 1 seq: 1 bytesused: 1036800 ts: 0.000000 (error, ts-monotonic, ts-src-eof)
cap dqbuf: 2 seq: 2 bytesused: 1036800 ts: 0.000000 (error, ts-monotonic, ts-src-eof)
cap dqbuf: 3 seq: 3 bytesused: 1036800 ts: 0.000000 (error, ts-monotonic, ts-src-eof)

5

hello walker.newell

I doubt it shown incorrect timestamp.
for instance,

cap dqbuf: 0 seq: 0 bytesused: 1036800 ts: 0.000000 (error, ts-monotonic, ts-src-eof)

could you please confirm…
(1) please check it’s outputting validate contents to output.raw
(2) let’s simply check the streaming by skipping --stream-to options, it shall print frame-rate per second below the pipeline.

6

The output.raw file is created but it is zero length. I don’t believe that I am getting valid packets and not sure how to troubleshoot in terms of seeing the CSI packet information in a trace log? Could you also provide the v4l2-ctl command without stream-to option? I ran it without stream-to and got the same output.

7

hello walker.newell,

it’s simply removing --stream-to option for fetching stream via v4l2,
for example, $ v4l2-ctl --verbose --device=/dev/video0 --set-fmt-video=width=720,height=720,pixelformat=YUYV --stream-mmap --stream-count=200

that should output < to indicate a frame has captured, and report frame-rate per second below the pipeline.
there should be streaming issue if that did not output anything, please dig into kernel logs, $ dmesg > klog.txt for investigation.

besides, may I also confirm which Jetpack release version you’re working with,
you should check the release tag, i.e. $ cat /etc/nv_tegra_release for confirmation.

8

removing –stream-to option produced no output.

I am attaching the dmesg output (klog.txt).
klog.txt (77.4 KB)

The main error I see in the logs after the start streaming is:
tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms

Jetpack version is 36.3 as shown below:

$ cat /etc/nv_tegra_release

R36 (release), REVISION: 3.0, GCID: 36191598, BOARD: generic, EABI: aarch64, DATE: Mon May 6 17:34:21 UTC 2024

KERNEL_VARIANT: oot

TARGET_USERSPACE_LIB_DIR=nvidia
TARGET_USERSPACE_LIB_DIR_PATH=usr/lib/aarch64-linux-gnu/nvidia

9

hello walker.newell,

according to the logs,
VI did not receiving validate frames, and it’s reporting time out failure.
for instance,

[   96.502180] ov131_isp 10-0036: vc_core_set_format(): Set format 0x2011 (YUY2)
[   96.502193] ov131_isp 10-0036: vc_core_try_format(): Try format 0x2011 (YUY2, format: 0x2a)
[   96.502197] ov131_isp 10-0036: vc_update_image_size_from_mode(): Update image size from mode0 (w: 720, h: 720)
[   96.702434] ov131_isp 10-0036: vc_sen_start_stream(): Start streaming
[   99.143699] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[   99.143732] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel

FYI,
you may see-also Jetson Orin Nano Developer Kit Carrier Board Specification for [3.2 Camera Connector].
The connectors support the following.

J20: CSI 1 x2 lane
J21: CSI 1 x2 lane or 1 x4 lane

CAM0 camera slot (J20) now streams on CSI-B instead of CSI-A. it’s only supported with 1-lane or 2-lane configuration.
please double check your device tree settings, you may connect to CAM1 if that’s 4-lane configure.
besides,
CSI0 D1 and CSI1 D0 P/N will always been swizzled for P/N, you may use device tree property, lane_polarity to configure a polarity swap on any lane.

10

Thanks for the feedback!

So what Port Index would be associated with CSI-B, is it (2). Also what serial address (serial_a or serial_b) this is for CAM0.

11

hello walker.newell,

you may refer to Topic 69753 for the explanation for port binding.

13

Still would like some help on setting up Dynamic Debug on the Orin Nano so I can see debug output.

14

hello walker.newell,

VI tracing logs /sys/kernel/debug/camrtc/log-level is disabled by default.
please probe the debug module to enable it, i.e. $ modprobe rtcpu_debug

after that,
you may follow below steps to enable VI tracing logs for gathering more details.

echo 1 > /sys/kernel/debug/tracing/tracing_on
echo 30720 > /sys/kernel/debug/tracing/buffer_size_kb
echo 1 > /sys/kernel/debug/tracing/events/tegra_rtcpu/enable
echo 1 > /sys/kernel/debug/tracing/events/freertos/enable
echo 2 > /sys/kernel/debug/camrtc/log-level
echo > /sys/kernel/debug/tracing/trace
cat /sys/kernel/debug/tracing/trace
15

Here is some output after I enabled the trace. Not sure how to interpret? Is there a reference document?

kworker/4:1-49 [004] … 1661.111063: rtcpu_string: tstamp:52540489909 id:0x04010000 str:“VM0 activating.”
kworker/4:1-49 [004] … 1661.111070: rtcpu_vinotify_event: tstamp:52541230222 cch:0 vi:0 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:1681302262912 data:0x799d580010000000
kworker/4:1-49 [004] … 1661.111071: rtcpu_vinotify_event: tstamp:52541230516 cch:0 vi:0 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:1681302319872 data:0x0000000031000001
kworker/4:1-49 [004] … 1661.111071: rtcpu_vinotify_event: tstamp:52541230807 cch:0 vi:0 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:1681302337984 data:0x799d550010000000
kworker/4:1-49 [004] … 1661.111071: rtcpu_vinotify_event: tstamp:52541231063 cch:0 vi:0 tag:VIFALC_ACTIONLST channel:0x23 frame:0 vi_tstamp:1681302358272 data:0x0000000007020001
kworker/4:1-49 [004] … 1661.111072: rtcpu_vinotify_event: tstamp:52541231345 cch:0 vi:0 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:1681302400672 data:0x0000000031000002
kworker/4:1-49 [004] … 1663.859046: rtcpu_vinotify_event: tstamp:52626408619 cch:0 vi:0 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:1684030267808 data:0x799d580010000000
kworker/4:1-49 [004] … 1663.859050: rtcpu_vinotify_event: tstamp:52626408916 cch:0 vi:0 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:1684030324736 data:0x0000000031000001
kworker/4:1-49 [004] … 1663.859051: rtcpu_vinotify_event: tstamp:52626409174 cch:0 vi:0 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:1684030342880 data:0x799d550010000000
kworker/4:1-49 [004] … 1663.859052: rtcpu_vinotify_event: tstamp:52626409458 cch:0 vi:0 tag:VIFALC_ACTIONLST channel:0x23 frame:0 vi_tstamp:1684030363200 data:0x0000000007020001
kworker/4:1-49 [004] … 1663.859052: rtcpu_vinotify_event: tstamp:52626409712 cch:0 vi:0 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:1684030405568 data:0x0000000031000002
kworker/4:1-49 [004] … 1666.391020: rtcpu_vinotify_event: tstamp:52706162527 cch:0 vi:0 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:1686590347296 data:0x799d580010000000
kworker/4:1-49 [004] … 1666.391024: rtcpu_vinotify_event: tstamp:52706162788 cch:0 vi:0 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:1686590389376 data:0x0000000031000001
kworker/4:1-49 [004] … 1666.391025: rtcpu_vinotify_event: tstamp:52706163073 cch:0 vi:0 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:1686590407520 data:0x799d550010000000
kworker/4:1-49 [004] … 1666.391025: rtcpu_vinotify_event: tstamp:52706163323 cch:0 vi:0 tag:VIFALC_ACTIONLST channel:0x23 frame:0 vi_tstamp:1686590427840 data:0x0000000007020001
kworker/4:1-49 [004] … 1666.391026: rtcpu_vinotify_event: tstamp:52706163605 cch:0 vi:0 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:1686590470208 data:0x0000000031000002

16

hello walker.newell,

that’s all tracing logs?
I don’t see any logs reported by VI engine, which meant there’s no validate frame packets on CSI channel.

17

What should the VI engine output look like in terms of specific events?

Thanks!

18

hello walker.newell,

let me give you VI tracing logs with a success image capture as an example,
for instance,
there must be one pair of CHANSEL_PXL_SOF/CHANSEL_PXL_EOF to indicate a frame has detected by VI engine.
afterwards, it’s ATOMP_FRAME_DONE to indicate it’s complete writing a frame to memory.

rtcpu_vinotify_event: tstamp:4058867917 cch:-1 vi:0 tag:CSIMUX_STREAM channel:0x00 frame:0 vi_tstamp:129874754944 data:0x0000000000000001
rtcpu_vinotify_event: tstamp:4059206674 cch:0 vi:0 tag:ATOMP_FS channel:0x00 frame:1 vi_tstamp:129891804896 data:0x0000000800000000
rtcpu_vinotify_event: tstamp:4059206818 cch:0 vi:0 tag:CHANSEL_PXL_SOF channel:0x23 frame:1 vi_tstamp:129891808928 data:0x0000000000000001
rtcpu_vinotify_event: tstamp:4059206976 cch:0 vi:0 tag:VIFALC_ACTIONLST channel:0x23 frame:1 vi_tstamp:129891815424 data:0x0000000008020001
rtcpu_vinotify_error: tstamp:4060164160 cch:0 vi:0 tag:CSIMUX_FRAME channel:0x00 frame:1 vi_tstamp:129925171392 data:0x00000000000000a0
rtcpu_vinotify_event: tstamp:4060166846 cch:0 vi:0 tag:CHANSEL_PXL_EOF channel:0x23 frame:1 vi_tstamp:129923836800 data:0x0000000004370002
rtcpu_vinotify_event: tstamp:4060167015 cch:0 vi:0 tag:ATOMP_FRAME_DONE channel:0x23 frame:1 vi_tstamp:129923837312 data:0x0000000000000000
19

Thanks again for your quick responses.

We have some new log entries after working with the device tree file. Can you please take a look and let us know the current state:

kworker/5:1-50 [005] … 2625.796974: rtcpu_string: tstamp:82689018536 id:0x04010000 str:“VM0 activating.”
kworker/5:1-50 [005] … 2625.796983: rtcpu_vinotify_event: tstamp:82689789761 cch:0 vi:0 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:2646056136320 data:0x799d580010000000
kworker/5:1-50 [005] … 2625.796984: rtcpu_vinotify_event: tstamp:82689790028 cch:0 vi:0 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:2646056208992 data:0x0000000031000001
kworker/5:1-50 [005] … 2625.796985: rtcpu_vinotify_event: tstamp:82689790319 cch:0 vi:0 tag:VIFALC_ACTIONLST channel:0x23 frame:0 vi_tstamp:2646056230688 data:0x0000000007020001
kworker/5:1-50 [005] … 2625.796985: rtcpu_vinotify_event: tstamp:82689790574 cch:0 vi:0 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:2646056298208 data:0x799d550010000000
kworker/5:1-50 [005] … 2625.796986: rtcpu_vinotify_event: tstamp:82689790854 cch:0 vi:0 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:2646056340928 data:0x0000000031000002
kworker/5:1-50 [005] … 2628.540984: rtcpu_vinotify_event: tstamp:82774425317 cch:0 vi:0 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:2648771331296 data:0x799d580010000000
kworker/5:1-50 [005] … 2628.540988: rtcpu_vinotify_event: tstamp:82774425621 cch:0 vi:0 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:2648771388224 data:0x0000000031000001

20

Please disregard as these log files are a repeat from earlier.

21

I am now getting a deskew issue:
Class: Global
Type: PHY_Interrupt0
PHY: 0
CIL: 0
Stream: 0
VC: 0
Status: 134217728
Timestamp: 3339202237

PHY_Interrupt0 : 0x0000000008000000
-dphy_cil_deskew_calib_err_ctrl [27]: 1
DPHY deskew calibration not complete on clock lane [A/B]. Happen when the calibration sequence length is not long enough

How does it compute the need for Deskew from the device tree file? I have tried changing max lane speed and csi pixel depth but keep getting this error.

Thanks!

22

hello walker.newell,

deskew calibration is a must if data-rate > 1.5 Gbps, Else the camera firmware will continue to wait for deskew signal from the sensor side. it’ll enable pixel parser when deskew calibration has completed.
you may try tune cil_settletime, please see-also Property-Value Pairs for calculation formulas.

besides, please refer to Infinite Timeout Support to test with enableCamInfiniteTimeout=1 .
BTW, since JetPack 6.1 is now available, is it possible for moving to the latest release version for verification.