GMSL camera config issue, can't grab image

Robotics & Edge Computing Jetson Systems Jetson Thor
1

Hi, Nvidia team:

I config a GMSL camera in Thor, the video node can be found in /dev, but when I try to get video, it report error as follow:

car@ubuntu:~$ gst-launch-1.0 v4l2src device=/dev/video11 ! ‘video/x-raw,format=UYVY,width=1920,height=1080’ ! videoconvert ! fpsdisplaysink video-sink=xvimagesink e
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Pipeline is PREROLLED …
Setting pipeline to PLAYING …
New clock: GstSystemClock
WARNING: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Signal lost
Additional debug info:
../sys/v4l2/gstv4l2src.c(693): gst_v4l2src_query_preferred_size (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
No input source was detected - video frames invalid
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
../libs/gst/base/gstbasesrc.c(3177): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.002394991
Setting pipeline to NULL …
Freeing pipeline …

“

and I find some error report when kernel startup, log as follow, is it the matter?:

“[ 13.935260] tegra-camrtc-capture-vi tegra-capture-vi: subdev imx390 9-0021 bound
[ 13.953401] tegra-nvcsi 8181200000.host1x:nvcsi@8188000000: Failed to create device link (0x180) with 9-0021

“

I just make modification base on tegra264-p3971-camera-p3762-a00-overlay.dts .

There is no much error report in Orin at JP6.

The HW is Thor custom board, SW is JP7.0

Could you pls. provide any suggestion, many thanks.

BR//

Camera can't work
3

SW is JP7.0 GA release, right?

4

Yes,JP7.0_GA

5

hello ekeechg,

is this porting issue? did you validate this sensor driver on JP-6/Orin?

is the camera device registration complete to setup a video device node to linux kernel?
do you have /dev/video0 created? can you check the sensor capability with.. $ v4l2-ctl -d /dev/video0 --list-formats-ext?

6

2.the camera device registration complete.

and I can see /dev/video0~11 .

car@car:~$ v4l2-ctl -d /dev/video11 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture

    [0]: 'YUYV' (YUYV 4:2:2)
            Size: Discrete 1920x1200
                    Interval: Discrete 0.033s (30.000 fps)
    [1]: 'YUYV' (YUYV 4:2:2)
            Size: Discrete 1920x1200
                    Interval: Discrete 0.033s (30.000 fps)

also

car@car:~$ sudo media-ctl -p -d /dev/media0
[sudo] password for car: 
[  490.294775] ar0234 9-0036: ar0234_open:
[  490.294905] ar0234 9-0034: ar0234_open:
[  490.295086] ar0234 9-0032: ar0234_open:
Media controller API version 6.8.1[  490.295339] ar0234 9-0030: ar0234_open:
2

Media device information
------------------------
driver          tegra-camrtc-ca
model           NVIDIA Tegra Video Input Device
serial          
bus info        platform:tegra-capture-vi
hw revision     0x3
driver version  6.8.12

Device topology
- entity 1: 8181200000.host1x:nvcsi@8188000 (2 pads, 1 link, 0 routes)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
        pad0: SINK
        pad1: SOURCE
                -> "vi-output, 8181200000.host1x:nv":0 [ENABLED]

- entity 4: vi-output, 8181200000.host1x:nv (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video0
        pad0: SINK
                <- "8181200000.host1x:nvcsi@8188000":1 [ENABLED]

- entity 12: 8181200000.host1x:nvcsi@8188000 (2 pads, 1 link, 0 routes)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev1
        pad0: SINK
        pad1: SOURCE
                -> "vi-output, 8181200000.host1x:nv":0 [ENABLED]

- entity 15: vi-output, 8181200000.host1x:nv (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video1
        pad0: SINK
                <- "8181200000.host1x:nvcsi@8188000":1 [ENABLED]

- entity 23: 8181200000.host1x:nvcsi@8188000 (2 pads, 1 link, 0 routes)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev2
        pad0: SINK
        pad1: SOURCE
                -> "vi-output, 8181200000.host1x:nv":0 [ENABLED]

- entity 26: vi-output, 8181200000.host1x:nv (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video2
        pad0: SINK
                <- "8181200000.host1x:nvcsi@8188000":1 [ENABLED]

- entity 34: 8181200000.host1x:nvcsi@8188000 (2 pads, 1 link, 0 routes)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev3
        pad0: SINK
        pad1: SOURCE
                -> "vi-output, 8181200000.host1x:nv":0 [ENABLED]

- entity 37: vi-output, 8181200000.host1x:nv (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video3
        pad0: SINK
                <- "8181200000.host1x:nvcsi@8188000":1 [ENABLED]

- entity 45: 8181200000.host1x:nvcsi@8188000 (2 pads, 1 link, 0 routes)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev4
        pad0: SINK
        pad1: SOURCE
                -> "vi-output, 8181200000.host1x:nv":0 [ENABLED]

- entity 48: vi-output, 8181200000.host1x:nv (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video4
        pad0: SINK
                <- "8181200000.host1x:nvcsi@8188000":1 [ENABLED]

- entity 56: 8181200000.host1x:nvcsi@8188000 (2 pads, 1 link, 0 routes)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev5
        pad0: SINK
        pad1: SOURCE
                -> "vi-output, 8181200000.host1x:nv":0 [ENABLED]

- entity 59: vi-output, 8181200000.host1x:nv (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video5
        pad0: SINK
                <- "8181200000.host1x:nvcsi@8188000":1 [ENABLED]

- entity 67: 8181200000.host1x:nvcsi@8188000 (2 pads, 1 link, 0 routes)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev6
        pad0: SINK
        pad1: SOURCE
                -> "vi-output, 8181200000.host1x:nv":0 [ENABLED]

- entity 70: vi-output, 8181200000.host1x:nv (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video6
        pad0: SINK
                <- "8181200000.host1x:nvcsi@8188000":1 [ENABLED]

- entity 78: 8181200000.host1x:nvcsi@8188000 (2 pads, 1 link, 0 routes)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev7
        pad0: SINK
        pad1: SOURCE
                -> "vi-output, 8181200000.host1x:nv":0 [ENABLED]

- entity 81: vi-output, 8181200000.host1x:nv (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video7
        pad0: SINK
                <- "8181200000.host1x:nvcsi@8188000":1 [ENABLED]

- entity 89: 8181200000.host1x:nvcsi@8188000 (2 pads, 2 links, 0 routes)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev8
        pad0: SINK
                <- "ar0234 9-0030":0 [ENABLED]
        pad1: SOURCE
                -> "vi-output, ar0234 9-0030":0 [ENABLED]

- entity 92: 8181200000.host1x:nvcsi@8188000 (2 pads, 2 links, 0 routes)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev9
        pad0: SINK
                <- "ar0234 9-0032":0 [ENABLED]
        pad1: SOURCE
                -> "vi-output, ar0234 9-0032":0 [ENABLED]

- entity 95: 8181200000.host1x:nvcsi@8188000 (2 pads, 2 links, 0 routes)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev10
        pad0: SINK
                <- "ar0234 9-0034":0 [ENABLED]
        pad1: SOURCE
                -> "vi-output, ar0234 9-0034":0 [ENABLED]

- entity 98: 8181200000.host1x:nvcsi@8188000 (2 pads, 2 links, 0 routes)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev11
        pad0: SINK
                <- "ar0234 9-0036":0 [ENABLED]
        pad1: SOURCE
                -> "vi-output, ar0234 9-0036":0 [ENABLED]

- entity 101: ar0234 9-0036 (1 pad, 1 link, 0 routes)
              type V4L2 subdev subtype Sensor flags 0
              device node name /dev/v4l-subdev12
        pad0: SOURCE
                [stream:0 fmt:YUYV8_1X16/1920x1200@1/30 field:none colorspace:srgb]
                -> "8181200000.host1x:nvcsi@8188000":0 [ENABLED]

- entity 103: vi-output, ar0234 9-0036 (1 pad, 1 link)
              type Node subtype V4L flags 0
              device node name /dev/video8
        pad0: SINK
                <- "8181200000.host1x:nvcsi@8188000":1 [ENABLED]

- entity 121: ar0234 9-0034 (1 pad, 1 link, 0 routes)
              type V4L2 subdev subtype Sensor flags 0
              device node name /dev/v4l-subdev13
        pad0: SOURCE
                [stream:0 fmt:YUYV8_1X16/1920x1200@1/30 field:none colorspace:srgb]
                -> "8181200000.host1x:nvcsi@8188000":0 [ENABLED]

- entity 123: vi-output, ar0234 9-0034 (1 pad, 1 link)
              type Node subtype V4L flags 0
              device node name /dev/video9
        pad0: SINK
                <- "8181200000.host1x:nvcsi@8188000":1 [ENABLED]

- entity 133: ar0234 9-0032 (1 pad, 1 link, 0 routes)
              type V4L2 subdev subtype Sensor flags 0
              device node name /dev/v4l-subdev14
        pad0: SOURCE
                [stream:0 fmt:YUYV8_1X16/1920x1200@1/30 field:none colorspace:srgb]
                -> "8181200000.host1x:nvcsi@8188000":0 [ENABLED]

- entity 135: vi-output, ar0234 9-0032 (1 pad, 1 link)
              type Node subtype V4L flags 0
              device node name /dev/video10
        pad0: SINK
                <- "8181200000.host1x:nvcsi@8188000":1 [ENABLED]

- entity 145: ar0234 9-0030 (1 pad, 1 link, 0 routes)
              type V4L2 subdev subtype Sensor flags 0
              device node name /dev/v4l-subdev15
        pad0: SOURCE
                [stream:0 fmt:YUYV8_1X16/1920x1200@1/30 field:none colorspace:srgb]
                -> "8181200000.host1x:nvcsi@8188000":0 [ENABLED]

- entity 147: vi-output, ar0234 9-0030 (1 pad, 1 link)
              type Node subtype V4L flags 0
              device node name /dev/video11
        pad0: SINK
                <- "8181200000.host1x:nvcsi@8188000":1 [ENABLED]
7

Our HW topology is like this, now I just config cam8-11 to CSI port4-5 in device-tree.

018b95c5-0cec-44d5-8854-39773d0acc2f
018b95c5-0cec-44d5-8854-39773d0acc2f511×675 105 KB

8

hello ekeechg,

we’ve test camera stream with dual IMX274 camera module, which using the overlay.. /boot/tegra264-p3971-camera-dual-imx274-overlay.dtbo

[   11.954357] imx274 17-001a: tegracam sensor driver:imx274_v2.0.6
[   11.991803] tegra-camrtc-capture-vi tegra-capture-vi: subdev imx274 17-001a bound
[   12.013360] imx274 17-001a: Detected IMX274 sensor
[   12.052165] imx274 18-001a: tegracam sensor driver:imx274_v2.0.6
[   12.070170] tegra-camrtc-capture-vi tegra-capture-vi: subdev imx274 18-001a bound
[   12.092207] imx274 18-001a: Detected IMX274 sensor

please try with v4l2 IOCTL to test camera stream,
for instance,
v4l2-ctl -d /dev/video0 --set-fmt-video=width=3840,height=2160,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=100

9

OK.I use the cmd:”v4l2-ctl -d /dev/video11 --set-fmt-video=width=1920,height=1080,pixelformat=YUYV --set-ctrl bypass_mode=0 --stream-mmap --stream-count=100”

shell output like this:

car@car:~$ v4l2-ctl --verbose -d /dev/video11 --set-fmt-video=width=1920,height=1080,pixelformat=YUYV --set-ctrl bypass_mode=0 --stream-mmap --stream-count=100
VIDIOC_QUERYCAP: ok
VIDIOC_S_EXT_CTRLS: ok
VIDIOC_G_FMT: ok
VIDIOC_S_FMT: ok
Format Video Capture:
Width/Height : 1920/1200
Pixel Format : ‘YUYV’ (YUYV 4:2:2)
Field : None
Bytes per Line : 3840
Size Image : 4608000
Colorspace : sRGB
Transfer Function : Default (maps to sRGB)
YCbCr/HSV Encoding: Default (maps to ITU-R 601)
Quantization : Default (maps to Limited Range)
Flags :
VIDIOC_REQBUFS returned 0 (Success)
VIDIOC_CREATE_BUFS returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_G_FMT returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_STREAMON returned 0 (Success)
cap dqbuf: 0 seq: 1 bytesused: 4608000 ts: 0.000000 field: None (error, ts-monotonic, ts-src-eof)
cap dqbuf: 0 seq: 0 bytesused: 4608000 ts: 0.000000 field: None (error, ts-monotonic, ts-src-eof)
cap dqbuf: 1 seq: 2 bytesused: 4608000 ts: 0.000000 field: None (error, ts-monotonic, ts-src-eof)
cap dqbuf: 2 seq: 3 bytesused: 4608000 ts: 0.000000 field: None (error, ts-monotonic, ts-src-eof)
cap dqbuf: 3 seq: 4 bytesused: 4608000 ts: 0.000000 field: None (error, ts-monotonic, ts-src-eof)
cap dqbuf: 0 seq: 1 bytesused: 4608000 ts: 0.000000 field: None (error, ts-monotonic, ts-src-eof)
cap dqbuf: 0 seq: 0 bytesused: 4608000 ts: 0.000000 field: None (error, ts-monotonic, ts-src-eof)

kernel print log as this attachment:

log_k.txt (9.2 KB)

attachment is rtcpu log and related device tree.

log3.txt (27.1 KB)

tegra264-camera-p3762-a00.dtsi.txt (48.5 KB)

tegra264-p3971-camera-p3762-a00-overlay.dts.txt (937 Bytes)

tegra264-p3971-camera-p3762-a00.dtsi.txt (10.1 KB)

10

hello ekeechg,

please try below commands to boost the clocks for testing.

sudo su
echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/emc/mrq_rate_locked
cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate |tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate
cat /sys/kernel/debug/bpmp/debug/clk/emc/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/emc/rate
11

Hello, Jerry:
following attachment is the log:

Screenshot from 2025-02-23 02-57-10
Screenshot from 2025-02-23 02-57-101139×295 51.7 KB

rtcpu.txt (33.4 KB)
dmesg.txt (13.6 KB)

12

hello ekeechg,

let’s try to configure higher log level to obtain more logs from rtcpu to debug further.

modprobe rtcpu-debug
echo 1 > /sys/kernel/debug/tracing/tracing_on
echo 30720 > /sys/kernel/debug/tracing/buffer_size_kb
echo 1 > /sys/kernel/debug/tracing/events/tegra_rtcpu/enable
echo 1 > /sys/kernel/debug/tracing/events/freertos/enable
echo 3 > /sys/kernel/debug/camrtc/log-level
echo > /sys/kernel/debug/tracing/trace
cat /sys/kernel/debug/tracing/trace

let me give you an example,
it shall looks like below for camera stream, it must be one pair of CHANSEL_PXL_SOF/CHANSEL_PXL_EOF to indicate a frame has detected by VI engine.
afterwards, it’s ATOMP_FRAME_DONE to indicate it’s complete writing a frame to memory.

     kworker/4:2-18405   [004] ..... 424921.928552: rtcpu_vinotify_event: tstamp:424949640648227 cch:0 vi:0 tag:CHANSEL_PXL_SOF channel:0x23 frame:0 vi_tstamp:424949640366436 data:0x0000000000000001
     kworker/4:2-18405   [004] ..... 424921.984534: rtcpu_vinotify_event: tstamp:424949655491301 cch:0 vi:0 tag:CHANSEL_PXL_EOF channel:0x23 frame:0 vi_tstamp:424949655327769 data:0x00000000086f0002
     kworker/4:2-18405   [004] ..... 424921.984534: rtcpu_vinotify_event: tstamp:424949655499097 cch:0 vi:0 tag:ATOMP_FRAME_DONE channel:0x23 frame:0 vi_tstamp:424949655328057 data:0x0000000000000000
13

Here is the log. it seem no VI engine log you mentioned.
Can we confirm data has been received in CSI? I have checked the 96712 register, there are MIPI data send to Thor.
rtcpu.txt (44.8 KB)

14

hello ekeechg,

there’s even no frame start from VI notify, it’s likely that VI engine did not received a frame packet.

please see-also Thor’s p3971 base camera device tree. $public_sources/kernel_src/hardware/nvidia/t264/nv-public/overlay/tegra264-p3971-camera-modules.dtsi,
it has ar0234 camera device tree ("tegra264-camera-ar0234-a00.dtsi") included by default, its using CSI-A and CSI-B.
please check whether there’re conflicts with yours, or, you may exclude that for verification.

besides.. please double check the board schematic for the i2c bus, for instance,i2c9: i2c@810c6a0000.
and.. here’s mismatch between your device tree and HW topology,
for instance, it’s not using i2c mux.

i2c@810c6a0000 {
	ar0234_i@30 {
		compatible = "nvidia,ar0234_hawk_owl";
		mode0 {/*mode IMX424_MODE_3840X1080_CROP_30FPS*/
				mclk_khz = "24000";
				num_lanes = "2";
				tegra_sinterface = "serial_e";
15

Yeah, as you said, there is a conflict in device tree.
I write the same “channel” in here.

c44f1492-347a-4e17-9f54-e618ad40930e
c44f1492-347a-4e17-9f54-e618ad40930e630×367 121 KB

Now, The ATOMP_FRAME_DONE is comming, log as follow.
rtcpu.txt (619.6 KB)

but kernel report some error when I grab image.
dmesg.txt (22.2 KB)

16

hello ekeechg,

could you please give it a try to modify the VI driver,
let’s ignore the frame_err for sensor bring-up.
for instance,

diff --git a/drivers/media/platform/tegra/camera/vi/vi5_fops.c b/drivers/media/platform/tegra/camera/vi/vi5_fops.c

@@ -581,7 +581,7 @@ static void vi5_capture_dequeue(struct tegra_channel *chan,
                                        "err_data %d\n",
                                        descr->status.frame_id, descr->status.flags,
                                        descr->status.err_data);
-                               frame_err = true;
+                               // frame_err = true;
17

Hi,Jerry:
It seems no error in v4l2-ctl, but still error report in tegra-capture-vi:

Screenshot from 2025-02-23 07-19-22
Screenshot from 2025-02-23 07-19-221596×709 252 KB

Screenshot from 2025-02-23 07-18-33
Screenshot from 2025-02-23 07-18-331596×407 58.4 KB

image
image1920×1500 90.3 KB

18

hello ekeechg,

may I know what’s your pipeline to enable preview stream?
you may refer to $ gst-inspect-1.0 v4l2src, you should use the YUY2 formats for your YUYV pixel format.
BTW, your capture results looks like an interlaced stream, please try to configure as progressive video for confirmation.

19

Hi,Jerry:
Do you mean the preview initialize process?
I config the serializer/deserializer by script, there should be image stream on MIPI, and then use v4l2-ctl grab image.
The interleave happened when I start two camera streams. The image is normal if I only start one stream.
I’m not sure the interleave happen in deserializer or Thor.
Following pic. is captured when I start one stream.

image1
image11920×1500 58.1 KB

20

hello ekeechg,

I see, just wondering why the snapshot image size is 1920x1500.
since your sensor capability is 1920x1200. I doubt it’s the redundant 300-lines in the bottom.

21

Hi,Jerry:
Sure, the 300-lines in the bottom is redundant.
Do you have any idea about the interleave issue? can it happen in Thor?
or Could you pls. provide any suggestion to locate the problem?

BR//