Jetson Orin NX + Sony IMX900 Direct CSI-2: System Hangs on V4L2 STREAMON / Frame Capture

Hello NVIDIA team,

I am currently bringing up a Sony IMX900 camera sensor on a Jetson Orin NX SOM and I am facing a system hang as soon as I attempt actual V4L2 streaming or frame capture.

I would appreciate guidance on debugging the CSI/NVCSI/VI path and identifying why the system becomes temporarily inaccessible during STREAMON.

Hardware

  • NVIDIA Jetson Orin NX SOM
  • Sony IMX900 sensor
  • Direct CSI-2 connection from IMX900 to Orin NX
  • 2 CSI-2 lanes
  • RAW10
  • 1024x720
  • 30 FPS
  • Sensor CSI data rate: 594 Mbps/lane

Software

  • Jetson Orin NX
  • Linux kernel: 5.15.148-tegra
  • Jetpack version: 36.4.4

Current bring-up status

The following stages are working successfully:

  1. IMX900 I2C communication
  2. Sensor reset GPIO
  3. Sensor driver probe
  4. Sensor detection
  5. V4L2 sensor subdevice creation
  6. NVCSI media link
  7. VI media link
  8. /dev/video0 creation

The driver probe log is:

fr_imx900: module verification failed: signature and/or required key missing - tainting kernel
imx900 9-001a: probing v4l2 sensor
imx900 9-001a: imx900_parse_dt: ADI GUI POC Device Tree Parser active
imx900 9-001a: IMX900 reset GPIO parsed: 397
imx900 9-001a: IMX900 reset GPIO initialized LOW: 397
imx900 9-001a: tegracam sensor driver:imx900_v2.0.6
imx900 9-001a: imx900_power_on: power on sequence starting
imx900 9-001a: IMX900 reset LOW gpio=397
imx900 9-001a: IMX900 reset HIGH gpio=397
imx900 9-001a: imx900_board_setup: Board setup successful
imx900 9-001a: imx900_power_off: power off sequence starting
imx900 9-001a: IMX900 reset LOW gpio=397
imx900 9-001a: imx900_board_setup: sensor left in RESET LOW state
tegra-camrtc-capture-vi tegra-capture-vi: subdev imx900 9-001a bound
imx900 9-001a: Detected imx900 sensor

The module is loaded:

fr_imx900 24576 0
tegra_camera 217088 4 nvhost_isp5,nvhost_nvcsi_t194,fr_imx900,nvhost_vi5

Media topology

media-ctl -p reports:

Media controller API version 5.15.148

Media device information
------------------------
driver          tegra-camrtc-ca
model           NVIDIA Tegra Video Input Device
driver version  5.15.148

Device topology
- entity 1: imx900 9-001a (1 pad, 1 link)
            type V4L2 subdev subtype Sensor
            device node name /dev/v4l-subdev0
    pad0: Source
        [fmt:SRGGB10_1X10/1024x720@1/30 field:none colorspace:srgb]
        -> "13e00000.host1x:nvcsi@15a00000-":0 [ENABLED]

- entity 3: 13e00000.host1x:nvcsi@15a00000- (2 pads, 2 links)
            type V4L2 subdev subtype Unknown
            device node name /dev/v4l-subdev1
    pad0: Sink
        <- "imx900 9-001a":0 [ENABLED]
    pad1: Source
        -> "vi-output, imx900 9-001a":1 [ENABLED]

- entity 6: vi-output, imx900 9-001a (1 pad, 1 link)
            type V4L flags
            device node name /dev/video0
    pad0: Sink
        <- "13e00000.host1x:nvcsi@15a00000-":1 [ENABLED]

So the media graph is:

IMX900
   |
   v
NVCSI
   |
   v
VI
   |
   v
/dev/video0

All media links are enabled.

V4L2 format

Running:

v4l2-ctl --list-formats-ext -d /dev/video0

returns:

[0]: 'RG10' (10-bit Bayer RGRG/GBGB)
    Size: Discrete 1024x720
        Interval: Discrete 0.033s (30.000 fps)

The format negotiation itself works.

Actual capture problem

I am unable to perform actual frame capture.

I run:

v4l2-ctl \
    -d /dev/video0 \
    --set-fmt-video=width=1024,height=720,pixelformat=RG10 \
    --set-parm=30 \
    --stream-mmap \
    --stream-count=1 \
    --stream-to=/tmp/imx900.raw \
    --verbose

The output is:

VIDIOC_QUERYCAP: ok
VIDIOC_G_FMT: ok
VIDIOC_S_FMT: ok
Format Video Capture:
    Width/Height      : 1024/720
    Pixel Format      : 'RG10' (10-bit Bayer RGRG/GBGB)
    Field             : None
    Bytes per Line    : 2048
    Size Image        : 1474560
    Colorspace        : sRGB

VIDIOC_S_PARM: ok
Frame rate set to 30.000 fps

VIDIOC_REQBUFS returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)

VIDIOC_QBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)

HANGS

The important observation is that the system becomes temporarily inaccessible immediately after starting the capture operation.

After approximately 10–20 seconds, access to the SOM returns.

The same behavior occurs when attempting V4L2 streaming and when attempting V4L2 operations that require the camera pipeline to become active.

Because the SOM temporarily becomes inaccessible, I am currently unable to reliably collect the kernel messages generated at the exact moment of the failure.

Questions

  1. What is the recommended way to debug a Jetson Orin NX system that becomes temporarily inaccessible during V4L2 STREAMON?
  2. Is there a recommended NVIDIA debug configuration, tracepoint, dynamic-debug setting, or debugfs interface for diagnosing NVCSI/VI failures?
  3. How can I determine whether the hang occurs in:
    • VIDIOC_STREAMON
    • the IMX900 sensor driver’s start_streaming()
    • NVCSI
    • VI
    • camera RTCPU/RCE
    • waiting for the first frame?
  4. Are there specific NVCSI/VI error counters or debug registers that I should inspect for a direct 2-lane CSI-2 RAW10 sensor?
  5. Is there a recommended way to capture kernel logs when the Jetson temporarily loses SSH/console access during the camera operation?
  6. Could the temporary system hang indicate an NVCSI/VI timeout caused by the sensor not transmitting valid CSI-2 packets or by an incorrect CSI timing/configuration?
  7. Are there any NVIDIA-specific commands you recommend running before/after STREAMON to diagnose this?

Any guidance on the correct debugging sequence would be greatly appreciated.

At this point my primary goal is simply to capture one RAW10 frame successfully from /dev/video0.

Thank you.

Hi,

If the board becomes unresponsive during capture, I recommend accessing the Jetson through the serial debug console using the external UART header pins and a USB-to-UART adapter:

NVIDIA Jetson Linux Developer Guide – UART Access

This will allow you to monitor the kernel messages even if the system becomes temporarily inaccessible through SSH. It may help identify whether the issue is related to the camera driver or to the CSI/VI capture pipeline, in tat case I also recommend enabling VI/RTCPU tracing before reproducing the issue:

NVIDIA Jetson Linux Developer Guide – Sensor Software Troubleshooting

After reproducing the problem, please share the kernel log and trace output. The trace should help determine whether the Jetson is receiving data from the sensor and whether there are any NVCSI/VI errors.

If you have doubts about the IMX900 driver configuration, there is also an open-source IMX900 driver that you can use as a reference:

It may be useful for comparing the sensor mode configuration, CSI parameters, and streaming sequence with your implementation.

Regards,

Manuel Leiva
Embedded SW Engineer at RidgeRun
Contact us: support@ridgerun.com
Developers wiki: https://developer.ridgerun.com
Website: www.ridgerun.com

Get the trace log.

sudo su
modprobe rtcpu_debug

echo 1 > /sys/kernel/debug/tracing/tracing_on
echo 30720 > /sys/kernel/debug/tracing/buffer_size_kb
echo 1 > /sys/kernel/debug/tracing/events/tegra_rtcpu/enable
echo 1 > /sys/kernel/debug/tracing/events/freertos/enable
echo 3 > /sys/kernel/debug/camrtc/log-level
echo 1 > /sys/kernel/debug/tracing/events/camera_common/enable
echo > /sys/kernel/debug/tracing/trace

v4l2-ctl --stream-mmap -c bypass_mode=0

cat /sys/kernel/debug/tracing/trace

Hang issue gone now I am struggling with fps read issue. Currently driver gets probed and video node created and also media-topology looks correct.

I am unable to get fps read using this cmd:

sudo i2cdetect -r -y 9
[sudo] password for orin:
0 1 2 3 4 5 6 7 8 9 a b c d e f
00: – – – – – – – –
10: – – – – – – – – – – UU – – – – –
20: – – – – – – – – – – – – – – – –
30: – – – – – – 36 – – – – – – – – –
40: – – – – – – – – – – – – – – – –
50: – – – – – – 56 – – – – – – – – –
60: – – – – – – – – – – – – – – – –
70: – – – – – – – –

ls /dev/video*
/dev/video0

v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture

[0]: 'RGGB' (8-bit Bayer RGRG/GBGB)
	Size: Discrete 2064x1552
		Interval: Discrete 0.008s (125.000 fps)
	Size: Discrete 1920x1080
		Interval: Discrete 0.006s (172.000 fps)
	Size: Discrete 1032x776
		Interval: Discrete 0.004s (230.000 fps)
	Size: Discrete 2064x154
		Interval: Discrete 0.002s (656.000 fps)
	Size: Discrete 1024x720
		Interval: Discrete 0.003s (397.000 fps)
[1]: 'GBRG' (8-bit Bayer GBGB/RGRG)
	Size: Discrete 2064x1552
		Interval: Discrete 0.008s (125.000 fps)
	Size: Discrete 1920x1080
		Interval: Discrete 0.006s (172.000 fps)
	Size: Discrete 1032x776
		Interval: Discrete 0.004s (230.000 fps)
	Size: Discrete 2064x154
		Interval: Discrete 0.002s (656.000 fps)
	Size: Discrete 1024x720
		Interval: Discrete 0.003s (397.000 fps)
[2]: 'RG10' (10-bit Bayer RGRG/GBGB)
	Size: Discrete 2064x1552
		Interval: Discrete 0.008s (125.000 fps)
	Size: Discrete 1920x1080
		Interval: Discrete 0.006s (172.000 fps)
	Size: Discrete 1032x776
		Interval: Discrete 0.004s (230.000 fps)
	Size: Discrete 2064x154
		Interval: Discrete 0.002s (656.000 fps)
	Size: Discrete 1024x720
		Interval: Discrete 0.003s (397.000 fps)
[3]: 'GB10' (10-bit Bayer GBGB/RGRG)
	Size: Discrete 2064x1552
		Interval: Discrete 0.008s (125.000 fps)
	Size: Discrete 1920x1080
		Interval: Discrete 0.006s (172.000 fps)
	Size: Discrete 1032x776
		Interval: Discrete 0.004s (230.000 fps)
	Size: Discrete 2064x154
		Interval: Discrete 0.002s (656.000 fps)
	Size: Discrete 1024x720
		Interval: Discrete 0.003s (397.000 fps)
[4]: 'RG12' (12-bit Bayer RGRG/GBGB)
	Size: Discrete 2064x1552
		Interval: Discrete 0.008s (125.000 fps)
	Size: Discrete 1920x1080
		Interval: Discrete 0.006s (172.000 fps)
	Size: Discrete 1032x776
		Interval: Discrete 0.004s (230.000 fps)
	Size: Discrete 2064x154
		Interval: Discrete 0.002s (656.000 fps)
	Size: Discrete 1024x720
		Interval: Discrete 0.003s (397.000 fps)
[5]: 'GB12' (12-bit Bayer GBGB/RGRG)
	Size: Discrete 2064x1552
		Interval: Discrete 0.008s (125.000 fps)
	Size: Discrete 1920x1080
		Interval: Discrete 0.006s (172.000 fps)
	Size: Discrete 1032x776
		Interval: Discrete 0.004s (230.000 fps)
	Size: Discrete 2064x154
		Interval: Discrete 0.002s (656.000 fps)
	Size: Discrete 1024x720
		Interval: Discrete 0.003s (397.000 fps)

This cmd doesnt give any output:
v4l2-ctl --stream-mmap -d /dev/video0

dmesg:

324.575115] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[ 324.575140] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[ 324.576426] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel
[ 327.134856] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[ 327.134880] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[ 327.136123] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel
[ 329.695235] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[ 329.695263] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel

trace log:

cat /sys/kernel/debug/tracing/trace

tracer: nop

entries-in-buffer/entries-written: 44/44 #P:4

_-------=> irqs-off

/ _------=> need-resched

| / _-----=> need-resched-lazy

|| / _----=> hardirq/softirq

||| / _β€”=> preempt-depth

|||| / _–=> preempt-lazy-depth

||||| / _-=> migrate-disable

|||||| / delay

TASK-PID CPU# ||||||| TIMESTAMP FUNCTION

| | | ||||||| | |

vi-output, imx9-1831 [001] … 342.240830: tegra_channel_capture_setup: vnc_id 0 W 2064 H 1552 fmt c4
vi-output, imx9-1830 [002] … 342.240929: vi_task_submit: class_id:48 ch:0 syncpt_id:35 syncpt_thresh:0 pid:1830 tid:1830
vi-output, imx9-1830 [002] … 342.240939: vi_task_submit: class_id:48 ch:0 syncpt_id:35 syncpt_thresh:0 pid:1830 tid:1830
vi-output, imx9-1830 [002] … 342.240940: vi_task_submit: class_id:48 ch:0 syncpt_id:35 syncpt_thresh:0 pid:1830 tid:1830
vi-output, imx9-1830 [002] … 342.240942: vi_task_submit: class_id:48 ch:0 syncpt_id:35 syncpt_thresh:0 pid:1830 tid:1830
vi-output, imx9-1831 [001] … 344.800982: tegra_channel_capture_setup: vnc_id 0 W 2064 H 1552 fmt c4
vi-output, imx9-1830 [002] … 344.801225: vi_task_submit: class_id:48 ch:0 syncpt_id:35 syncpt_thresh:0 pid:1830 tid:1830
vi-output, imx9-1830 [002] … 344.801235: vi_task_submit: class_id:48 ch:0 syncpt_id:35 syncpt_thresh:0 pid:1830 tid:1830
vi-output, imx9-1830 [002] … 344.801237: vi_task_submit: class_id:48 ch:0 syncpt_id:35 syncpt_thresh:0 pid:1830 tid:1830
vi-output, imx9-1830 [002] … 344.801238: vi_task_submit: class_id:48 ch:0 syncpt_id:35 syncpt_thresh:0 pid:1830 tid:1830
v4l2-ctl-1829 [003] … 346.865554: tegra_channel_close: vi-output, imx900 9-001a
vi-output, imx9-1831 [001] … 347.328709: tegra_channel_capture_setup: vnc_id 0 W 2064 H 1552 fmt c4
v4l2-ctl-1829 [003] … 347.328820: tegra_channel_set_stream: enable : 0x0
v4l2-ctl-1829 [003] … 347.328823: tegra_channel_set_stream: imx900 9-001a : 0x0
v4l2-ctl-1829 [003] … 348.048021: tegra_channel_set_stream: 13e00000.host1x:nvcsi@15a00000- : 0x0
v4l2-ctl-1829 [003] … 348.048026: csi_s_stream: enable : 0x0
v4l2-ctl-1829 [003] … 348.051034: tegra_channel_set_power: imx900 9-001a : 0x0
v4l2-ctl-1829 [003] … 348.051059: camera_common_s_power: status : 0x0
v4l2-ctl-1829 [003] … 348.051412: tegra_channel_set_power: 13e00000.host1x:nvcsi@15a00000- : 0x0
v4l2-ctl-1829 [003] … 348.051415: csi_s_power: enable : 0x0
v4l2-ctl-1851 [000] … 352.282456: tegra_channel_open: vi-output, imx900 9-001a
v4l2-ctl-1851 [000] … 352.291884: tegra_channel_set_power: imx900 9-001a : 0x1
v4l2-ctl-1851 [000] … 352.291910: camera_common_s_power: status : 0x1
v4l2-ctl-1851 [003] … 352.324063: tegra_channel_set_power: 13e00000.host1x:nvcsi@15a00000- : 0x1
v4l2-ctl-1851 [003] … 352.324067: csi_s_power: enable : 0x1
v4l2-ctl-1851 [003] … 352.325041: tegra_channel_capture_setup: vnc_id 0 W 2064 H 1552 fmt c4
vi-output, imx9-1852 [001] … 352.325532: vi_task_submit: class_id:48 ch:0 syncpt_id:35 syncpt_thresh:0 pid:1852 tid:1852
vi-output, imx9-1852 [001] … 352.325543: vi_task_submit: class_id:48 ch:0 syncpt_id:35 syncpt_thresh:0 pid:1852 tid:1852
vi-output, imx9-1852 [001] … 352.325546: vi_task_submit: class_id:48 ch:0 syncpt_id:35 syncpt_thresh:0 pid:1852 tid:1852
vi-output, imx9-1852 [001] … 352.325548: vi_task_submit: class_id:48 ch:0 syncpt_id:35 syncpt_thresh:0 pid:1852 tid:1852
v4l2-ctl-1851 [003] … 352.325841: tegra_channel_set_stream: enable : 0x1
v4l2-ctl-1851 [003] … 352.326470: tegra_channel_set_stream: 13e00000.host1x:nvcsi@15a00000- : 0x1
v4l2-ctl-1851 [003] … 352.326473: csi_s_stream: enable : 0x1
v4l2-ctl-1851 [003] … 352.326582: tegra_channel_set_stream: imx900 9-001a : 0x1
vi-output, imx9-1853 [002] … 355.040619: tegra_channel_capture_setup: vnc_id 0 W 2064 H 1552 fmt c4
vi-output, imx9-1852 [001] … 355.040735: vi_task_submit: class_id:48 ch:0 syncpt_id:35 syncpt_thresh:0 pid:1852 tid:1852
vi-output, imx9-1852 [001] … 355.040748: vi_task_submit: class_id:48 ch:0 syncpt_id:35 syncpt_thresh:0 pid:1852 tid:1852
vi-output, imx9-1852 [001] … 355.040751: vi_task_submit: class_id:48 ch:0 syncpt_id:35 syncpt_thresh:0 pid:1852 tid:1852
vi-output, imx9-1852 [001] … 355.040753: vi_task_submit: class_id:48 ch:0 syncpt_id:35 syncpt_thresh:0 pid:1852 tid:1852
vi-output, imx9-1853 [002] … 357.600990: tegra_channel_capture_setup: vnc_id 0 W 2064 H 1552 fmt c4
vi-output, imx9-1852 [001] … 357.601236: vi_task_submit: class_id:48 ch:0 syncpt_id:35 syncpt_thresh:0 pid:1852 tid:1852
vi-output, imx9-1852 [001] … 357.601248: vi_task_submit: class_id:48 ch:0 syncpt_id:35 syncpt_thresh:0 pid:1852 tid:1852
vi-output, imx9-1852 [001] … 357.601250: vi_task_submit: class_id:48 ch:0 syncpt_id:35 syncpt_thresh:0 pid:1852 tid:1852
vi-output, imx9-1852 [001] … 357.601251: vi_task_submit: class_id:48 ch:0 syncpt_id:35 syncpt_thresh:0 pid:1852 tid:1852

Have a look and help me with the root cause and solution to this.

Thanks in advance.

The trace log shows doesn’t receive any validate data from the sensor.

Maybe probe the sensor output to make sure follow MIPI spec.

Thanks

Could you please guide me where I look forward into to debug this issue.

dmesg:

[ 486.110261] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[ 486.110283] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[ 486.110947] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel

v4l2-ctl --stream-mmap -d /dev/video0 this prints nothing.

I am using Framos FSM IMX900 Camera Sensor.

Here is the github link where imx900 driver are available which I used.

Here in this driver 37MHz clk is in used and the same I used.

Have a look into my device tree may be you find any fault in my device tree

IMX900.txt (21.6 KB)

Please review my IMX900 device tree file, I would be a great help for me to move further.

Thanks in advance.

@ShaneCCC @ManuelLeiva @proventusnova

I have solve the above issue, currenlty I am looking to get stream from imx900 camera sensor.

Here are the possible formats,datarate I have, but somehow I am unable to get viewable video feed.

v4l2-ctl -d /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture

[0]: 'RGGB' (8-bit Bayer RGRG/GBGB)
	Size: Discrete 2064x1552
		Interval: Discrete 0.008s (125.000 fps)
	Size: Discrete 1920x1080
		Interval: Discrete 0.006s (172.000 fps)
	Size: Discrete 1032x776
		Interval: Discrete 0.004s (230.000 fps)
	Size: Discrete 2064x154
		Interval: Discrete 0.002s (656.000 fps)
	Size: Discrete 1024x720
		Interval: Discrete 0.003s (397.000 fps)
[1]: 'GBRG' (8-bit Bayer GBGB/RGRG)
	Size: Discrete 2064x1552
		Interval: Discrete 0.008s (125.000 fps)
	Size: Discrete 1920x1080
		Interval: Discrete 0.006s (172.000 fps)
	Size: Discrete 1032x776
		Interval: Discrete 0.004s (230.000 fps)
	Size: Discrete 2064x154
		Interval: Discrete 0.002s (656.000 fps)
	Size: Discrete 1024x720
		Interval: Discrete 0.003s (397.000 fps)
[2]: 'RG10' (10-bit Bayer RGRG/GBGB)
	Size: Discrete 2064x1552
		Interval: Discrete 0.008s (125.000 fps)
	Size: Discrete 1920x1080
		Interval: Discrete 0.006s (172.000 fps)
	Size: Discrete 1032x776
		Interval: Discrete 0.004s (230.000 fps)
	Size: Discrete 2064x154
		Interval: Discrete 0.002s (656.000 fps)
	Size: Discrete 1024x720
		Interval: Discrete 0.003s (397.000 fps)
[3]: 'GB10' (10-bit Bayer GBGB/RGRG)
	Size: Discrete 2064x1552
		Interval: Discrete 0.008s (125.000 fps)
	Size: Discrete 1920x1080
		Interval: Discrete 0.006s (172.000 fps)
	Size: Discrete 1032x776
		Interval: Discrete 0.004s (230.000 fps)
	Size: Discrete 2064x154
		Interval: Discrete 0.002s (656.000 fps)
	Size: Discrete 1024x720
		Interval: Discrete 0.003s (397.000 fps)
[4]: 'RG12' (12-bit Bayer RGRG/GBGB)
	Size: Discrete 2064x1552
		Interval: Discrete 0.008s (125.000 fps)
	Size: Discrete 1920x1080
		Interval: Discrete 0.006s (172.000 fps)
	Size: Discrete 1032x776
		Interval: Discrete 0.004s (230.000 fps)
	Size: Discrete 2064x154
		Interval: Discrete 0.002s (656.000 fps)
	Size: Discrete 1024x720
		Interval: Discrete 0.003s (397.000 fps)
[5]: 'GB12' (12-bit Bayer GBGB/RGRG)
	Size: Discrete 2064x1552
		Interval: Discrete 0.008s (125.000 fps)
	Size: Discrete 1920x1080
		Interval: Discrete 0.006s (172.000 fps)
	Size: Discrete 1032x776
		Interval: Discrete 0.004s (230.000 fps)
	Size: Discrete 2064x154
		Interval: Discrete 0.002s (656.000 fps)
	Size: Discrete 1024x720
		Interval: Discrete 0.003s (397.000 fps)

v4l2-ctl -d /dev/video0 --list-ctrls

Camera Controls

                 group_hold 0x009a2003 (bool)   : default=0 value=0 flags=execute-on-write
                sensor_mode 0x009a2008 (int64)  : min=0 max=15 step=1 default=0 value=0 flags=slider
                       gain 0x009a2009 (int64)  : min=0 max=480 step=1 default=0 value=0 flags=slider
                   exposure 0x009a200a (int64)  : min=5 max=660000 step=1 default=10000 value=9 flags=update, slider
                 frame_rate 0x009a200b (int64)  : min=1500000 max=229554498 step=1 default=229554498 value=1500000 flags=update, slider
       sensor_configuration 0x009a2032 (u32)    : min=0 max=4294967295 step=1 default=0 dims=[22] flags=read-only, volatile, has-payload
     sensor_mode_i2c_packet 0x009a2033 (u32)    : min=0 max=4294967295 step=1 default=0 dims=[1026] flags=read-only, volatile, has-payload
  sensor_control_i2c_packet 0x009a2034 (u32)    : min=0 max=4294967295 step=1 default=0 dims=[1026] flags=read-only, volatile, has-payload
                bypass_mode 0x009a2064 (intmenu): min=0 max=1 default=0 value=0 (0 0x0)
            override_enable 0x009a2065 (intmenu): min=0 max=1 default=0 value=0 (0 0x0)
               height_align 0x009a2066 (int)    : min=1 max=16 step=1 default=1 value=1
                 size_align 0x009a2067 (intmenu): min=0 max=2 default=0 value=0 (1 0x1)
           write_isp_format 0x009a2068 (int)    : min=1 max=1 step=1 default=1 value=1
   sensor_signal_properties 0x009a2069 (u32)    : min=0 max=4294967295 step=1 default=0 dims=[30][18] flags=read-only, has-payload
    sensor_image_properties 0x009a206a (u32)    : min=0 max=4294967295 step=1 default=0 dims=[30][16] flags=read-only, has-payload
  sensor_control_properties 0x009a206b (u32)    : min=0 max=4294967295 step=1 default=0 dims=[30][36] flags=read-only, has-payload
          sensor_dv_timings 0x009a206c (u32)    : min=0 max=4294967295 step=1 default=0 dims=[30][16] flags=read-only, has-payload
           low_latency_mode 0x009a206d (bool)   : default=0 value=0
           preferred_stride 0x009a206e (int)    : min=0 max=65535 step=1 default=0 value=0
override_capture_timeout_ms 0x009a206f (int)    : min=-1 max=2147483647 step=1 default=2500 value=2500
                  data_rate 0x009a2078 (menu)   : min=0 max=4 default=0 value=0 (2376 Mbps/lane)
               test_pattern 0x009a2079 (menu)   : min=0 max=5 default=0 value=0 (No pattern)
             operation_mode 0x009a207a (menu)   : min=0 max=1 default=0 value=0 (Master Mode)
                black_level 0x009a207d (int64)  : min=0 max=255 step=1 default=15 value=15 flags=slider, execute-on-write
               sensor_modes 0x009a2082 (int)    : min=0 max=30 step=1 default=30 value=15 flags=read-only
        global_shutter_mode 0x009a2083 (menu)   : min=0 max=2 default=0 value=0 (Normal Mode)
            conversion_gain 0x009a2085 (bool)   : default=0 value=0 flags=update

Here is the attached image of the video feed, I am getting from the camera senosr:

I would be a great help for me, if you guide how to solve this issue.

It’s Bayer raw sensor?

Why the driver report 5 pixel format?

What’s the APP to show the sensor frame?

@ShaneCCC

Yes, the IMX900 is a Bayer RAW sensor.

I am currently testing it through the direct CSI β†’ VI β†’ V4L2 path without using the ISP. The sensor is streaming successfully and I can capture RAW data from /dev/video0.

Regarding the 5 pixel formats reported by the driver, I am checking the output of v4l2-ctl --list-formats-ext to understand the exact Bayer format/bit-depth representations exposed by the driver.

Could you please suggest the recommended application/tool or pipeline ?

I also provide the complete v4l2-ctl --list-formats-ext output.

What’s the resolution?

Do you use tools to check the raw file? Do you adjust the alignment?

The resolution I am testing currently is 2064 Γ— 1552, at the IMX900 125 FPS mode.

I captured the RAW frame using v4l2-ctl:

v4l2-ctl -d /dev/video0 --set-fmt-video=width=2064,height=1552,pixelformat=RGGB --stream-mmap --stream-count=1 --stream-to=imx900.raw

The captured file size is 3,575,808 bytes, and V4L2 reports 2304 bytes per line.

I have checked the RAW file using od and compared captures under different illumination conditions. The RAW data changes when the lighting changes, so the sensor data appears to be reaching V4L2.

I have not yet performed any manual alignment/unpacking of the RAW data. I tried passing the V4L2 Bayer output directly to GStreamer bayer2rgb and then encoding to JPEG, but the resulting image contains scattered RGB pixels rather than a recognizable image.

Could you please advise what alignment/unpacking is required for this RAW format before debayering, and which tool you recommend for viewing the captured IMX900 RAW file?

I am using Framos Github repo.

Using the below cmd, camera streams but the output is coloured, I want to get the monochrome output.

Could you please provide me gtsreamer pipeline, without isp.

gst-launch-1.0 nvarguscamerasrc sensor-id=0 sensor-mode=0 ! β€œvideo/x-raw(memory:NVMM), width=2064, height=1552, framerate=30/1” ! nvvidconv ! β€œvideo/x-raw(memory:NVMM), format=I420” ! nvv4l2h264enc bitrate=8000000 ! h264parse ! rtph264pay config-interval=1 pt=96 ! udpsink host=192.168.55.100 port=5000 sync=false

You can use v4l2src with bayer2rgb but need to handle the alignment or adjust the out width alignment to 256 like 2304x1552

Could you please share the sample gstreamer pipeline I can use to get monochrome image.

Also to confirm, using this GitHub - framosimaging/framos-jetson-libsv Β· GitHub I able to capture image.

You can use videoconvert convert to GRAY8 after the bayer2rgb.

Thanks

Could you please confirm that do I need to make any changes to support gray formats for monochrome output in OrinNX Source or It is by default supported by orin nx ?

If changes, required where do I need to make changes in the public source ?

I am using L4T version 36.4.4

For the driver you don’t need.

Suppose you need format convert for it.

Thanks

Hi Shane,

Thanks for the clarification.

My requirement is to obtain the sensor output as 8-bit grayscale (Y8/GRAY) for my application.

Could you please clarify the recommended NVIDIA pipeline for this?

  1. Should I keep the IMX900 driver and Device Tree configured as RAW10/RAW12 and perform RAW β†’ GRAY8 conversion in GStreamer/userspace?

  2. If yes, what GStreamer pipeline/plugin should I use on Jetson to convert the IMX900 monochrome RAW10/RAW12 data to GRAY8?

  3. Is there any NVIDIA-supported way to make /dev/video0 directly expose V4L2_PIX_FMT_GREY / Y8 for this monochrome sensor without modifying camera_common.c, sensor_common.c, or vi5_formats.h?

  4. If direct Y8 output is possible, could you point me to the relevant public source/configuration in L4T 36.4.4?

For reference, my current sensor mode is 2064x1552, 2-lane CSI, monochrome IMX900, and V4L2 capture is working correctly.

I want to avoid modifying the NVIDIA camera driver unnecessarily and would like to use the recommended NVIDIA-supported approach for obtaining grayscale data.

Thanks.

  1. Yes, you can
  2. Using the videoconvert videoconvert
  3. Yes, but your sensor is Bayer that not possible to convert to Y8 by NVIDIA VI
  4. Output Y8 only for YUV sensor like output YUV422 sensor.

Could you please help me how to stream or take image using v4l2 ?

Just provide me sample formats or cmd which will help me out.