Imx 296 mipi camera sensor for xavier nx dev kit

i am trying to get imx 296 sensor up on xavier nx devkit. sensor is connected to CSI port 2. sensor is getting detected but /dev/video is not created.

/dev/video0 is getting created when two camera modules are enabled for csi port 0 and 2. even if there is no sensor connected to csi port 0.

hello shivlal12345,

may I know what’s the actual use-case?
you should have same device tree definition to match your hardware connections.
for example, if you have single camera connected to csi-2, you should have DT to define csi-2 only.

starting from l4t-r35.x, it’s supporting device tree overlay,
you may create a device-tree overlay file to register the camera module.
thanks

Hi JerryChang,

I did that. enabled only that csi port. imx sensor is detected but /dev/video0 is not created. it’s being created only when two cameras csi port 0 and csi port 2 are enabled in device tree.

now I have connected sensor to csi port 0. enabled only single camera in device tree. sensor is getting detected. /dev/video0 is also created. captured images using v4l2-ctl. but I am getting all black images. checked dmesg log. there is no error. also I am getting a “no camera available” error with nvidia argus.

Thanks,

hello shivlal12345,

can you capture a complete frame successfully with v4l2-ctl?
for example,
$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=10
it should shows ‘<’ to indicate each success capture frames, and also report average fps below the command-line.

yes I can capture frames using this command. but images are black. data bytes are 0

hello shivlal12345,

please review formats by $ v4l2-ctl -d /dev/video0 --list-formats-ext, you should assign exactly same support formats to the v4l pipeline.
you should review your sensor signaling if the capture frames still shows all zero.

it supports RG10, resolution 1440x1080. I am passing rg10 to v4l capture command. it captures without any error. but the images are black. bytes are zero.

v4l2-ctl -d /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: ‘RG10’
Name : 10-bit Bayer RGRG/GBGB
Size: Discrete 1440x1080
Interval: Discrete infs (0.000 fps)

dmesg log.
dmesg | grep i2c
[ 0.725338] iommu: Adding device 3160000.i2c to group 10
[ 0.725673] iommu: Adding device c240000.i2c to group 11
[ 0.726044] iommu: Adding device 3180000.i2c to group 12
[ 0.726419] iommu: Adding device 3190000.i2c to group 13
[ 0.726872] iommu: Adding device 31b0000.i2c to group 14
[ 0.727284] iommu: Adding device 31c0000.i2c to group 15
[ 0.727631] iommu: Adding device c250000.i2c to group 16
[ 0.727982] iommu: Adding device 31e0000.i2c to group 17
[ 0.844237] tegra-i2c 31e0000.i2c: could not find pctldev for node /host1x/dpaux@155F0000/pinmux@0, deferring probe
[ 1.958759] i2c /dev entries driver
[ 1.960766] i2c i2c-2: Added multiplexed i2c bus 9
[ 1.960776] i2c-mux-gpio cam_i2cmux: 1 port mux on 3180000.i2c adapter
[ 2.064080] i2c 9-0010: ±-- VC MIPI Camera -----------------------------------+
[ 2.064089] i2c 9-0010: | MANUF. | Vision Components MID: 0x0427 |
[ 2.064097] i2c 9-0010: | MODULE | ID: 0x0296 REV: 0040 |
[ 2.064103] i2c 9-0010: | SENSOR | SONY IMX296C |
[ 2.064109] i2c 9-0010: ±-------±--------------------------------------------+
[ 2.064114] i2c 9-0010: ±-- Sensor Registers ------±-------±-------±-------+
[ 2.064119] i2c 9-0010: | | low | mid | high |
[ 2.064124] i2c 9-0010: ±--------------------------±-------±-------±-------+
[ 2.064130] i2c 9-0010: | idle | 0x3000 | | |
[ 2.064136] i2c 9-0010: | horizontal start | 0x3310 | 0x3311 | |
[ 2.064142] i2c 9-0010: | vertical start | 0x3312 | 0x3313 | |
[ 2.064148] i2c 9-0010: | horizontal end | 0x0000 | 0x0000 | |
[ 2.064154] i2c 9-0010: | vertical end | 0x4182 | 0x4183 | |
[ 2.064160] i2c 9-0010: | hor. output width | 0x3314 | 0x3315 | |
[ 2.064166] i2c 9-0010: | ver. output height | 0x3316 | 0x3317 | |
[ 2.064172] i2c 9-0010: | exposure | 0x308d | 0x308e | 0x308f |
[ 2.064178] i2c 9-0010: | gain | 0x3204 | 0x3205 | |
[ 2.064183] i2c 9-0010: ±--------------------------±-------±-------±-------+
[ 2.064190] i2c 9-0010: | clock for ext. trigger | 54000000 Hz |
[ 2.064196] i2c 9-0010: | pixel clock | 74250000 Hz |
[ 2.064202] i2c 9-0010: | shutter offset | 770 us |
[ 2.064207] i2c 9-0010: ±--------------------------±-------------------------+
[ 2.064212] i2c 9-0010: ±-- Module Modes -------±--------±--------±--------+
[ 2.064217] i2c 9-0010: | # | rate | lanes | format | type | binning |
[ 2.064222] i2c 9-0010: ±—±--------±--------±--------±--------±--------+
[ 2.064230] i2c 9-0010: | 0 | 1188 | 1 | RAW10 | STREAM | 0 |
[ 2.064237] i2c 9-0010: | 1 | 1188 | 1 | RAW10 | EXT.TRG | 0 |
[ 2.064243] i2c 9-0010: ±—±--------±--------±--------±--------±--------+
[ 2.064250] i2c 9-0010: vc_init_ctrl_imx296(): Initialising module control for IMX296
[ 2.064258] i2c 9-0010: ±------±-------±---------±----------+
[ 2.064263] i2c 9-0010: | lanes | format | exposure | framerate |
[ 2.064268] i2c 9-0010: | | | max [us] | max [mHz] |
[ 2.064273] i2c 9-0010: ±------±-------±---------±----------+
[ 2.064280] i2c 9-0010: | 1 | RAW10 | 15533515 | 60816 |
[ 2.064285] i2c 9-0010: ±------±-------±---------±----------+
[ 2.064290] i2c 9-0010: VC MIPI Core succesfully initialized
[ 2.064306] i2c 9-0010: vc_mod_set_trigger_mode(): Set trigger mode: DISABLED
[ 2.064312] i2c 9-0010: vc_mod_set_io_mode(): Set IO mode: DISABLED
[ 217.766911] i2c 9-0010: vc_mod_set_mode(): Set module mode: 0 (lanes: 1, format: RG10, type: STREAM)
[ 217.767521] i2c 9-0010: vc_mod_set_power(): Set module power: down
[ 217.767873] i2c 9-0010: vc_mod_set_power(): Set module power: up

nvidia argus api not able to detect /dev/video0. i am getting an error “No cameras available”.

cam_path = /proc/devicetree/cam_i2cmux/i2c@0/vc_mipi@1a

media-ctl -p
Media controller API version 0.1.0

Media device information

driver tegra194-vi5
model NVIDIA Tegra Video Input Device
serial
bus info
hw revision 0x3
driver version 0.0.0

Device topology

  • entity 1: vc_mipi 9-001a (1 pad, 1 link)
    type V4L2 subdev subtype Sensor flags 0
    device node name /dev/v4l-subdev0
    pad0: Source
    [fmt:SRGGB10_1X10/1440x1080 field:none colorspace:srgb]
    → “15a00000.nvcsi–1”:0 [ENABLED]

  • entity 3: 15a00000.nvcsi–1 (2 pads, 2 links)
    type V4L2 subdev subtype Unknown flags 0
    device node name /dev/v4l-subdev1
    pad0: Sink
    ← “vc_mipi 9-001a”:0 [ENABLED]
    pad1: Source
    → “vi-output, vc_mipi 9-001a”:0 [ENABLED]

  • entity 6: vi-output, vc_mipi 9-001a (1 pad, 1 link)
    type Node subtype V4L flags 0
    device node name /dev/video0
    pad0: Sink
    ← “15a00000.nvcsi–1”:1 [ENABLED]

/dev/video0 is present but it’s not detected by nvarguscamerasrc .
cam_path = /proc/device-tree/cam_i2cmux/i2c@0/vc_mipi@1a
path order matches with original device tree.

gst-launch-1.0 nvarguscamerasrc sensor-id=0
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:725 No cameras available
^Chandling interrupt.
Interrupt: Stopping pipeline …
Execution ended after 0:00:02.360822706
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …

hello shivlal12345,

this looks incorrect.
BTW, you’ll need to verify v4l2 able to capture images first.

Interval: Discrete infs (0.000 fps)

I also suspected this. there should be some number other than 0 for fps? how do i fix it? can capture using v4l2-ctrl
tonbo@tonbo-xavier:~/vc_mipi_nvidia/target$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=1440,height=1080,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1
<

dmesg log while capturing
[70794.661124] vc_mipi 9-001a: vc_core_set_format(): Set format 0x300f (RG10)
[70794.661326] vc_mipi 9-001a: vc_core_try_format(): Try format 0x300f (RG10, format: 0x2b)
[70794.661333] vc_mipi 9-001a: vc_update_image_size_from_mode(): Update image size from mode0 (l: 0, t: 0, w: 1440, h: 1080)
[70794.661565] vc_mipi 9-001a: vc_sen_set_gain(): Set sensor gain: 100
[70794.662014] vc_mipi 9-001a: vc_sen_set_exposure(): Set sensor exposure: 155335 us
[70794.663006] vc_mipi 9-001a: vc_core_set_framerate(): Set framerate 60816 mHz
[70794.663130] vc_mipi 9-001a: vc_sen_set_exposure(): Set sensor exposure: 155335 us
[70794.664121] vc_mipi 9-001a: vc_sen_set_roi(): Set sensor roi: (left: 0, top: 0, width: 1440, height: 1080)
[70794.665944] vc_mipi 9-001a: vc_sen_set_exposure(): Set sensor exposure: 155335 us
[70794.667077] vc_mipi 9-001a: vc_sen_start_stream(): Start streaming
[70794.689310] [RCE] vi5_hwinit: firmware CL2018101701 protocol version 2.2
[70796.067480] vc_mipi 9-001a: vc_sen_stop_stream(): Stop streaming

image bytes are zero. I didn’t see any error in dmesg log or syslog.

hello shivlal12345,

how about increase stream counts, i.e. --stream-count=100.

tonbo@tonbo-xavier:~/vc_mipi_nvidia/target$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=1440,height=1080,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=100
<< 0.71 fps
< 0.76 fps
< 0.80 fps
< 0.78 fps
<< 0.84 fps
<<< 1.03 fps
<<< 1.13 fps
<< 1.26 fps
<< 1.32 fps
< 1.17 fps
<< 1.21 fps
< 1.23 fps
< 1.20 fps
<<< 1.28 fps
< 1.29 fps
<< 1.34 fps
< 1.27 fps
< 1.26 fps
< 1.27 fps
<< 1.28 fps
< 1.27 fps
< 1.22 fps
< 1.15 fps
<< 1.18 fps
<< 1.21 fps
< 1.15 fps
< 1.12 fps
< 1.07 fps
<< 1.07 fps
< 1.01 fps
<< 1.01 fps
< 1.01 fps
< 1.02 fps
< 0.96 fps
< 0.97 fps
<<< 1.01 fps
< 1.00 fps
<<< 1.04 fps
<<< 1.05 fps
< 1.05 fps
< 1.06 fps
<< 1.08 fps
<<< 1.11 fps
<<< 1.13 fps
< 1.12 fps
< 1.13 fps
<< 1.14 fps
<< 1.13 fps
<< 1.11 fps
< 1.10 fps
<<<<< 1.16 fps
<< 1.16 fps
< 1.16 fps
< 1.16 fps
<<< 1.18 fps
<< 1.20 fps
<<<<< 1.24 fps

hello shivlal12345,

that’s looks odd, do you know what’s the output frame-rate of this camera sensor?

may I also know which JetPack release you’re using? please enable dynamic debug flag if you’re using k-4.9.
it’ll enable dev_dbg() to print messages to kernel logs.
for example,
# cd /sys/kernel/debug/dynamic_debug/
# echo file channel.c +p > control

output frame rate 60. jetpack 4.6. release 32.6.1 xavier nx dev kit.

https://www.vision-components.com/fileadmin/external/documentation/hardware/VC_MIPI_Camera_Module/index.html

imx sensor 296c

camera dtsi vc_mipi_nvidia/src/devicetree/NV_DevKit_XavierNX at master · VC-MIPI-modules/vc_mipi_nvidia · GitHub

v4l2-ctl -d /dev/video0 --set-fmt-video=width=1440,height=1080,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=100

dmesg log after dynamic debug flag

[74279.832278] [RCE] vi5_hwinit: firmware CL2018101701 protocol version 2.2
[74279.832368] vc_mipi 9-001a: vc_core_set_format(): Set format 0x300f (RG10)
[74279.832539] vc_mipi 9-001a: vc_core_try_format(): Try format 0x300f (RG10, format: 0x2b)
[74279.832544] vc_mipi 9-001a: vc_core_try_format(): Checking mode 0 (format: 0x2b)
[74279.832550] vc_mipi 9-001a: vc_core_calculate_max_exposure(): period_1H_ns: 14814, vmax.max: 1048575, vmax.min: 5
[74279.832556] vc_mipi 9-001a: vc_core_calculate_max_frame_rate(): period_1H_ns: 14814, vmax: 1110/1110
[74279.832559] vc_mipi 9-001a: vc_core_update_controls(): num_lanes: 1, format 43, exposure.max: 15533515 us, framerate.max: 60816 mHz
[74279.832573] vc_mipi 9-001a: vc_update_image_size_from_mode(): Update image size from mode0 (l: 0, t: 0, w: 1440, h: 1080)
[74279.832779] vc_mipi 9-001a: vc_sen_set_gain(): Set sensor gain: 100
[74279.832899] vc_mipi 9-001a: vc_sen_set_gain(): addr: 0x3204 <= value: 0x64
[74279.833080] vc_mipi 9-001a: vc_sen_set_gain(): addr: 0x3205 <= value: 0x00
[74279.833228] vc_mipi 9-001a: vc_sen_set_exposure(): Set sensor exposure: 155335 us
[74279.833365] vc_mipi 9-001a: vc_core_calculate_vmax(): framerate: 60816 mHz, frametime: 16443041 ns, 1109 1H
[74279.833370] vc_mipi 9-001a: vc_calculate_exposure(): flags: 0x1692, period_1H_ns: 14814, shs: 5/5, vmax: 10490/1110
[74279.833374] vc_mipi 9-001a: vc_sen_write_shs(): Write sensor SHS: 0x00000005 (5)
[74279.833378] vc_mipi 9-001a: vc_sen_write_shs(): addr: 0x308d <= value: 0x05
[74279.833532] vc_mipi 9-001a: vc_sen_write_shs(): addr: 0x308e <= value: 0x00
[74279.833697] vc_mipi 9-001a: vc_sen_write_shs(): addr: 0x308f <= value: 0x00
[74279.833868] vc_mipi 9-001a: vc_sen_write_vmax(): Write sensor VMAX: 0x000028fa (10490)
[74279.833873] vc_mipi 9-001a: vc_sen_write_vmax(): addr: 0x3010 <= value: 0xfa
[74279.834018] vc_mipi 9-001a: vc_sen_write_vmax(): addr: 0x3011 <= value: 0x28
[74279.834163] vc_mipi 9-001a: vc_sen_write_vmax(): addr: 0x3012 <= value: 0x00
[74279.834309] vc_mipi 9-001a: vc_sen_set_exposure(): (VMAX: 10490, SHS: 5), (RETC: 0, EXPC: 0)
[74279.834315] vc_mipi 9-001a: vc_core_set_framerate(): Set framerate 60816 mHz
[74279.834439] vc_mipi 9-001a: vc_sen_set_exposure(): Set sensor exposure: 155335 us
[74279.834589] vc_mipi 9-001a: vc_core_calculate_vmax(): framerate: 60816 mHz, frametime: 16443041 ns, 1109 1H
[74279.834594] vc_mipi 9-001a: vc_calculate_exposure(): flags: 0x1692, period_1H_ns: 14814, shs: 5/5, vmax: 10490/1110
[74279.834598] vc_mipi 9-001a: vc_sen_write_shs(): Write sensor SHS: 0x00000005 (5)
[74279.834601] vc_mipi 9-001a: vc_sen_write_shs(): addr: 0x308d <= value: 0x05
[74279.834749] vc_mipi 9-001a: vc_sen_write_shs(): addr: 0x308e <= value: 0x00
[74279.834893] vc_mipi 9-001a: vc_sen_write_shs(): addr: 0x308f <= value: 0x00
[74279.835037] vc_mipi 9-001a: vc_sen_write_vmax(): Write sensor VMAX: 0x000028fa (10490)
[74279.835041] vc_mipi 9-001a: vc_sen_write_vmax(): addr: 0x3010 <= value: 0xfa
[74279.835185] vc_mipi 9-001a: vc_sen_write_vmax(): addr: 0x3011 <= value: 0x28
[74279.835396] vc_mipi 9-001a: vc_sen_write_vmax(): addr: 0x3012 <= value: 0x00
[74279.835559] vc_mipi 9-001a: vc_sen_set_exposure(): (VMAX: 10490, SHS: 5), (RETC: 0, EXPC: 0)
[74279.835570] i2c 9-0010: vc_mod_find_mode(): Checking mode (#00, lanes: 1, format: 0x2b, type: 0x01, binning: 0x00)
[74279.835579] i2c 9-0010: vc_mod_set_mode(): Module mode 0 already set!
[74279.835586] vc_mipi 9-001a: vc_sen_set_roi(): Set sensor roi: (left: 0, top: 0, width: 1440, height: 1080)
[74279.835779] vc_mipi 9-001a: vc_sen_set_roi(): addr: 0x3310 <= value: 0x00
[74279.835931] vc_mipi 9-001a: vc_sen_set_roi(): addr: 0x3311 <= value: 0x00
[74279.836099] vc_mipi 9-001a: vc_sen_set_roi(): addr: 0x3312 <= value: 0x00
[74279.836401] vc_mipi 9-001a: vc_sen_set_roi(): addr: 0x3313 <= value: 0x00
[74279.836572] vc_mipi 9-001a: vc_sen_set_roi(): addr: 0x3314 <= value: 0xa0
[74279.836720] vc_mipi 9-001a: vc_sen_set_roi(): addr: 0x3315 <= value: 0x05
[74279.836865] vc_mipi 9-001a: vc_sen_set_roi(): addr: 0x3316 <= value: 0x38
[74279.837010] vc_mipi 9-001a: vc_sen_set_roi(): addr: 0x3317 <= value: 0x04
[74279.837153] vc_mipi 9-001a: vc_sen_set_roi(): addr: 0x4182 <= value: 0x38
[74279.837297] vc_mipi 9-001a: vc_sen_set_roi(): addr: 0x4183 <= value: 0x04
[74279.837441] vc_mipi 9-001a: vc_sen_set_exposure(): Set sensor exposure: 155335 us
[74279.837581] vc_mipi 9-001a: vc_core_calculate_vmax(): framerate: 60816 mHz, frametime: 16443041 ns, 1109 1H
[74279.837586] vc_mipi 9-001a: vc_calculate_exposure(): flags: 0x1692, period_1H_ns: 14814, shs: 5/5, vmax: 10490/1110
[74279.837589] vc_mipi 9-001a: vc_sen_write_shs(): Write sensor SHS: 0x00000005 (5)
[74279.837593] vc_mipi 9-001a: vc_sen_write_shs(): addr: 0x308d <= value: 0x05
[74279.837741] vc_mipi 9-001a: vc_sen_write_shs(): addr: 0x308e <= value: 0x00
[74279.837897] vc_mipi 9-001a: vc_sen_write_shs(): addr: 0x308f <= value: 0x00
[74279.838048] vc_mipi 9-001a: vc_sen_write_vmax(): Write sensor VMAX: 0x000028fa (10490)
[74279.838053] vc_mipi 9-001a: vc_sen_write_vmax(): addr: 0x3010 <= value: 0xfa
[74279.838196] vc_mipi 9-001a: vc_sen_write_vmax(): addr: 0x3011 <= value: 0x28
[74279.838339] vc_mipi 9-001a: vc_sen_write_vmax(): addr: 0x3012 <= value: 0x00
[74279.838484] vc_mipi 9-001a: vc_sen_set_exposure(): (VMAX: 10490, SHS: 5), (RETC: 0, EXPC: 0)
[74279.838487] vc_mipi 9-001a: vc_sen_start_stream(): Start streaming
[74279.838615] vc_mipi 9-001a: vc_sen_start_stream(): MM: 0x00, TM: 0x00, IO: 0x00
[74279.838619] vc_mipi 9-001a: vc_sen_write_mode(): Write sensor mode: operating
[74279.838622] vc_mipi 9-001a: vc_sen_write_mode(): addr: 0x300a <= value: 0x00
[74279.838767] vc_mipi 9-001a: vc_sen_write_mode(): addr: 0x3000 <= value: 0x00
[74279.838911] i2c 9-0010: vc_mod_write_io_mode(): Write IO mode: OFF
[74279.838915] i2c 9-0010: vc_mod_write_io_mode(): addr: 0x0103 <= value: 0x00
[74279.839058] i2c 9-0010: vc_mod_write_trigger_mode(): Write trigger mode: 0x00
[74279.839062] i2c 9-0010: vc_mod_write_trigger_mode(): addr: 0x0108 <= value: 0x00
[74335.332029] vc_mipi 9-001a: vc_sen_stop_stream(): Stop streaming
[74335.332273] i2c 9-0010: vc_mod_write_trigger_mode(): Write trigger mode: 0x00
[74335.332281] i2c 9-0010: vc_mod_write_trigger_mode(): addr: 0x0108 <= value: 0x00
[74335.332502] i2c 9-0010: vc_mod_write_io_mode(): Write IO mode: OFF
[74335.332518] i2c 9-0010: vc_mod_write_io_mode(): addr: 0x0103 <= value: 0x00
[74335.332685] vc_mipi 9-001a: vc_sen_write_mode(): Write sensor mode: standby
[74335.332695] vc_mipi 9-001a: vc_sen_write_mode(): addr: 0x3000 <= value: 0x01
[74335.332856] vc_mipi 9-001a: vc_sen_write_mode(): addr: 0x300a <= value: 0x01
[74335.333013] vc_mipi 9-001a: vc_sen_stop_stream(): ----------------------------------------------------------

this is the output after setting framerate to 60
v4l2-ctl -d /dev/video0 --set-fmt-video=width=1440,height=1080,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=100
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 60.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 47.76 fps
<<<

this looks promising, may I know what’s the modification you’ve done?

set framerate to 60 v4l2-ctl -d /dev/video0 --set-ctrl frame_rate=60000mHz

EVERYTHIING LOOKs okay.

please tell me why it’s not showing none zero fps? why captured frames have all bytes 0000?
v4l2-ctl -d /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: ‘RG10’
Name : 10-bit Bayer RGRG/GBGB
Size: Discrete 1440x1080
Interval: Discrete infs (0.000 fps)