Issue with porting sensor driver on Jetson Linux 35.4.1

Hello nvidia support,
I use Jetson Linux 35.4.1 and modules Xavier NX and Orin NX 16Gb. For both modules I use same custom carrier board.
My target is get image from sensor imx565. For this I prepared imx565.ko and made changes in device tree for both modules.

With module Xavier NX the sensor works well but I have a problem with Orin NX.

when I try to check sensor with command:
v4l2-ctl --set-fmt-video=width=4128,height=3008,pixelformat=RG10 --stream-mmap --set-ctrl=sensor_mode=0 --stream-count=100 -d /dev/video0

I got error messages:

[  867.429335] bwmgr API not supported
[  867.437559] imx565_set_mode, mode =  0
[  867.472447] imx565_start_streaming, start streaming from csi 0
[  867.478474] imx565_start_streaming auto black level disabled
[  867.484851] imx565_start_streaming master mode
[  868.993984] imx565_start_streaming, without errors 
[  870.088805] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[  870.097948] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[  870.108258] (NULL device *): vi_capture_control_message: NULL VI channel received
[  870.115983] t194-nvcsi 13e40000.host1x:nvcsi@15a00000: csi5_stream_close: Error in closing stream_id=0, csi_port=0
[  870.126640] (NULL device *): vi_capture_control_message: NULL VI channel received

You can see full log
dmesg.log (68.7 KB)

There is output for media-ctl -p -d /dev/media0

Media controller API version 5.10.120

Media device information
------------------------
driver          tegra-camrtc-ca
model           NVIDIA Tegra Video Input Device
serial          
bus info        
hw revision     0x3
driver version  5.10.120

Device topology
- entity 1: 13e40000.host1x:nvcsi@15a00000- (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
	pad0: Sink
		<- "imx565 0-0036":0 [ENABLED]
	pad1: Source
		-> "vi-output, imx565 0-0036":0 [ENABLED]

- entity 4: 13e40000.host1x:nvcsi@15a00000- (2 pads, 0 link)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev1
	pad0: Sink
	pad1: Source

- entity 7: imx565 0-0036 (1 pad, 1 link)
            type V4L2 subdev subtype Sensor flags 0
            device node name /dev/v4l-subdev2
	pad0: Source
		[fmt:SRGGB10_1X10/4128x3008 field:none colorspace:srgb]
		-> "13e40000.host1x:nvcsi@15a00000-":0 [ENABLED]

- entity 9: vi-output, imx565 0-0036 (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video0
	pad0: Sink
		<- "13e40000.host1x:nvcsi@15a00000-":1 [ENABLED]

and for v4l2-compliance -d /dev/video0

v4l2-compliance SHA: not available, 64 bits

Compliance test for tegra-video device /dev/video0:

Driver Info:
	Driver name      : tegra-video
	Card type        : vi-output, imx565 0-0036
	Bus info         : platform:tegra-capture-vi:0
	Driver version   : 5.10.120
	Capabilities     : 0x84200001
		Video Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04200001
		Video Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : tegra-camrtc-ca
	Model            : NVIDIA Tegra Video Input Device
	Serial           : 
	Bus info         : 
	Media version    : 5.10.120
	Hardware revision: 0x00000003 (3)
	Driver version   : 5.10.120
Interface Info:
	ID               : 0x0300000b
	Type             : V4L Video
Entity Info:
	ID               : 0x00000009 (9)
	Name             : vi-output, imx565 0-0036
	Function         : V4L2 I/O
	Pad 0x0100000a   : 0: Sink
	  Link 0x0200000f: from remote pad 0x1000003 of entity '13e40000.host1x:nvcsi@15a00000-': Data, Enabled

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video0 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls (Input 0):
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 1 Private Controls: 32

Format ioctls (Input 0):
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
		fail: v4l2-test-formats.cpp(1280): ret && node->has_frmintervals
	test VIDIOC_G/S_PARM: FAIL
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls (Input 0):
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls (Input 0):
		fail: v4l2-test-buffers.cpp(715): q.create_bufs(node, 1, &fmt) != EINVAL
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
	test VIDIOC_EXPBUF: OK
	test Requests: OK (Not Supported)

Total for tegra-video device /dev/video0: 45, Succeeded: 43, Failed: 2, Warnings: 0

Can you help me? Thanks

hello kulikov.mh,

may I know which CSI port you’re using,

FYI,
as you can see Jetson Orin NX Series and Orin Nano Series Design Guide for [Figure 10-1. CSI 2-Lane Connection Options]
there’s polarity swap on Orin NX module. you may try below DT setting for polarity swap configuration.
for instance,

 * lane_polarity
 * Based on the camera connector pin.
 * CSIx_D0 | CSIx_D1 | CSI(X+1)_D0 | CSI(X+1)CSIx_D1
 *    LSB  |   BIT1  |     BIT2    |      MSB 

Thank you for your answer.

I use CSI0 port, 4-Lane.
After I have added lane_polarity = "6"; in device tree I checked sensor with command:
v4l2-ctl --set-fmt-video=width=4128,height=3008,pixelformat=RG10 --stream-mmap --set-ctrl=sensor_mode=0 --stream-count=100 -d /dev/video0

At this one I got messages:

[  100.849439] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 1, flags: 0, err_data 2048
[  101.042545] imx565_start_streaming, without errors 
[  101.814063] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 18, flags: 0, err_data 2048
[  101.984439] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 21, flags: 0, err_data 2048
[  102.097240] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 23, flags: 0, err_data 512

Also I was able to save images from the sensor

gst-launch-1.0 \
nvarguscamerasrc sensor-id=0 sensor-mode=0 ! 'video/x-raw(memory:NVMM), width=4128, height=3008,format=(string)NV12, framerate=(fraction)17/1' ! \
nvvidconv flip-method=0 ! nvjpegenc ! multifilesink location=~/out/%d.jpg

It’s look like I received only one half of frame data. What could be the reason of this issue?

hello kulikov.mh,

the discarding frame messages reported as warning messages, it usually due to unsuccessful capture state; VI engine dropping frame and request for new buffers.

according to your sample JPG file, it seems the image deminesion is 1920x1399 via metadata, which mismatch with your command-line settings.
you may double check the sensor format dumps, i.e. $ v4l2-ctl -d /dev/video0 --list-formats-ext

Hello JerryChang,
I fexed this issue. The reason was not corerct pixel_clk_hz.
After I changed it according formula pixel_clk_hz = 891 Mbps × 4 / 10 = 356400000 sensor works well.

But now I have another question.

I try to display video stream.

after I run

gst-launch-1.0 -e nvarguscamerasrc sensor-id=0 sensor_mode=0 ! "video/x-raw(memory:NVMM), width=4128, height=3008, format=NV12,framerate=17/1" ! \
  nvvidconv flip-method=0 ! "video/x-raw,width=1920, height=1080" ! nvvidconv ! nvoverlaysink

I got message: WARNING: erroneous pipeline: no element "nvoverlaysink"

After I have changed nvoverlaysink with nvegltransform I got errors:

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 4128 x 3008 FR = 17.000000 fps Duration = 58823528 ; Analog Gain range min 0.000000, max 48.000000; Exposure Range min 72000, max 56052000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 0 
   Output Stream W = 4128 H = 3008 
   seconds to Run    = 0 
   Frame Rate = 17.000000 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
ERROR: from element /GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3072): gst_base_src_loop (): /GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc0:
streaming stopped, reason error (-5)
EOS on shutdown enabled -- waiting for EOS after Error

How can I display video stream?

hello kulikov.mh,

it’s more like incorrect pipeline were used. nvoverlaysink isn’t available for JP-5.x release version.

please try to disable preview and shows frame-rate only to confirm sensor streaming.
for instance,
$ gst-launch-1.0 -e -v nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM),width=4128, height=3008, framerate=17/1, format=NV12' ! nvvidconv ! 'video/x-raw(memory:NVMM),format=I420' ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0

if above works, you may try rendering preview frames to display.
here’s sample command-line to add video converter to downscale the resolution to 1920x1080 (in case of display monitor limitation)
for instance,
$ export DISPLAY=:0
$ gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM), width=4128, height=3008, format=NV12, framerate=17/1' ! nvvidconv ! 'video/x-raw, format=(string)I420, width=1920, height=1080' ! queue ! xvimagesink -e

Thank you for your support.
Now everything works correctly

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.