Problems on porting ov9281 csi camera on our carrier board

  • 1 BSP environment:
    TX2 jetpack 4.6 L4T R32.6.1 kernel 4.9 aarch64
  • 2 operation:
    DT modify & i2c config → done
    nvcsi vi binding → done
root@t-desktop:~# media-ctl -p -d /dev/media0 
Media controller API version 0.1.0

Media device information
------------------------
driver          tegra-vi4
model           NVIDIA Tegra Video Input Device
serial          
bus info        
hw revision     0x3
driver version  0.0.0

Device topology
- entity 1: 150c0000.nvcsi--1 (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
	pad0: Sink
		<- "ov9281 2-0060":0 [ENABLED]
	pad1: Source
		-> "vi-output, ov9281 2-0060":0 [ENABLED]

- entity 4: ov9281 2-0060 (1 pad, 1 link)
            type V4L2 subdev subtype Sensor flags 0
            device node name /dev/v4l-subdev1
	pad0: Source
		[fmt:SBGGR10_1X10/1280x800 field:none colorspace:srgb]
		-> "150c0000.nvcsi--1":0 [ENABLED]

- entity 6: vi-output, ov9281 2-0060 (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video0
	pad0: Sink
		<- "150c0000.nvcsi--1":1 [ENABLED]

v4l2-ctl --all → done

root@t-desktop:~# v4l2-ctl --all
Driver Info (not using libv4l2):
	Driver name   : tegra-video
	Card type     : vi-output, ov9281 2-0060
	Bus info      : platform:15700000.vi:2
	Driver version: 4.9.253
	Capabilities  : 0x84200001
		Video Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps   : 0x04200001
		Video Capture
		Streaming
		Extended Pix Format
Priority: 2
Video input : 0 (Camera 2: no power)
Format Video Capture:
	Width/Height      : 1280/800
	Pixel Format      : 'BG10'
	Field             : None
	Bytes per Line    : 2560
	Size Image        : 2048000
	Colorspace        : sRGB
	Transfer Function : Default (maps to sRGB)
	YCbCr/HSV Encoding: Default (maps to ITU-R 601)
	Quantization      : Default (maps to Full Range)
	Flags             : 

Camera Controls

                   frame_length 0x009a2000 (int)    : min=1 max=65535 step=1 default=1820 value=1820 flags=slider
                    coarse_time 0x009a2001 (int)    : min=1 max=1048575 step=1 default=10896 value=10896 flags=slider
                     group_hold 0x009a2003 (intmenu): min=0 max=1 default=0 value=0
                     hdr_enable 0x009a2004 (intmenu): min=0 max=1 default=0 value=0
                       otp_data 0x009a2006 (str)    : min=0 max=64 step=2 value='0000000000000000000000000000000000000000000000000000000000000000' flags=read-only, has-payload
                        fuse_id 0x009a2007 (str)    : min=0 max=32 step=2 value='00000000000000000000000000000000' flags=read-only, has-payload
                           gain 0x009a2009 (int)    : min=1 max=8191 step=1 default=16 value=16 flags=slider
                    bypass_mode 0x009a2064 (intmenu): min=0 max=1 default=0 value=0
                override_enable 0x009a2065 (intmenu): min=0 max=1 default=0 value=0
                   height_align 0x009a2066 (int)    : min=1 max=16 step=1 default=1 value=1
                     size_align 0x009a2067 (intmenu): min=0 max=2 default=0 value=0
               write_isp_format 0x009a2068 (int)    : min=1 max=1 step=1 default=1 value=1
       sensor_signal_properties 0x009a2069 (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
        sensor_image_properties 0x009a206a (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
      sensor_control_properties 0x009a206b (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
              sensor_dv_timings 0x009a206c (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
               low_latency_mode 0x009a206d (bool)   : default=0 value=0
               preferred_stride 0x009a206e (int)    : min=0 max=65535 step=1 default=0 value=0
                   sensor_modes 0x009a2082 (int)    : min=0 max=30 step=1 default=30 value=3 flags=read-only

v4l2-ctl --list-formats-ext → done

root@t-desktop:~# v4l2-ctl  --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'BG10'
	Name        : 10-bit Bayer BGBG/GRGR
		Size: Discrete 1280x800
			Interval: Discrete 0.017s (60.000 fps)
		Size: Discrete 1280x720
			Interval: Discrete 0.017s (60.000 fps)
		Size: Discrete 640x400
			Interval: Discrete 0.017s (60.000 fps)
  • 3 problems

3.1. we use the following command to build pipeline.

gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM), width=(int)640, height=(int)400, format=(string)NV12, framerate=(fraction)30/1' ! nvvidconv ! nvegltransform ! nveglglessink -e

however gst failed, and the console shows that:

Setting pipeline to PAUSED ...

Using winsys: x11 
Pipeline is live and does not need PREROLL ...
Got context from element 'eglglessink0': gst.egl.EGLDisplay=context, display=(GstEGLDisplay)NULL;
Setting pipeline to PLAYING ...
New clock: GstSystemClock
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 2592 x 1944 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 16.000000; Exposure Range min 34000, max 550385000;

GST_ARGUS: 2592 x 1458 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 16.000000; Exposure Range min 34000, max 550385000;

GST_ARGUS: 1280 x 720 FR = 120.000005 fps Duration = 8333333 ; Analog Gain range min 1.000000, max 16.000000; Exposure Range min 22000, max 358733000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 2 
   Output Stream W = 1280 H = 720 
   seconds to Run    = 0 
   Frame Rate = 120.000005 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
nvbuf_utils: dmabuf_fd -1 mapped entry NOT found
nvbuf_utils: Can not get HW buffer from FD... Exiting...
CONSUMER: Done Success
Got EOS from element "pipeline0".
Execution ended after 0:00:04.082451992
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
GST_ARGUS: Cleaning up
GST_ARGUS: Done Success
(Argus) Error Timeout:  (propagating from src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 137)
(Argus) Error Timeout:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
Setting pipeline to NULL ...
Freeing pipeline ...

what`s more, the gst-launch command above can work normally on the kit board.

3.2. two data lanes differential signals are captured by oscilloscope, however, the clock lane goes flatline.

Any help would be greatly appreciated.
yours
henry

Suppose it could be sensor driver or HW problem if clock lanes didn’t at correct state.
Maybe consult with sensor vendor first.

thans for your reply.
ov9281 is a greyscale sensor, which output pixel format is 10 bits V4L2_PIX_FMT_Y10P. My questions are:

  • 1 how to modify the DT to match the V4L2_PIX_FMT_Y10P format. we modified mode_type to “yuv”, pixel_phase to ‘yuyv’, is it right? if not, how to set these two properties?

  • 2 is there any patch to support this V4L2_PIX_FMT_Y10P “extract_pixel_format”?

YUI
system log shows that:

[    3.579190] ov9281 2-0060: probing v4l2 sensor.
[    3.589589] extract_pixel_format: Need to extend formatyuv_yuyv10
[    3.589591] ov9281 2-0060: Unsupported pixel format
[    3.589594] ov9281 2-0060: Failed to read mode0 image props
[    3.589599] ov9281 2-0060: Could not initialize sensor properties.
[    3.589601] ov9281 2-0060: Failed to initialize ov9281
[    3.589659] ov9281: probe of 2-0060 failed with error -22

Any help would be greatly appreciated.
yours
henry

It’s Bayer format?
Have reference to below topic.

Thanks for your reply.
Accoring to the reference above, we made V4L2_PIX_FMT_Y10 format supported, and the errors of probing ov9281 disappeared.
v4l2-ctl --list-formats-ext shows that pixel format has changed to Y10, which is comforting.
My questions are:

  • 1. “v4l2-ctl --all” and “media-ctl -p -d /dev/media0” still shows BG10 format, is this normal? I suppose all format could change to Y10.

  • 2. Run “v4l2-ctl --stream-mmap -d /dev/video0” failed, and system log shows “tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -1 tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel”.

FYI

root@t-desktop:~# v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'Y10 '
	Name        : 10-bit Greyscale
		Size: Discrete 1280x800
			Interval: Discrete 0.017s (60.000 fps)
		Size: Discrete 1280x720
			Interval: Discrete 0.017s (60.000 fps)
		Size: Discrete 640x400
			Interval: Discrete 0.017s (60.000 fps)

root@t-desktop:~# v4l2-ctl --all
Driver Info (not using libv4l2):
	Driver name   : tegra-video
	Card type     : vi-output, ov9281 2-0060
	Bus info      : platform:15700000.vi:2
	Driver version: 4.9.253
	Capabilities  : 0x84200001
		Video Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps   : 0x04200001
		Video Capture
		Streaming
		Extended Pix Format
Priority: 2
Video input : 0 (Camera 2: no power)
Format Video Capture:
	Width/Height      : 1280/800
	Pixel Format      : 'BG10'
	Field             : None
	Bytes per Line    : 2560
	Size Image        : 2048000
	Colorspace        : sRGB
	Transfer Function : Default (maps to sRGB)
	YCbCr/HSV Encoding: Default (maps to ITU-R 601)
	Quantization      : Default (maps to Full Range)
	Flags             : 
root@t-desktop:~# media-ctl -p -d /dev/media0 
Media controller API version 0.1.0

Media device information
------------------------
driver          tegra-vi4
model           NVIDIA Tegra Video Input Device
serial          
bus info        
hw revision     0x3
driver version  0.0.0

Device topology
- entity 1: 150c0000.nvcsi--1 (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
	pad0: Sink
		<- "ov9281 2-0060":0 [ENABLED]
	pad1: Source
		-> "vi-output, ov9281 2-0060":0 [ENABLED]

- entity 4: ov9281 2-0060 (1 pad, 1 link)
            type V4L2 subdev subtype Sensor flags 0
            device node name /dev/v4l-subdev1
	pad0: Source
		[fmt:SBGGR10_1X10/1280x800 field:none colorspace:srgb]
		-> "150c0000.nvcsi--1":0 [ENABLED]

- entity 6: vi-output, ov9281 2-0060 (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video0
	pad0: Sink
		<- "150c0000.nvcsi--1":1 [ENABLED]

Any help would be greatly appreciated.
yours
henry

Have a check the trace log for more information.

https://elinux.org/Jetson/l4t/Camera_BringUp

No help, same response as before.

I mean to check the trace log to figure if any more information.

media-ctl still shows BG10 format, is this normal?

Suppose not normal.
Maybe double check the device tree.

hi
Thanks for your reply.
My questions are.

  • How to move out the irrelevant DT. For example, there are DTS supporting imx cameras in the tegra186-quill-camera-modules.dtsi. If we comment them, compiling failed. We need some introductions on how these devices (such as e3333 fragment, imx185, etc) are build together by DT.

  • How to show the nvcsi and vi trace log. We need you offer some commands to change debug level to show more imformation about nvcsi and vi. So that we can throubleshooting efficiently.

Thank you!

  1. I don’t have experience for this kind of sensor need to check.
  2. Below is the trace log enable command.
echo 1 > /sys/kernel/debug/tracing/tracing_on
echo 30720 > /sys/kernel/debug/tracing/buffer_size_kb
echo 1 > /sys/kernel/debug/tracing/events/tegra_rtcpu/enable
echo 1 > /sys/kernel/debug/tracing/events/freertos/enable
echo 2 > /sys/kernel/debug/camrtc/log-level
echo 1 > /sys/kernel/debug/tracing/events/camera_common/enable
echo > /sys/kernel/debug/tracing/trace
cat /sys/kernel/debug/tracing/trac

Hi @ShaneCCC
Thanks for your replay.
We run the command “echo 1 > /sys/kernel/debug/tracing/events/freertos/enable”, and then, run “v4l2-ctl --set-fmt-video=width=280,height=800,pixelformat=Y10 --stream-mmap -d /dev/video0”. Console shows that stream is transfered at 60fps (see below), which matches with the sensor reg configration.
However, the tracelog shows “rtos_queue_send_failed”. And no frame got by gst-launch. We need your help.
Gegards,
Henry.

FUI

root@t-desktop:/home/t# v4l2-ctl --set-fmt-video=width=280,height=800,pixelformat=Y10 --stream-mmap -d /dev/video0
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 61.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 60.69 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 60.66 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 60.59 fps
     kworker/0:2-1148  [000] ....   283.421104: rtos_queue_send_from_isr_failed: tstamp:9004081815 queue:0x0b4b04d8
     kworker/0:2-1148  [000] ....   283.421105: rtos_queue_send_from_isr_failed: tstamp:9004081952 queue:0x0b4b1298
     kworker/0:2-1148  [000] ....   283.421106: rtos_queue_send_from_isr_failed: tstamp:9004082090 queue:0x0b4b2058
     kworker/0:2-1148  [000] ....   283.421107: rtos_queue_send_failed: tstamp:9004082720 queue:0x0b4a7698
     kworker/0:2-1148  [000] ....   283.421108: rtos_queue_send_from_isr_failed: tstamp:9004087838 queue:0x0b4a7698
     kworker/0:2-1148  [000] ....   283.421109: rtos_queue_send_from_isr_failed: tstamp:9004088008 queue:0x0b4ab1a8
     kworker/0:2-1148  [000] ....   283.421109: rtos_queue_send_from_isr_failed: tstamp:9004088148 queue:0x0b4acdd8
     kworker/0:2-1148  [000] ....   283.421110: rtos_queue_send_from_isr_failed: tstamp:9004088288 queue:0x0b4ae958
     kworker/0:2-1148  [000] ....   283.421110: rtos_queue_send_from_isr_failed: tstamp:9004088426 queue:0x0b4af718
     kworker/0:2-1148  [000] ....   283.421111: rtos_queue_send_from_isr_failed: tstamp:9004088564 queue:0x0b4b04d8
     kworker/0:2-1148  [000] ....   283.421112: rtos_queue_send_from_isr_failed: tstamp:9004088703 queue:0x0b4b1298
     kworker/0:2-1148  [000] ....   283.421112: rtos_queue_send_from_isr_failed: tstamp:9004088865 queue:0x0b4b2058
     kworker/0:2-1148  [000] ....   283.421113: rtos_queue_send_failed: tstamp:9004090248 queue:0x0b4a7698
     kworker/0:2-1148  [000] ....   283.589190: rtos_queue_peek_from_isr_failed: tstamp:9009018088 queue:0x0b4b4940
     kworker/0:2-1148  [000] ....   283.757181: rtos_queue_peek_from_isr_failed: tstamp:9014018092 queue:0x0b4b4940
     kworker/0:2-1148  [000] ....   283.925188: rtos_queue_peek_from_isr_failed: tstamp:9019018101 queue:0x0b4b4940
     kworker/0:2-1148  [000] ....   284.093143: rtos_queue_peek_from_isr_failed: tstamp:9024018109 queue:0x0b4b4940
root@t-desktop:/home/t# gst-launch-1.0 v4l2src device="/dev/video0" ! "video/x-raw, width=(int)1280, height=(int)800, format=(string)UYVY" ! nvvidconv ! xvimagesink -e
Setting pipeline to PAUSED ... 
Pipeline is live and does not need PREROLL ... 
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to PAUSED ... 
Setting pipeline to READY ... 
Setting pipeline to NULL ... 
Freeing pipeline ... 

The result of v4l2-ctl command tell able capture frames from sensor normally that confirm the driver implement without problem.
However the v4l2src don’t support Y10 that could be the root cause the gst-launch-1.0 v4l2src failed.

Hi @ShaneCCC
Thanks for your reply.

Is there any src pad or other tools support Y10?

Regards,
Henry.

Sorry, I don’t have idea about it. However you should be able to implement v4l2 base APP for it.
Or check if able configure sensor output Grey8 or Y16.

Hi @ShaneCCC
As you know, nvgstcapture does not support greyscale format frame, unfortunatly which is very common in the Industrial control field. So that, is any feature focusing on the situation being developed on NVIDA schedule? Or simply, port v4l2 to support.
Regards,
Henry.

Sorry, nvgstcapture only support Bayer Raw sensor and don’t have plan for it now.

Hi @ShaneCCC
There is a ov5693 camera board on the dev kit. And its number is 180-83326-DAAF-A01. We need its schematic.
Regards,
Henry.

@Henry.Lou
Sorry to tell we can’t public it. However you can check the camera design guide from download center.