MIPI camera driver probing fail

Robotics & Edge Computing Jetson & Embedded Systems Jetson Nano
,
1

Hello I am in the middle of a MIPI camera driver development for the Jetson Nano. I found multiple posts with the similar topic, and tried to follow them and the ‘kernel customization’, ‘camera development’ guides.

I added the corresponding dtsi files to the
Linux_for_Tegra/hardware/nvidia/platform/t210/porg/kernel-dts/porg-platforms
and modified the board basic device tree file to include my file. I also let default imx219 include in the base device tree.

I added my ov5647.c and ov5647_mode_tbls.h files to the: Linux_for_Tegra/kernel/nvidia/drivers/i2c/

Modified the Kconfig and Makefile, and also added the CONFIG_VIDEO_OV5647=m to the tegra_defconfig file. I compiled the sources and flashed the board. Powered off the board and connected the ov5647 sensor and turned on. During the boot I could see error messages from my driver implementation.

$ dmesg | grep ov5647
ov5647 7-0036: probing v4l2 sensor.
ov5647 7-0036: v4l2 driver check.
ov5647 7-0036: i2c_client->name = ov5647
ov5647 7-0036: i2c_client->dev = (null)
ov5647 7-0036: node = (ptrval)
ov5647 7-0036: probing v4l2 sensor at addr 0x36
ov5647 7-0036: Failed to find clocks
ov5647 7-0036: unable to get platform data
ov5647 7-0036: tegra camera driver registration failed with code-14
ov5647: probe of 7-0036 failed with error -14
ov5647 8-0036: probing v4l2 sensor.
ov5647 8-0036: v4l2 driver check.
ov5647 8-0036: i2c_client->name = ov5647
ov5647 8-0036: i2c_client->dev = (null)
ov5647 8-0036: node = (ptrval)
ov5647 8-0036: probing v4l2 sensor at addr 0x36
ov5647 8-0036: Failed to find clocks
ov5647 8-0036: unable to get platform data
ov5647 8-0036: tegra camera driver registration failed with code-14

I also checked the sensor configurations:

zcat /proc/config.gz| grep ‘OV5647’
CONFIG_VIDEO_OV5647=m

I disabled the plugin manager for non EEPROM based camera. I try to debug my driver file but I has a terrible feeling that I messed up the device tree that’s why my driver won’t work anyway.

uname -r (4.9.337-tegra)

I started to implement the driver based on the ov5693 example from the kernel and internet.

3

It’s good idea base on ov5693 to implement it.

Thanks

4

Thank you for your response, can you help me with a technical step?

I am a novice with Jetson and Linux. I tried to compile my sensor driver source (.c) on board to avoid kernel rebuild and re-flashing during development. I am tried to follow the Kernel customization guide and it mentioned to method for build modules on Jetson system:

To build out-of-tree modules, specify the kernel directory as:
/usr/src/linux-headers-$(uname -r)-ubuntu18.04_aarch64
or
/lib/modules/$(uname -r)/build

I tried to implement a Makefile based on the guide:

obj-m += ov5647.o

all:
	make -C /usr/src/linux-headers-$(shell uname -r)-ubuntu18.04_aarch64 M=$(PWD) modules

clean:
	make -C /usr/src/linux-headers-$(shell uname -r)-ubuntu18.04_aarch64 M=$(PWD) clean

response: No rule to make target ‘modules’

obj-m += ov5647.o

all:
	make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules

clean:
	make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean

response: build No such file or directory
I checked and I have ‘build’ at the target directory but it colored red. I already updated the board and installed build-essential. Can I resolve this or I have to cross compile on a different machine?

5

I didn’t try this. I build it by cross compile on x86 ubuntu 18.04 system.

Have reference to below document for it.

https://docs.nvidia.com/jetson/archives/r35.4.1/DeveloperGuide/text/SD/Kernel/KernelCustomization.html

6

I would able to build the driver file on board. And after some further improvement and refactoring I was able to implement a driver version where the device is bound as a I2C. And I was also able to read out the ID of the camera. I added multiple debug messages, for the power up sequence I tried to follow the datasheet but I was not able to power up properly until I found a repo. I can see the video0 and also see the i2c address via i2cdetect.

[ 111.875350] ov5647: loading out-of-tree module taints kernel.
[ 111.881788] ov5647 7-0036: i2c_client->name = ov5647
[ 111.887187] ov5647 7-0036: i2c_client->dev = (null)
[ 111.892183] ov5647 7-0036: node = 00000000022df123
[ 111.897069] ov5647 7-0036: ov5647_parse_dt parse
[ 111.897140] ov5647 7-0036: ov5647_power_get
[ 111.901616] ov5647 7-0036: tegracam sensor driver:ov5647_v2.0.6
[ 111.901620] ov5647 7-0036: ov5647_board_setup: :setup
[ 111.901623] ov5647 7-0036: ov5647_power_on: power on
[ 111.901626] ov5647 7-0036: ov5647_power_on: pdata->power_on Not present, going for gpio reset
[ 111.901630] ov5647 7-0036: ov5647_power_on: In skip_power_seqn, going for gpio reset
[ 111.926391] ov5647 7-0036: ov5647_board_setup: :sensor id: 5647
[ 111.926483] vi 54080000.vi: subdev ov5647 7-0036 bound

But when I try to open with nvgstcapture-1 it says:

Opening in BLOCKING MODE 
** Message: 16:57:47.601: <main:4648> iterating capture loop ....
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:740 No cameras available
NvMMLiteOpen : Block : BlockType = 4

I am a bit clueless that I should work more on the start_streaming function or the power_up function requires further development?

I would be gratefull any help I think the finish line not far.

7

This tell incorrect context in the tega-camera-platform{} special check the strings of the devname.

8

Thank you for your response you were right and I could move forward. But I still unable to capture image from the camera. Now I have the following report from the capture attempt:

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 3 
   Output Stream W = 640 H = 480 
   seconds to Run    = 0 
   Frame Rate = 90.000001 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
nvbuf_utils: dmabuf_fd -1 mapped entry NOT found
nvbuf_utils: Can not get HW buffer from FD... Exiting...
CONSUMER: ERROR OCCURRED
ERROR on bus: by /GstPipeline:capture_native_pipeline/GstBin:cap_bin/GstNvArgusCameraSrc:nvarguscamerasrc0: CANCELLED
debug info:
Argus Error Status
GST_ARGUS: Cleaning up

I also checked the dmesg messages from this period and its repeat only this sequence 3 times before timeout (these debug messages from the power_on function):

[ 71.711294] ov5647 7-0036: ov5647_power_on: power on
[ 71.711552] ov5647 7-0036: ov5647_power_on: going for gpio reset
[ 71.711559] ov5647 7-0036: ov5647_power_on: going for set gpio reset

Based on that I think the device not powered properly to access the SCCB. The current ov5647_power_on function is based on the imx219, I tried to modify it to make it more similar to the ov5693.c and also tried to implement the power up sequence from the ov5647 datasheet, but it looks like it does not have a

pw->pwdn_gpio

Also I find it that there is the first condition in the power_on function:
if (pdata && pdata->power_on)
It’s never True and the program move to the following which is actually the exit:

if (unlikely(!(pw->avdd || pw->iovdd || pw->dvdd)))
		goto skip_power_seqn;
.........

skip_power_seqn:
	if (pw->reset_gpio) {
		dev_info(dev, "%s: In skip_power_seqn, going for gpio reset\n", __func__);
		if (gpio_cansleep(pw->reset_gpio))
			gpio_set_value_cansleep(pw->reset_gpio, 1);
		else
			gpio_set_value(pw->reset_gpio, 1);
	}

It looks like the avdd, iovdd and dvdd all null. I am so confused. This implementation can be used to enumerate and bound the device as i2c. I can also read out via i2c the camera ID but when I try to open the camera it stuck in the power_up as a loop. Can you help me with this, I did not find any documentation about the power rail related coding. And the ov5693.c contains elements which I can not find in the imx219.c.

9

You can ignore those power if your HW don’t need them.
If the i2c command working without problem you can ignore the power function.

Get the trace log to check.

https://elinux.org/Jetson/l4t/Camera_BringUp#Steps_to_enable_more_debug_messages

10

Thank you for your response, I enabled further logging and made the same with imx219. I compared the logs and found quiet similar the two logs for a certain point:

PowerServiceCore:setCameraBW: totalIsoBW:…

After this line Errors appeared in the ov5647 log, I followed these errors and found a previous thread where you suggested to change the “discontinuous_clk in DT”. I changed it to the opposite and now I have different error messages:

SCF: Error Timeout: ISP port 0 timed out! (in src/services/capture/NvIspHw.cpp, function waitIspFrameEnd(), line 492)
SCF: Error InvalidState: Something went wrong with waiting on Isp frame end (in src/services/capture/NvIspHw.cpp, function waitIspFrameEnd(), line 550)
SCF: Error InvalidState:  (propagating from src/common/Utils.cpp, function workerThread(), line 116)
SCF: Error Timeout: ISP Stats timed out! (in src/services/capture/NvIspHw.cpp, function waitIspStatsFinished(), line 608)
SCF: Error Timeout: During capture abort, syncpoint wait timeout waiting for current frame to finish (in src/services/capture/CaptureServiceDevice.cpp, function handleCancelSourceRequests(), line 1034)
CC 103 session 1 processing step 7 in fiber 0x7f34000f70

Is this errors timing related, I started to check the device tree values. But the pixel clock values are from the datasheet. Which parameter can cause this kind of errors?

11

Please confirm the sensor driver by v4l2-ctl for example.

v4l2-ctl --set-fmt-video=width=1920,height=1080,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap
12

Thank you for you fast response,
I tried the suggested line but when I typed in the terminal only hangs out without any output. I tried it with different resolutions too.
I also found a other threads about the topic, one of them was answered by you, but I was not able to use your answer from there. With the v4l2-ctl -all command I got the following which looks correct:

Driver Info (not using libv4l2):
	Driver name   : tegra-video
	Card type     : vi-output, ov5647 7-0036
	Bus info      : platform:54080000.vi:0
	Driver version: 4.9.255
	Capabilities  : 0x84200001
		Video Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps   : 0x04200001
		Video Capture
		Streaming
		Extended Pix Format
Priority: 2
Video input : 0 (Camera 0: ok)
Format Video Capture:
	Width/Height      : 2592/1944
	Pixel Format      : 'RG10'
	Field             : None
	Bytes per Line    : 5184
	Size Image        : 10077696
	Colorspace        : sRGB
	Transfer Function : Default (maps to sRGB)
	YCbCr/HSV Encoding: Default (maps to ITU-R 601)
	Quantization      : Default (maps to Full Range)
	Flags             : 

Camera Controls

                     group_hold 0x009a2003 (bool)   : default=0 value=0 flags=execute-on-write
                    sensor_mode 0x009a2008 (int64)  : min=0 max=0 step=0 default=0 value=0 flags=slider
                           gain 0x009a2009 (int64)  : min=0 max=0 step=0 default=0 value=16 flags=slider
                       exposure 0x009a200a (int64)  : min=0 max=0 step=0 default=0 value=1 flags=slider
                     frame_rate 0x009a200b (int64)  : min=0 max=0 step=0 default=0 value=1500000 flags=slider
                    bypass_mode 0x009a2064 (intmenu): min=0 max=1 default=0 value=0
                override_enable 0x009a2065 (intmenu): min=0 max=1 default=0 value=0
                   height_align 0x009a2066 (int)    : min=1 max=16 step=1 default=1 value=1
                     size_align 0x009a2067 (intmenu): min=0 max=2 default=0 value=0
               write_isp_format 0x009a2068 (bool)   : default=0 value=0
       sensor_signal_properties 0x009a2069 (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
        sensor_image_properties 0x009a206a (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
      sensor_control_properties 0x009a206b (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
              sensor_dv_timings 0x009a206c (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
               low_latency_mode 0x009a206d (bool)   : default=0 value=0
               preferred_stride 0x009a206e (int)    : min=0 max=65535 step=1 default=0 value=0
                   sensor_modes 0x009a2082 (int)    : min=0 max=30 step=1 default=30 value=4 flags=read-only

The hangout of the terminal points a problem with my driver or it can be still a device tree configuration problem?

13

Looks like could be your sensor configure or HW problem.
Get the trace log to check if more clue.

https://elinux.org/Jetson/l4t/Camera_BringUp#Steps_to_enable_more_debug_messages

14

I checked the camera module with a raspberry and with that it works properly, so the HW is OK I think. I also checked the trace log of the ov5647 and the imx219’s log to to have an example of a working one. I also added more debug prints to the sensor driver and also have the nvgstcapture log.

Based on the dmesg logs I can say the driver enters the start_streaming function and also write the required register values without error. The register value pairs are based on the linux repository

If I check the trace files (ov5647 and imx219) I can see almost the same messages

gst-plugin-scan-6128 [000] … 137.629763: tegra_channel_open: vi-output, ov5647 7-0036
gst-plugin-scan-6128 [000] … 137.629785: tegra_channel_close: vi-output, ov5647 7-0036
nvargus-daemon-6152 [001] … 179.039631: tegra_channel_open: vi-output, ov5647 7-0036
nvargus-daemon-6152 [001] … 179.039748: tegra_channel_close: vi-output, ov5647 7-0036
nvargus-daemon-6152 [001] … 179.042057: tegra_channel_open: vi-output, ov5647 7-0036
nvargus-daemon-6152 [001] … 179.042145: tegra_channel_close: vi-output, ov5647 7-0036
nvargus-daemon-6152 [002] … 179.085065: tegra_channel_open: vi-output, ov5647 7-0036
nvargus-daemon-6152 [002] … 179.085086: tegra_channel_close: vi-output, ov5647 7-0036
nvargus-daemon-6152 [002] … 179.085102: tegra_channel_open: vi-output, ov5647 7-0036
nvargus-daemon-6152 [002] … 179.085178: tegra_channel_close: vi-output, ov5647 7-0036
nvargus-daemon-6152 [002] … 179.085236: tegra_channel_open: vi-output, ov5647 7-0036
nvargus-daemon-6152 [002] … 179.085338: tegra_channel_close: vi-output, ov5647 7-0036
nvargus-daemon-6152 [002] … 179.122424: tegra_channel_open: vi-output, ov5647 7-0036
nvargus-daemon-6152 [002] … 179.122445: tegra_channel_close: vi-output, ov5647 7-0036
nvargus-daemon-6152 [003] … 179.250876: camera_common_s_power: status : 0x1
nvargus-daemon-6152 [001] … 179.392078: camera_common_s_power: status : 0x0
CaptureSchedule-6171 [000] … 179.552658: tegra_channel_open: vi-output, ov5647 7-0036
CaptureSchedule-6171 [000] … 179.553351: tegra_channel_set_power: ov5647 7-0036 : 0x1
CaptureSchedule-6171 [000] … 179.553358: camera_common_s_power: status : 0x1
CaptureSchedule-6171 [002] … 179.554743: tegra_channel_set_power: nvcsi–2 : 0x1
CaptureSchedule-6171 [002] … 179.554747: csi_s_power: enable : 0x1
CaptureSchedule-6171 [002] … 179.554750: tegra_channel_set_stream: enable : 0x1
CaptureSchedule-6171 [002] … 179.555121: tegra_channel_set_stream: nvcsi–2 : 0x1
CaptureSchedule-6171 [002] … 179.555124: csi_s_stream: enable : 0x1
CaptureSchedule-6171 [002] … 179.555124: tegra_channel_set_stream: ov5647 7-0036 : 0x1
kworker/1:2-1049 [001] … 189.949376: camera_common_s_power: status : 0x1
kworker/1:2-1049 [001] … 190.056526: camera_common_s_power: status : 0x0

until this line

CaptureSchedule-6171 [002] … 179.555124: tegra_channel_set_stream: ov5647 7-0036 : 0x1

In the case of imx219 it continues more CaptureSchedule related lines, but the ov5647 version continues with these:

kworker/1:2-1049 [001] … 189.949376: camera_common_s_power: status : 0x1
kworker/1:2-1049 [001] … 190.056526: camera_common_s_power: status : 0x0

The nvgstcapture log also almost the same as the imx219 log but ends with the following errors:

SCF: Error InvalidState: 1 buffers still pending during EGLStreamProducer destruction (propagating from src/services/gl/EGLStreamProducer.cpp, function freeBuffers(), line 309)
SCF: Error InvalidState: (propagating from src/services/gl/EGLStreamProducer.cpp, function ~EGLStreamProducer(), line 50)
waitForIdleLocked remaining request 103
SCF: Error Timeout: waitForIdle() timed out (in src/api/Session.cpp, function waitForIdleLocked(), line 928)
(Argus) Error Timeout: (propagating from src/api/CaptureSessionImpl.cpp, function destroy(), line 169)
=== nvgstcapture-1.0[6447]: Connection established (7F3F7FE1D0)=== nvgstcapture-1.0[6447]: CameraProvider initialized (0x7f40000c20)(Argus) Error AlreadyAllocated: Device 0 (of 1) is in use (in src/api/CameraProviderImpl.cpp, function createCaptureSessionInternal(), line 274)

I repeated the test with the v4l2-ctl commands and saved the trace log from this period of time and I got different output. After the initialization there is a lot of repetition of the following line with incremental numbers

tegra_channel_capture_frame: sof:471.318570237

If it means the successful frame capture based on the timestamps between 2 capture there is 0.21 sec which is more like 5fps not 30fps. I repeated the command with the imx219 and it gave back the actual fps values, so I can say something hinders the frame acquiring in the case of ov5647. Sorry for the many copy paste messages I do not know how can I share the most effectively the logs without attaching too much unnecessary information. Can be the problem pixel clock related or other device tree parameter?

15

Get the trace log to confirm if NVCSI receive any data from sensor.

Thanks

16

I double checked the camera reconnected it and now it works. I think I made a mistake when I attached to the CSI connector. I am so grateful for your help. My topic was solved. Now I have live video feed with a dark, almost binary only the light sources can be seen picture my last question would be about this. Can this problem device tree parameter related? Thank you again for your help.

17

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.