Image sensor IMX415 interface with nano

sndp · May 11, 2024, 8:28pm

Hello,
I am working on IMX415 to interface with jetson nano.
after running v4l2-compliance -d /dev/video0 command i am getting below error in dmesg.

[ 1122.600199] imx415 6-001a: Error updating control ranges -22

v4l2-compliance_dmesg.txt (3.4 KB)
v4l2-compliance.txt (2.8 KB)

And, to check video out, i used below command but i am green screen.

` gst-launch-1.0 nvarguscamerasrc sensor_id=0 ! ‘video/x-raw(memory:NVMM),width=3840, height=2160, framerate=15/1’ ! nvvidconv flip-method=0 ! ‘video/x-raw,width=960, height=540’ ! nvvidconv ! nvegltransform ! nveglglessink -e

streamOn_dmesg.txt (58.5 KB)

can you please help to looks into attached logs and guide on this topic

FYI :
media-ctl output : media-ctl.txt (953 Bytes)
device tree : extracted_proc.txt (301.0 KB)

ShaneCCC · May 13, 2024, 3:27am

Confirm the sensor driver by v4l2-ctl first.

https://docs.nvidia.com/jetson/archives/r35.4.1/DeveloperGuide/text/SD/CameraDevelopment/SensorSoftwareDriverProgramming.html#verifying-the-v4l2-sensor-driver

sndp · May 13, 2024, 5:19am

Hello @ShaneCCC,
Thanks for your input.
I tried v4l2-ctl with 3840x2160 resolution and 15 fps.

v4l2-ctl --set-fmt-video=width=3840,height=2160,pixelformat=RG10 --stream-mmap --set-ctrl=sensor_mode=0 --stream-count=100 -d /dev/video0 
<<<<<<<<<<<<<<<<< 15.09 fps
<<<<<<<<<<<<<< 15.00 fps
<<<<<<<<<<<<<<< 15.00 fps
<<<<<<<<<<<<<<< 15.00 fps
<<<<<<<<<<<<<<< 15.00 fps
<<<<<<<<<<<<<<< 15.00 fps
<<<<<<<<<

In dmesg, got below error logs

[ 1025.792687] vi 54080000.vi: tegra_channel_error_status:error 20022 frame 0
[ 1032.861103] video4linux video0: tegra_channel_capture_done: MW_ACK_DONE syncpoint time out!0

detailed logs :
v4l2_ctl_output.txt (1.2 KB)

ShaneCCC · May 13, 2024, 8:30am

Enter below command first then gst-launch-1.0 … command.

export DISPLAY=:0

sndp · May 13, 2024, 8:41am

Hello @ShaneCCC ,
Thanks for you input.
I tried export DISPLAY=:0 then v4l2-ctl , but still same issue, please find attached logs

user@user-desktop:~$ export DISPLAY=:0
user@user-desktop:~$ v4l2-ctl --set-fmt-video=width=3840,height=2160,pixelformat=RG10 --stream-mmap --set-ctrl=sensor_mode=0 --stream-count=100 -d /dev/video0 
<<<<<<<<<<<<<<<<< 15.09 fps
<<<<<<<<<<<<<<< 15.04 fps
<<<<<<<<<<<<<<< 15.03 fps
<<<<<<<<<<<<<<< 15.02 fps
<<<<<<<<<<<<<<< 15.01 fps
<<<<<<<<<<<<<<< 15.01 fps
<<<<<<<<

v4l2_ctl_output_01.txt (1.2 KB)

ShaneCCC · May 13, 2024, 9:26am

The v4l2-ctl looks fine if only one MW_ACK_DONE in the beginning.
Please check the gst-launch-1.0 nvarguscamerasrc …

sndp · May 13, 2024, 10:26am

Hello @ShaneCCC,
Thank you for your quick response.

gst-launch-1.0 nvarguscamerasrc

I already checked this command and in comment#1, I given
streamOn_dmesg.txt (58.5 KB) logs.

For your reference, I tried below command :

gst-launch-1.0 nvarguscamerasrc sensor_id=0 ! ‘video/x-raw(memory:NVMM),width=3840, height=2160, framerate=15/1’ ! nvvidconv flip-method=0 ! ‘video/x-raw,width=960, height=540’ ! nvvidconv ! nvegltransform ! nveglglessink -e

After execution , I got below error on same terminal,

nvbuf_utils: dmabuf_fd -1 mapped entry NOT found
nvbuf_utils: Can not get HW buffer from FD… Exiting…
CONSUMER: ERROR OCCURRED
ERROR: from element /GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc0: CANCELLED
Additional debug info:
Argus Error Status
EOS on shutdown enabled – waiting for EOS after Error
Waiting for EOS…

And , In dmesg, got some fence timeout error prints. after this many dmesg traces (please check attached log file for more details)

[ 2285.096698] fence timeout on [ffffffc0f3e23480] after 1500ms
[ 2285.096705] name=[nvhost_sync:17], current value=3 waiting value=4

Also , In output of same command , below parameters are shown, specifically can please confirm Duration, Exposure Range min-max values here ?

GST_ARGUS: 3840 x 2160 FR = 15.000000 fps Duration = 66666668 ; Analog Gain range min 0.300000, max 72.000000; Exposure Range min 238000, max 66706000;

How we can solve MW_ACK_DONE syncpoint time out!0 issue ? or can we ignore it ?

ShaneCCC · May 13, 2024, 11:59am

Looks like unable to capture frame data from the sensor.
Please modify the sensor driver to make the control function like xxx_set_exposure/… to dummy function.

Thanks

sndp · May 13, 2024, 12:18pm

Hello @ShaneCCC,
Thanks for your input.
In control operation (struct tegracam_ctrl_ops) ,I have already one callback function for exposure.

Please check below snapshot.

In this callback function i am updating shutter register values.

apart from above mentioned details, are saying to make one more dummy function for set_exposure ?

ShaneCCC · May 14, 2024, 1:26am

Yes, the exposure REG make cause the capture failed.
And it’s better to make others to dummy function to clarify the problem.

Thanks

sndp · May 14, 2024, 2:50am

Hello @ShaneCCC,
In imx415_set_exposure function, I just added return 0 and tested but still I can see same issue :-( .

static int imx415_set_exposure(struct tegracam_device *tc_dev, s64 val)
{
  return 0;
}

can please help to confirm on , below parameters, specifically can please confirm Duration, Exposure Range min-max values here ?
if these values are in micro second then it looks issue for me ?

sndp · May 17, 2024, 12:37pm

Hello @ShaneCCC ,

Also, Can you please help to suggest any solution for below error ?

sndp · May 19, 2024, 6:08pm

Hello @ShaneCCC
Using below command , I captured one frame (in .raw file) and then converted into jpeg format.

v4l2-ctl -d /dev/video0 --set-fmt-video=width=3840,height=2160 --set-ctrl bypass_mode=0 --set-ctrl gain=40 --set-ctrl exposure=1925 --set-ctrl sensor_mode=0 --stream-mmap --stream-count=1 --stream-to=/home/user/Pictures/frame.raw

convert -size 3840x2160 -depth 10 uyvy:/home/user/Pictures/frame.raw /home/user/Pictures/outImage.jp

It is green coloured image but at least i can see clearly objects in it.

In dmesg, I enabled some extra debug logs , please help to check below logs. still having MW_ACK_DONE issue .

 [ 1081.708375] vi 54080000.vi: cil_settingtime was autocalculated
 [ 1081.708382] vi 54080000.vi: csi clock settle time: 13, cil settle time: 10
 
 [ 1082.069753] imx415_start_streaming(975) : Streaming started >>>>> 
 [ 1082.528283] video4linux video0: tegra_channel_capture_done: MW_ACK_DONE syncpoint time out!0
 [ 1082.536810] vi 54080000.vi: TEGRA_CSI_PIXEL_PARSER_STATUS 0x00000000
 [ 1082.536814] vi 54080000.vi: TEGRA_CSI_CIL_STATUS 0x00000000
 [ 1082.536818] vi 54080000.vi: TEGRA_CSI_CILX_STATUS 0x00000000
 [ 1082.536857] vi 54080000.vi: cil_settingtime was autocalculated
 [ 1082.536862] vi 54080000.vi: csi clock settle time: 13, cil settle time: 10

I tried below command for live streaming and it got failed with same error.

can you please help to looks into it and suggest some solution ?

ShaneCCC · May 20, 2024, 3:39am

Try have some delay after sensor stream on REG programed.

Thanks

sndp · May 20, 2024, 6:03pm

Hello @ShaneCCC,
Thank you for your input

I tried adding a delay, but it didn’t help.

This time, I observed new logs Syncpoint already enabled at capture done!0 and TEGRA_VI_CSI_ERROR_STATUS = 0x0000000a

**[ 3805.240400] video4linux video0: Syncpoint already enabled at capture done!0**
[ 3805.444188] video4linux video0: tegra_channel_capture_done: MW_ACK_DONE syncpoint time out!0
[ 3805.452655] video4linux video0: TEGRA_VI_CSI_ERROR_STATUS 0x0000000a
[ 3805.452661] vi 54080000.vi: TEGRA_CSI_PIXEL_PARSER_STATUS 0x00000000
[ 3805.452666] vi 54080000.vi: TEGRA_CSI_CIL_STATUS 0x00000000
[ 3805.452671] vi 54080000.vi: TEGRA_CSI_CILX_STATUS 0x00000000

Can you please help me understand the meaning of these logs?

ShaneCCC · May 21, 2024, 6:58am

Please check the daemon log. Currently the MW_ACK_DONE only the v4l2-ctl path that didn’t help on the for the nvarguscamerasrc.

Also, could you confirm the sensor output is continuous clocks or discontinuous clocks.

Below to get the daemon log.

sudo service nvargus-daemon stop
sudo su
/usr/sbin/nvargus-daemon
 
launch camera from another console

sndp · May 21, 2024, 3:37pm

Hello @ShaneCCC
I appreciate your input.

In device tree discontinuous clock is enabled

discontinuous_clk = “yes”;

I performed three rounds of testing, please find below daemon logs.

TestLogs.zip (61.9 KB)

ShaneCCC · May 22, 2024, 2:58am

I mean the sensor output mode instead of device tree configuration for the clocks.

And add set_mode_delay_ms to device to try.

sndp · May 22, 2024, 12:11pm

Hello @ShaneCCC,
Thanks for your input.

I tried set_mode_delay_ms with 1000,1500, 3000 but it didn’t help

sndp · June 3, 2024, 5:53pm

Hello @ShaneCCC

The sensor is operating in master mode and has a separate external crystal for its clock.
I NOT found any sensor register configuration for continuous clocks or discontinuous clocks.

1.One observation, as per mode configuration, on stream start, sensor starts sending CSI data, Vsync , Hsync but CSI clock is NOT showing in oscilloscope.
2. In master mode, can we use Vsync , Hsync signal for frame sync and to resolve MW_ACK_DONE error ?

Thanks