On a Xavier NX, I am trying to receive 2 MIPI CSI2 streams (without i2c) from a Xilinx UltraScale+ and experience a strange behavior and I am running out of ideas.
I have the following configuration:
-
Stream 1 is a 4x MIPI (ignored for now)
- FPGA is currently not streaming yet.
- mapped to /dev/video0 -
Stream 2 is a 2x MIPI CSI
- 640x480@50fps
- YUV422
- mapped to /dev/video1
I have created a custom driver (stubbing all I2C communication) and modified the dtsi file(s) for both streams get the driver and tegra camera platform loaded. So far so good, I am getting the /dev/video0 and /dev/video1. Since stream1 is not yet available, the testing is done on stream 2 only (/dev/video1). However, I am not able to receive any frames from the FPGA (see the log output below).
When the FPGA is modified to transmit a test pattern, there are some special cases when frames are received:
- all horizontal data set to 0’s → OKAY
Lane 0: <data id> <wc msb> <0><0><0>...<crc-lsb>
Lane 1: <wc lsb > < ecc > <0><0><0>...<crc-msb>
- all horizontal data set to 0xff’s → OKAY
Lane 0: <data id> <wc msb> <ff><ff><ff>...<crc-lsb>
Lane 1: <wc lsb > < ecc > <ff><ff><ff>...<crc-msb>
- changing the horizontal data, but keeping them the same in the even and odd lane → OKAY
Lane 0: <data id> <wc msb> <1><2><3>...<crc-lsb>
Lane 1: <wc lsb > < ecc > <1><2><3>...<crc-msb>
But, having the bytes different on even and odd lane → FAIL
4)
Lane 0: <data id> <wc msb> <1><3><5>...<crc-lsb>
Lane 1: <wc lsb > < ecc > <2><4><6>...<crc-msb>
There is communication, since I can see the frame numbers and, in some special cases, receive valid frames (validated the raw stream data output of v4l2-ctl with respect to the data send by the FPGA).
Question 1. Is there some explanation for this behavior?
Question 2. What is the meaning of the flags and err_data (flags: 96, err_data 4194403)?
Question 3. Is the trace data providing any info on what is going on here?
LOG OUTPUTS:
v4l2-ctl --device /dev/video1 --all
Driver Info (not using libv4l2):
Driver name : tegra-video
Card type : vi-output, mydriver 10-0020
Bus info : platform:15c10000.vi:3
Driver version: 4.9.201
Capabilities : 0x84200001
Video Capture
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04200001
Video Capture
Streaming
Extended Pix Format
Priority: 2
Video input : 0 (Camera 3: ok)
Format Video Capture:
Width/Height : 640/480
Pixel Format : 'YUYV'
Field : None
Bytes per Line : 1280
Size Image : 614400
Colorspace : sRGB
Transfer Function : Default (maps to sRGB)
YCbCr/HSV Encoding: Default (maps to ITU-R 601)
Quantization : Default (maps to Limited Range)
Flags :
Camera Controls
group_hold 0x009a2003 (bool) : default=0 value=0 flags=execute-on-write
gain 0x009a2009 (int64) : min=0 max=0 step=0 default=0 value=10 flags=slider
exposure 0x009a200a (int64) : min=0 max=0 step=0 default=0 value=13 flags=slider
frame_rate 0x009a200b (int64) : min=0 max=0 step=0 default=0 value=50000000 flags=slider
exposure_short 0x009a200c (int64) : min=0 max=0 step=0 default=0 value=13 flags=slider
sensor_configuration 0x009a2032 (u32) : min=0 max=0 step=0 default=0 flags=read-only, volatile, has-payload
sensor_mode_i2c_packet 0x009a2033 (u32) : min=0 max=0 step=0 default=0 flags=read-only, volatile, has-payload
sensor_control_i2c_packet 0x009a2034 (u32) : min=0 max=0 step=0 default=0 flags=read-only, volatile, has-payload
bypass_mode 0x009a2064 (intmenu): min=0 max=1 default=0 value=0
override_enable 0x009a2065 (intmenu): min=0 max=1 default=0 value=0
height_align 0x009a2066 (int) : min=1 max=16 step=1 default=1 value=1
size_align 0x009a2067 (intmenu): min=0 max=2 default=0 value=0
write_isp_format 0x009a2068 (int) : min=1 max=1 step=1 default=1 value=1
sensor_signal_properties 0x009a2069 (u32) : min=0 max=0 step=0 default=0 flags=read-only, has-payload
sensor_image_properties 0x009a206a (u32) : min=0 max=0 step=0 default=0 flags=read-only, has-payload
sensor_control_properties 0x009a206b (u32) : min=0 max=0 step=0 default=0 flags=read-only, has-payload
sensor_dv_timings 0x009a206c (u32) : min=0 max=0 step=0 default=0 flags=read-only, has-payload
low_latency_mode 0x009a206d (bool) : default=0 value=0
preferred_stride 0x009a206e (int) : min=0 max=65535 step=1 default=0 value=0
sensor_modes 0x009a2082 (int) : min=0 max=30 step=1 default=30 value=1 flags=read-only
media-ctl -p
Media controller API version 0.1.0
Media device information
------------------------
driver tegra194-vi5
model NVIDIA Tegra Video Input Device
serial
bus info
hw revision 0x3
driver version 0.0.0
Device topology
- entity 1: mydriver 9-0010 (1 pad, 1 link)
type V4L2 subdev subtype Sensor flags 0
device node name /dev/v4l-subdev0
pad0: Source
[fmt:SRGGB10_1X10/640x480 field:none colorspace:srgb]
-> "15a00000.nvcsi--2":0 [ENABLED]
- entity 3: 15a00000.nvcsi--2 (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev1
pad0: Sink
<- "mydriver 9-0010":0 [ENABLED]
pad1: Source
-> "vi-output, mydriver 9-0010":0 [ENABLED]
- entity 6: vi-output, mydriver 9-0010 (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video0
pad0: Sink
<- "15a00000.nvcsi--2":1 [ENABLED]
- entity 18: mydriver 10-0020 (1 pad, 1 link)
type V4L2 subdev subtype Sensor flags 0
device node name /dev/v4l-subdev2
pad0: Source
[fmt:YUYV8_1X16/640x480 field:none colorspace:srgb]
-> "15a00000.nvcsi--1":0 [ENABLED]
- entity 20: 15a00000.nvcsi--1 (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev3
pad0: Sink
<- "mydriver 10-0020":0 [ENABLED]
pad1: Source
-> "vi-output, mydriver 10-0020":0 [ENABLED]
- entity 23: vi-output, mydriver 10-0020 (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video1
pad0: Sink
<- "15a00000.nvcsi--1":1 [ENABLED]
v4l2-ctl -d /dev/video1 --set-fmt-video=width=640,height=480,pixelformat=YUYV --set-ctrl bypass_mode=0 --stream-mmap --stream-count=30 --stream-to=csi.raw --verbose
VIDIOC_QUERYCAP: ok
VIDIOC_S_EXT_CTRLS: ok
VIDIOC_G_FMT: ok
VIDIOC_S_FMT: ok
Format Video Capture:
Width/Height : 640/480
Pixel Format : 'YUYV'
Field : None
Bytes per Line : 1280
Size Image : 614400
Colorspace : sRGB
Transfer Function : Default (maps to sRGB)
YCbCr/HSV Encoding: Default (maps to ITU-R 601)
Quantization : Default (maps to Limited Range)
Flags :
VIDIOC_REQBUFS: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_STREAMON: ok
<NOTE: MIPI Stream is kept in reset>
[ 871.509216] tegra194-vi5 15c10000.vi: no reply from camera processor
[ 871.509616] tegra194-vi5 15c10000.vi: uncorr_err: request timed out after 2500 ms
[ 871.510022] tegra194-vi5 15c10000.vi: err_rec: attempting to reset the capture channel
Index : 0
Type : Video Capture
Flags : mapped, error
Field : None
Sequence : 0
Length : 614400
Bytesused: 614400
Timestamp: 0.000000s (Monotonic, End-of-Frame)
Index : 0
Type : Video Capture
Flags : mapped, error
Field : None
Sequence : 0
Length : 614400
Bytesused: 614400
Timestamp: 0.000000s (Monotonic, End-of-Frame)
Index : 1
Type : Video Capture
Flags : mapped, error
Field : None
Sequence : 1
Length : 614400
Bytesused: 614400
Timestamp: 0.000000s (Monotonic, End-of-Frame)
Index : 2
Type : Video Capture
Flags : mapped, error
Field : None
Sequence : 2
Length : 614400
Bytesused: 614400
Timestamp: 0.000000s (Monotonic, End-of-Frame)
Index : 3
Type : Video Capture
Flags : mapped, error
Field : None
Sequence : 3
Length : 614400
Bytesused: 614400
Timestamp: 0.000000s (Monotonic, End-of-Frame)[ 871.528973] tegra194-vi5 15c10000.vi: err_rec: successfully reset the capture channel
[ 874.069777] tegra194-vi5 15c10000.vi: no reply from camera processor
[ 874.070141] tegra194-vi5 15c10000.vi: uncorr_err: request timed out after 2500 ms
Index : 0
Type : Video Capture
Flags : mapped, error
Field : None
Sequence : 0
Length : 614400
Bytesused: 614400
Timestamp: 0.000000s (Monotonic, End-of-Frame)
[ 874.078231] tegra194-vi5 15c10000.vi: err_rec: attempting to reset the capture channel
Index : 0
Type : Video Capture
Flags : mapped, error
Field : None
Sequence : 0
Length : 614400
Bytesused: 614400
Timestamp: 0.000000s (Monotonic, End-of-Frame)
[ 874.085152] tegra194-vi5 15c10000.vi: err_rec: successfully reset the capture channel
Index : 1
Type : Video Capture
Flags : mapped, error
Field : None
Sequence : 1
Length : 614400
Bytesused: 614400
Timestamp: 0.000000s (Monotonic, End-of-Frame)
Index : 2
Type : Video Capture
Flags : mapped, error
Field : None
Sequence : 2
Length : 614400
Bytesused: 614400
Timestamp: 0.000000s (Monotonic, End-of-Frame)
Index : 3
Type : Video Capture
Flags : mapped, error
Field : None
Sequence : 3
Length : 614400
Bytesused: 614400
Timestamp: 0.000000s (Monotonic, End-of-Frame)
<NOTE: MIPI Stream is starting>
[ 131.920784] tegra194-vi5 15c10000.vi: corr_err: discarding frame 0, flags: 96, err_data 4194403
[ 131.974594] tegra194-vi5 15c10000.vi: corr_err: discarding frame 2, flags: 96, err_data 4194403
[ 132.029881] tegra194-vi5 15c10000.vi: corr_err: discarding frame 3, flags: 96, err_data 4194403
[ 132.083178] tegra194-vi5 15c10000.vi: corr_err: discarding frame 4, flags: 96, err_data 4194403
[ 132.137318] tegra194-vi5 15c10000.vi: corr_err: discarding frame 1, flags: 96, err_data 4194403
[ 132.191064] tegra194-vi5 15c10000.vi: corr_err: discarding frame 2, flags: 96, err_data 4194403
[ 132.245743] tegra194-vi5 15c10000.vi: corr_err: discarding frame 3, flags: 96, err_data 4194403
[ 132.299048] tegra194-vi5 15c10000.vi: corr_err: discarding frame 4, flags: 96, err_data 4194403
[ 132.353161] tegra194-vi5 15c10000.vi: corr_err: discarding frame 1, flags: 96, err_data 4194403
[ 132.406808] tegra194-vi5 15c10000.vi: corr_err: discarding frame 2, flags: 96, err_data 4194403
[ 132.461321] tegra194-vi5 15c10000.vi: corr_err: discarding frame 3, flags: 96, err_data 4194403
[ 132.515006] tegra194-vi5 15c10000.vi: corr_err: discarding frame 4, flags: 96, err_data 4194403
[ 132.568698] tegra194-vi5 15c10000.vi: corr_err: discarding frame 1, flags: 96, err_data 4194403
[ 132.623083] tegra194-vi5 15c10000.vi: corr_err: discarding frame 2, flags: 96, err_data 4194403
[ 132.677360] tegra194-vi5 15c10000.vi: corr_err: discarding frame 3, flags: 96, err_data 4194403
[ 132.730892] tegra194-vi5 15c10000.vi: corr_err: discarding frame 4, flags: 96, err_data 4194403
[ 132.785274] tegra194-vi5 15c10000.vi: corr_err: discarding frame 1, flags: 96, err_data 4194403
[ 132.838906] tegra194-vi5 15c10000.vi: corr_err: discarding frame 2, flags: 96, err_data 4194403
[ 132.893111] tegra194-vi5 15c10000.vi: corr_err: discarding frame 3, flags: 96, err_data 4194403
[ 132.947328] tegra194-vi5 15c10000.vi: corr_err: discarding frame 4, flags: 96, err_data 4194403
[ 133.001059] tegra194-vi5 15c10000.vi: corr_err: discarding frame 1, flags: 96, err_data 4194403
[ 133.054948] tegra194-vi5 15c10000.vi: corr_err: discarding frame 2, flags: 96, err_data 4194403
[ 133.109064] tegra194-vi5 15c10000.vi: corr_err: discarding frame 3, flags: 96, err_data 4194403
[ 133.162919] tegra194-vi5 15c10000.vi: corr_err: discarding frame 4, flags: 96, err_data 4194403
[ 133.217560] tegra194-vi5 15c10000.vi: corr_err: discarding frame 1, flags: 96, err_data 4194403
[ 133.271160] tegra194-vi5 15c10000.vi: corr_err: discarding frame 2, flags: 96, err_data 4194403
[ 133.324883] tegra194-vi5 15c10000.vi: corr_err: discarding frame 3, flags: 96, err_data 4194403
[ 133.379205] tegra194-vi5 15c10000.vi: corr_err: discarding frame 4, flags: 96, err_data 4194403
[ 133.432911] tegra194-vi5 15c10000.vi: corr_err: discarding frame 1, flags: 96, err_data 4194403
[ 133.487023] tegra194-vi5 15c10000.vi: corr_err: discarding frame 2, flags: 96, err_data 4194403
[ 133.541722] tegra194-vi5 15c10000.vi: corr_err: discarding frame 3, flags: 96, err_data 4194403
[ 133.595144] tegra194-vi5 15c10000.vi: corr_err: discarding frame 4, flags: 96, err_data 4194403
[ 133.649627] tegra194-vi5 15c10000.vi: corr_err: discarding frame 1, flags: 96, err_data 4194403
<CTRL-C>
trace output:
kworker/0:0-4 [000] .... 124.497361: rtcpu_nvcsi_intr: tstamp:4210369679 class:GLOBAL type:STREAM_NOVC phy:0 cil:0 st:3 vc:0 status:0x00000001
kworker/0:0-4 [000] .... 124.497377: rtcpu_nvcsi_intr: tstamp:4210369679 class:CORRECTABLE_ERR type:STREAM_NOVC phy:0 cil:0 st:3 vc:0 status:0x00000001
kworker/0:0-4 [000] .... 124.497390: rtcpu_nvcsi_intr: tstamp:4210372743 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:3 vc:0 status:0x00000004
kworker/0:0-4 [000] .... 124.497402: rtcpu_nvcsi_intr: tstamp:4210372743 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:3 vc:0 status:0x00000004
kworker/0:0-4 [000] .... 124.497437: rtcpu_nvcsi_intr: tstamp:4210374565 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:3 vc:0 status:0x00000004
kworker/0:0-4 [000] .... 124.497449: rtcpu_nvcsi_intr: tstamp:4210374565 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:3 vc:0 status:0x00000004
kworker/0:0-4 [000] .... 124.497461: rtcpu_nvcsi_intr: tstamp:4210377264 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:3 vc:0 status:0x00000004
kworker/0:0-4 [000] .... 124.497473: rtcpu_nvcsi_intr: tstamp:4210377264 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:3 vc:0 status:0x00000004
kworker/0:0-4 [000] .... 124.497485: rtcpu_nvcsi_intr: tstamp:4210379963 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:3 vc:0 status:0x00000004
kworker/0:0-4 [000] .... 124.497497: rtcpu_nvcsi_intr: tstamp:4210379963 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:3 vc:0 status:0x00000004
kworker/0:0-4 [000] .... 124.497509: rtcpu_nvcsi_intr: tstamp:4210382663 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:3 vc:0 status:0x00000004
kworker/0:0-4 [000] .... 124.497521: rtcpu_nvcsi_intr: tstamp:4210382663 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:3 vc:0 status:0x00000004
kworker/0:0-4 [000] .... 124.497533: rtcpu_nvcsi_intr: tstamp:4210385365 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:3 vc:0 status:0x00000004
kworker/0:0-4 [000] .... 124.497545: rtcpu_nvcsi_intr: tstamp:4210385365 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:3 vc:0 status:0x00000004
kworker/0:0-4 [000] .... 124.497557: rtcpu_nvcsi_intr: tstamp:4210388062 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:3 vc:0 status:0x00000004
kworker/0:0-4 [000] .... 124.497570: rtcpu_nvcsi_intr: tstamp:4210388062 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:3 vc:0 status:0x00000004
kworker/0:0-4 [000] .... 124.497582: rtcpu_nvcsi_intr: tstamp:4210390762 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:3 vc:0 status:0x00000004
kworker/0:0-4 [000] .... 124.497594: rtcpu_nvcsi_intr: tstamp:4210390762 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:3 vc:0 status:0x00000004
kworker/0:0-4 [000] .... 124.497606: rtcpu_nvcsi_intr: tstamp:4210393464 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:3 vc:0 status:0x00000004
kworker/0:0-4 [000] .... 124.497618: rtcpu_nvcsi_intr: tstamp:4210393464 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:3 vc:0 status:0x00000004
kworker/0:0-4 [000] .... 124.497630: rtcpu_nvcsi_intr: tstamp:4210396160 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:3 vc:0 status:0x00000004
kworker/0:0-4 [000] .... 124.497642: rtcpu_nvcsi_intr: tstamp:4210396160 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:3 vc:0 status:0x00000004
<...repeated many times...>
kworker/0:0-4 [000] .... 124.499632: rtcpu_vinotify_event: tstamp:4210559809 tag:FS channel:0x00 frame:1 vi_tstamp:4210369770 data:0x00000013
kworker/0:0-4 [000] .... 124.499647: rtcpu_vinotify_event: tstamp:4210559950 tag:ATOMP_FS channel:0x00 frame:1 vi_tstamp:4210369773 data:0x00000000
kworker/0:0-4 [000] .... 124.499660: rtcpu_vinotify_event: tstamp:4210560115 tag:CHANSEL_PXL_SOF channel:0x23 frame:1 vi_tstamp:4210369970 data:0x00000001
kworker/0:0-4 [000] .... 124.499672: rtcpu_vinotify_event: tstamp:4210560258 tag:RESERVED_19 channel:0x23 frame:1 vi_tstamp:5882830784 data:0x08020001
kworker/0:0-4 [000] .... 124.499684: rtcpu_nvcsi_intr: tstamp:4210560863 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:3 vc:0 status:0x00000004
kworker/0:0-4 [000] .... 124.499696: rtcpu_nvcsi_intr: tstamp:4210560863 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:3 vc:0 status:0x00000004
kworker/0:0-4 [000] .... 124.499708: rtcpu_nvcsi_intr: tstamp:4210563562 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:3 vc:0 status:0x00000004
kworker/0:0-4 [000] .... 124.499720: rtcpu_nvcsi_intr: tstamp:4210563562 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:3 vc:0 status:0x00000004
kworker/0:0-4 [000] .... 124.499732: rtcpu_nvcsi_intr: tstamp:4210566263 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:3 vc:0 status:0x00000004
<...repeated many times...>
vi-output, cere-7464 [000] .... 124.512567: tegra_channel_capture_setup: vnc_id 0 W 640 H 480 fmt 10
kworker/0:0-4 [000] .... 124.553038: rtcpu_nvcsi_intr: tstamp:4210587863 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:3 vc:0 status:0x00000004
kworker/0:0-4 [000] .... 124.553070: rtcpu_nvcsi_intr: tstamp:4210590562 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:3 vc:0 status:0x00000004
kworker/0:0-4 [000] .... 124.553086: rtcpu_nvcsi_intr: tstamp:4210590562 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:3 vc:0 status:0x00000004
kworker/0:0-4 [000] .... 124.553099: rtcpu_nvcsi_intr: tstamp:4210593262 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:3 vc:0 status:0x00000004
kworker/0:0-4 [000] .... 124.553111: rtcpu_nvcsi_intr: tstamp:4210593262 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:3 vc:0 status:0x00000004
kworker/0:0-4 [000] .... 124.553124: rtcpu_nvcsi_intr: tstamp:4210595959 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:3 vc:0 status:0x00000004