Flashing green screen using custom carrier

Autonomous Machines Jetson & Embedded Systems Jetson TX1
1

Hello,

I’m building custom carrier with interface to Sony Block cameras.
FPGA is outputting UYVY data stream at 1920x1080@29.97.

I have managed to get video output from camera https://youtu.be/uy282qG25Hw
but it has flashing green screen and framerate reported by vlc is 26fps.

gst pipeline:

gst-launch-1.0 -v v4l2src device=/dev/video0 ! 'video/x-raw, format=(string)UYVY, width=(int)1920, height=(int)1080, framerate=(fraction)30/1' ! nvvidconv ! 'video/x-raw(memory:NVMM), format=(string)NV12' ! omxh264enc ! qtmux ! filesink location=test.mp4  -ev

Output from console:

[  731.288853] kmtifpga 6-0036: camera_common_mclk_enable: enable MCLK with 24000000 Hz
[  731.302988] kmtifpga 6-0036: camera_common_dpd_disable: csi 0
[  731.315017] kmtifpga 6-0036: camera_common_dpd_disable: csi 1
[  731.327043] kmtifpga 6-0036: ov5693_power_on: power on
[  731.364918] kmtifpga 6-0036: camera_common_try_fmt: size 800 x 480
[  731.377440] kmtifpga 6-0036: camera_common_try_fmt: size 800 x 480
[  731.391211] kmtifpga 6-0036: camera_common_try_fmt: size 800 x 480
[  731.403743] kmtifpga 6-0036: camera_common_try_fmt: size 1920 x 1080
[  731.416471] kmtifpga 6-0036: camera_common_try_fmt: size 1920 x 1080
[  731.429093] kmtifpga 6-0036: camera_common_try_fmt: size 1920 x 1080
[  731.441550] kmtifpga 6-0036: camera_common_try_fmt: size 1 x 1
[  731.453420] kmtifpga 6-0036: camera_common_try_fmt: invalid resolution supplied to set mode 1920 1080
[  731.468692] kmtifpga 6-0036: camera_common_try_fmt: size 32768 x 32768
[  731.481244] kmtifpga 6-0036: camera_common_try_fmt: invalid resolution supplied to set mode 1920 1080
[  731.496652] kmtifpga 6-0036: camera_common_try_fmt: size 1920 x 1080
[  731.509086] kmtifpga 6-0036: camera_common_try_fmt: size 1920 x 1080
[  731.521565] kmtifpga 6-0036: camera_common_try_fmt: size 1920 x 1080
[  731.533899] kmtifpga 6-0036: camera_common_try_fmt: size 800 x 480
[  731.546008] kmtifpga 6-0036: camera_common_try_fmt: size 800 x 480
[  731.558217] kmtifpga 6-0036: camera_common_try_fmt: size 800 x 480
[  731.570391] kmtifpga 6-0036: camera_common_try_fmt: size 800 x 480
[  731.582452] kmtifpga 6-0036: camera_common_try_fmt: size 800 x 480
[  731.594466] kmtifpga 6-0036: camera_common_try_fmt: size 1920 x 1080
[  731.606754] kmtifpga 6-0036: camera_common_try_fmt: size 1920 x 1080
[  731.619034] kmtifpga 6-0036: camera_common_try_fmt: size 1920 x 1080
[  731.631223] kmtifpga 6-0036: camera_common_try_fmt: size 1920 x 1080
[  731.643399] kmtifpga 6-0036: camera_common_try_fmt: size 1920 x 1080
[  731.655641] kmtifpga 6-0036: camera_common_try_fmt: size 800 x 480
[  731.667562] kmtifpga 6-0036: camera_common_try_fmt: size 800 x 480
[  731.679525] kmtifpga 6-0036: camera_common_try_fmt: size 800 x 480
[  731.691299] kmtifpga 6-0036: camera_common_try_fmt: size 800 x 480
[  731.703011] kmtifpga 6-0036: camera_common_try_fmt: size 800 x 480
[  731.714781] kmtifpga 6-0036: camera_common_try_fmt: size 1920 x 1080
[  731.726689] kmtifpga 6-0036: camera_common_try_fmt: size 1920 x 1080
[  731.738642] kmtifpga 6-0036: camera_common_try_fmt: size 1920 x 1080
[  731.750572] kmtifpga 6-0036: camera_common_try_fmt: size 1920 x 1080
[  731.762465] kmtifpga 6-0036: camera_common_try_fmt: size 1920 x 1080
[  731.774442] kmtifpga 6-0036: camera_common_try_fmt: size 800 x 480
[  731.786190] kmtifpga 6-0036: camera_common_try_fmt: size 800 x 480
[  731.797959] kmtifpga 6-0036: camera_common_try_fmt: size 800 x 480
[  731.809715] kmtifpga 6-0036: camera_common_try_fmt: size 800 x 480
[  731.821545] kmtifpga 6-0036: camera_common_try_fmt: size 800 x 480
[  731.833254] kmtifpga 6-0036: camera_common_try_fmt: size 1920 x 1080
[  731.845221] kmtifpga 6-0036: camera_common_try_fmt: size 1920 x 1080
[  731.857271] kmtifpga 6-0036: camera_common_try_fmt: size 1920 x 1080
[  731.869239] kmtifpga 6-0036: camera_common_try_fmt: size 1920 x 1080
[  731.881160] kmtifpga 6-0036: camera_common_try_fmt: size 1920 x 1080
[  731.893093] kmtifpga 6-0036: camera_common_try_fmt: size 800 x 480
[  731.904911] kmtifpga 6-0036: camera_common_try_fmt: size 800 x 480
[  731.916765] kmtifpga 6-0036: camera_common_try_fmt: size 800 x 480
[  731.928500] kmtifpga 6-0036: camera_common_try_fmt: size 800 x 480
[  731.940303] kmtifpga 6-0036: camera_common_try_fmt: size 800 x 480
[  731.952035] kmtifpga 6-0036: camera_common_try_fmt: size 1920 x 1080
[  731.963943] kmtifpga 6-0036: camera_common_try_fmt: size 1920 x 1080
[  731.975995] kmtifpga 6-0036: camera_common_try_fmt: size 1920 x 1080
[  731.987879] kmtifpga 6-0036: camera_common_try_fmt: size 1920 x 1080
[  731.999843] kmtifpga 6-0036: camera_common_try_fmt: size 1920 x 1080
[  732.012548] kmtifpga 6-0036: camera_common_try_fmt: size 1920 x 1080
[  732.024484] kmtifpga 6-0036: camera_common_try_fmt: size 1920 x 1080
[  732.036526] kmtifpga 6-0036: camera_common_s_fmt(8207) size 1920 x 1080
[  732.048706] kmtifpga 6-0036: camera_common_try_fmt: size 1920 x 1080
[  732.102127] kmtifpga 6-0036: camera_common_g_fmt++
[  732.112664] misc tegra_camera_ctrl: vi_v4l2_update_isobw:Set iso bw 143078 kbyteps at 17885 KHz
[  732.127425] misc tegra_camera_ctrl: tegra_camera_isomgr_request++ bw=143078, lt=4
[  732.140950] misc tegra_camera_ctrl: tegra_camera_isomgr_request: tegra_camera isomgr latency is 4 usec
[  732.156669] vi 54080000.vi: Calibrate csi port 0
[  732.167599] kmtifpga 6-0036: ov5693_s_stream++
[  732.178072] kmtifpga 6-0036: ov5693_s_stream--
[  734.392148] kmtifpga 6-0036: ov5693_s_stream++
[  734.402739] misc tegra_camera_ctrl: vi_v4l2_update_isobw:Set iso bw 0 kbyteps at 0 KHz
[  734.417474] misc tegra_camera_ctrl: tegra_camera_isomgr_request++ bw=0, lt=4
[  734.431635] misc tegra_camera_ctrl: tegra_camera_isomgr_request: tegra_camera isomgr latency is 4 usec
[  734.447432] kmtifpga 6-0036: camera_common_g_fmt++
[  734.458802] misc tegra_camera_ctrl: vi_v4l2_update_isobw:Set iso bw 143078 kbyteps at 17885 KHz
[  734.474180] misc tegra_camera_ctrl: tegra_camera_isomgr_request++ bw=143078, lt=4
[  734.488430] misc tegra_camera_ctrl: tegra_camera_isomgr_request: tegra_camera isomgr latency is 4 usec
[  734.505185] vi 54080000.vi: Calibrate csi port 0
[  734.516806] kmtifpga 6-0036: ov5693_s_stream++
[  734.529483] kmtifpga 6-0036: ov5693_s_stream--
[  734.584521] video4linux video0: Syncpoint already enabled at capture done!0
[  734.675447] video4linux video0: MW_ACK_DONE syncpoint time out!0
[  734.688200] video4linux video0: TEGRA_VI_CSI_ERROR_STATUS 0x00000004
[  734.701077] vi 54080000.vi: TEGRA_CSI_PIXEL_PARSER_STATUS 0x00000090
[  734.714000] vi 54080000.vi: TEGRA_CSI_CIL_STATUS 0x00000000
[  734.726205] vi 54080000.vi: TEGRA_CSI_CILX_STATUS 0x00000000
[  734.738452] kmtifpga 6-0036: ov5693_s_stream++
[  734.749478] misc tegra_camera_ctrl: vi_v4l2_update_isobw:Set iso bw 0 kbyteps at 0 KHz
[  734.763934] misc tegra_camera_ctrl: tegra_camera_isomgr_request++ bw=0, lt=4
[  734.777548] misc tegra_camera_ctrl: tegra_camera_isomgr_request: tegra_camera isomgr latency is 4 usec
[  734.793596] kmtifpga 6-0036: camera_common_g_fmt++
[  734.804920] misc tegra_camera_ctrl: vi_v4l2_update_isobw:Set iso bw 143078 kbyteps at 17885 KHz
[  734.820210] misc tegra_camera_ctrl: tegra_camera_isomgr_request++ bw=143078, lt=4
[  734.834246] misc tegra_camera_ctrl: tegra_camera_isomgr_request: tegra_camera isomgr latency is 4 usec
[  734.850815] misc tegra_camera_ctrl: vi_v4l2_update_isobw:Set iso bw 0 kbyteps at 0 KHz
[  734.865237] misc tegra_camera_ctrl: tegra_camera_isomgr_request++ bw=0, lt=4
[  734.878914] misc tegra_camera_ctrl: tegra_camera_isomgr_request: tegra_camera isomgr latency is 4 usec
[  734.915042] kmtifpga 6-0036: ov5693_power_off: power off
[  734.927727] kmtifpga 6-0036: camera_common_dpd_enable: csi 0
[  734.940116] kmtifpga 6-0036: camera_common_dpd_enable: csi 1
[  734.952253] kmtifpga 6-0036: camera_common_mclk_disable: disable MCLK

There seems to be error on first frame, but I think this is unimportant. Or is this framerate problem? (29.96 vs 30fps)
I have checked CSI signals and there seems to be no blank data signals. Could this be a problem with encoding?

2

@matas.razgunas

The log show the short frame. The sensor output hight may not as expected.

[  734.688200] video4linux video0: TEGRA_VI_CSI_ERROR_STATUS 0x00000004
[  734.701077] vi 54080000.vi: TEGRA_CSI_PIXEL_PARSER_STATUS 0x00000090
3

Hi matas.razgunas,

Have you clarified the cause and resolved the problem?
Any result can be shared?

Thanks

4

It was frame height error. 1 line was missing. Here is result from Sony FCB-EV7500 https://youtu.be/XHuvr-3lXJw

It would be nice if error would be reported continuously. Not only when stream is stopped.

Right now I’m working to get LWIR camera working, so it will be dual camera carrier board.

5

I have run into same problem with LWIR camera.

[  886.790694] kmtifpga 6-0037: camera_common_try_fmt: size 640 x 480
[  886.800242] kmtifpga 6-0037: camera_common_try_fmt: size 384 x 288
[  886.809690] kmtifpga 6-0037: camera_common_try_fmt: size 384 x 288
[  886.819118] kmtifpga 6-0037: camera_common_try_fmt: size 384 x 288
[  886.828522] kmtifpga 6-0037: camera_common_try_fmt: size 384 x 288
[  886.837910] kmtifpga 6-0037: camera_common_try_fmt: size 384 x 288
[  886.848092] kmtifpga 6-0037: camera_common_try_fmt: size 384 x 288
[  886.857414] kmtifpga 6-0037: camera_common_try_fmt: size 384 x 288
[  886.866591] kmtifpga 6-0037: camera_common_s_fmt(8207) size 384 x 288
[  886.876022] kmtifpga 6-0037: camera_common_try_fmt: size 384 x 288
[  886.898826] kmtifpga 6-0037: camera_common_g_fmt++
[  886.906634] misc tegra_camera_ctrl: vi_v4l2_update_isobw:Set iso bw 7630 kbyteps at 954 KHz
[  886.918169] misc tegra_camera_ctrl: tegra_camera_isomgr_request++ bw=7630, lt=4
[  886.928840] misc tegra_camera_ctrl: tegra_camera_isomgr_request: tegra_camera isomgr latency is 4 usec
[  886.941780] vi 54080000.vi: Calibrate csi port 2
[  886.949834] kmtifpga 6-0037: ov5693_s_stream++
[  886.957325] kmtifpga 6-0037: ov5693_s_stream--
[  904.376503] video4linux video1: Syncpoint already enabled at capture done!0
[  904.465761] video4linux video1: MW_ACK_DONE syncpoint time out!0
[  904.474953] video4linux video1: TEGRA_VI_CSI_ERROR_STATUS 0x00000004
[  904.484410] vi 54080000.vi: TEGRA_CSI_PIXEL_PARSER_STATUS 0x00000080
[  904.493980] vi 54080000.vi: TEGRA_CSI_CIL_STATUS 0x00000000
[  904.502644] vi 54080000.vi: TEGRA_CSI_CILX_STATUS 0x00000000
[  904.511470] kmtifpga 6-0037: ov5693_s_stream++

Error displays frame height is short. But I have verified with oscilloscope (Counting both LineValid signals and CSI signals) that 288 lines are transferred. Is there way to check what frame height CSI is expecting?

6

Please check the TRM for below error.

[ 904.484410] vi 54080000.vi: TEGRA_CSI_PIXEL_PARSER_STATUS 0x00000080

7

As I understand it is this error

PPA_SHORT_FRAME: Set when CSI-PPA receives a short frame. This bit gets set even if 
CSI_PPA_PAD_FRAME specifies that short frames are to be padded to the correct line length.

1.Does frame in error mean one line (single long packet) or full frame (line count)?
2. I have tried increasing Word count (line length) but error still persists. Could this be problem with low resolution(384x288@50 UYVY)?
3. Is there way to check what resolution CSI is expecting?

8

1 Short line means the line is not as expect. Try decrease the height like 288 to 280
2. Doesn’t matter with the sensor line length.
3. It’s the report from sensor resolution.

9

I have manage to fix issue. It was bug in Lattice IP, where data type field was RAW10 instead of YUV422. Here is sample video captured by TX1 https://youtu.be/E6W55oWKS1E

10

Hello matas.razgunas,

Can you please share your driver code snippet? what all changes you have done for YUV422 and
have you used mipi calibration functionalities?

TIA.

11

Code here, it was done for TX1, commit history does not really reflect changes, but you could try to find diffs with original kernel sources

13

Hi matas.razgunas,
Which’s function you changed?