Gst-launch-1.0出图异常为绿色

[sorry.shao]

客户自定义板子
agx orin
jp6.2

1.确认GMSL摄像头的真实输出格式

$ v4l2-ctl --list-formats-ext -d /dev/video4
v4l2-ctl --get-fmt-video -d /dev/video4
ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture

        [0]: 'UYVY' (UYVY 4:2:2)
                Size: Discrete 1920x1080
                        Interval: Discrete 0.033s (30.000 fps)
        [1]: 'NV16' (Y/CbCr 4:2:2)
                Size: Discrete 1920x1080
                        Interval: Discrete 0.033s (30.000 fps)
        [2]: 'UYVY' (UYVY 4:2:2)
                Size: Discrete 1920x1080
                        Interval: Discrete 0.033s (30.000 fps)
Format Video Capture:
        Width/Height      : 1920/1080
        Pixel Format      : 'UYVY' (UYVY 4:2:2)
        Field             : None
        Bytes per Line    : 3840
        Size Image        : 4147200
        Colorspace        : sRGB
        Transfer Function : Default (maps to sRGB)
        YCbCr/HSV Encoding: Default (maps to ITU-R 601)
        Quantization      : Default (maps to Limited Range)

2.使用命令出图是绿的的，如附件

gst-launch-1.0 -v v4l2src device=/dev/video4 ! video/x-raw,width=1920,height=1080,format=UYVY,framerate=30/1 ! videoconvert ! fpsdisplaysink video-sink=xvimagesink sync=false

请问一下，正确出图指令是什么谢谢

补充说明一下：
用下面指令能正常出图，但是format=5或者format=4，但是有较低的概率互换，也不稳定，但是4或者5肯定有一个能正常显示，但是这样客户没办法稳定使用

gst-launch-1.0 -ev v4l2src device=/dev/video$1 ! rawvideoparse format=4 width=1920 height=1080 ! videoconvert ! fpsdisplaysink video-sink=xvimagesink sync=false
或者
gst-launch-1.0 -ev v4l2src device=/dev/video$1 ! rawvideoparse format=5 width=1920 height=1080 ! videoconvert ! fpsdisplaysink video-sink=xvimagesink sync=false

[JerryChang]

hello sorry.shao,

it seems like color channel has swapped. please double check the sensor output formats.
please give it a try to capture a JPEG file for reference,
for instance,
$ gst-launch-1.0 -e v4l2src device=/dev/video0 num-buffers=1 ! "video/x-raw, format=(string)UYVY, width=(int)1920, height=(int)1080" ! jpegenc ! multifilesink location=capture%d.jpeg

BTW,
please refer to developer guide, Applications Using V4L2 IOCTL Directly.
let’s dump the sensor output directly with the --stream-to=test.yuv options,
you may using 3rdparty tool, such as 7yuv to examine the file content for examination as well.

[sorry.shao]

使用这个命令都没有信号

gst-launch-1.0 -e v4l2src device=/dev/video0 num-buffers=1 ! "video/x-raw, format=(string)UYVY, width=(int)1920, height=(int)1080" ! jpegenc ! multifilesink location=capture%d.jpeg

g

[JerryChang]

hello sorry.shao,

@sorry.shao

device=/dev/video4

according to your camera connections. it should be video4 in the test pipeline.

[sorry.shao]

我们有8个摄像头，我是随便用的一个，0到8现象是一样的

[sorry.shao]

@JerryChang

7yuv

please refer to developer guide, Applications Using V4L2 IOCTL Directly.
let’s dump the sensor output directly with the --stream-to=test.yuv options,
you may using 3rdparty tool, such as 7yuv to examine the file content for examination as well.
—>通过指令下载test.yuv格式数据，在通过7yuv打开，选择YUV422 YUYV能正常出图

如果直接预览，需要怎么设置

[JerryChang]

hello sorry.shao,

is the actual output format is YUYV? FYI, YUYV is referred as YUY2 in gstreamer pipeline.
please give it another test with the following,
$ gst-launch-1.0 v4l2src device=/dev/video2 ! video/x-raw,width=1920,height=1080,format=YUY2,framerate=30/1 ! xvimagesink

[sorry.shao]

这是用7yuv应用打开的，选择的YUV422 YUYV

这个指令没办法出图，会报错

$ gst-launch-1.0 v4l2src device=/dev/video2 ! video/x-raw,width=1920,height=1080,format=YUY2,framerate=30/1 ! xvimagesink

[JerryChang]

hello sorry.shao,

please share the error logs.

[sorry.shao]

报错信息如下，

$ gst-launch-1.0 v4l2src device=/dev/video4 ! video/x-raw,width=1920,height=1080,format=YUY2,framerate=30/1 ! xvimagesink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
WARNING: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Signal lost
Additional debug info:
../sys/v4l2/gstv4l2src.c(556): gst_v4l2src_query_preferred_size (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
No input source was detected - video frames invalid
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
../libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.000277216
Setting pipeline to NULL ...
Freeing pipeline ...

[JerryChang]

hello sorry.shao,

@sorry.shao

No input source was detected - video frames invalid
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.

please see-also VI driver,
$public_sources/kernel_src/kernel_src/nvidia-oot/drivers/media/platform/tegra/camera/sensor_common.c
here’s conversion from device tree properties as v4l formats,
you should double check you’ve define the settings to match your sensor output format types.
for instance, mode_type + pixel_phase + csi_pixel_bit_depth.

static int extract_pixel_format(
        const char *pixel_t, u32 *format)
{
...
        else if (strncmp(pixel_t, "yuv_yuyv16", size) == 0)
                *format = V4L2_PIX_FMT_YUYV;
        else if (strncmp(pixel_t, "yuv_yvyu16", size) == 0)
                *format = V4L2_PIX_FMT_YVYU;
        else if (strncmp(pixel_t, "yuv_uyvy16", size) == 0)
                *format = V4L2_PIX_FMT_UYVY;
        else if (strncmp(pixel_t, "yuv_vyuy16", size) == 0)
                *format = V4L2_PIX_FMT_VYUY;

let’s give it another try to disable preview and report frame-rate for checking.
$ gst-launch-1.0 v4l2src device=/dev/video2 ! 'video/x-raw,width=1920,height=1080,format=YUY2' ! videoconvert ! 'video/x-raw,format=I420' ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 -v

if that still cannot report anything, please enable VI tracing log for gather more details..
for instance,

  sudo modprobe rtcpu_debug
  sudo -s
  echo 1 > /sys/kernel/debug/tracing/tracing_on
  echo 30720 > /sys/kernel/debug/tracing/buffer_size_kb
  echo 1 > /sys/kernel/debug/tracing/events/tegra_rtcpu/enable
  echo 1 > /sys/kernel/debug/tracing/events/freertos/enable
  echo 2 > /sys/kernel/debug/camrtc/log-level
  echo 1 > /sys/kernel/debug/tracing/events/camera_common/enable
  echo > /sys/kernel/debug/tracing/trace

[sorry.shao]

@JerryChang

  sudo modprobe rtcpu_debug
  sudo -s
  echo 1 > /sys/kernel/debug/tracing/tracing_on
  echo 30720 > /sys/kernel/debug/tracing/buffer_size_kb
  echo 1 > /sys/kernel/debug/tracing/events/tegra_rtcpu/enable
  echo 1 > /sys/kernel/debug/tracing/events/freertos/enable
  echo 2 > /sys/kernel/debug/camrtc/log-level
  echo 1 > /sys/kernel/debug/tracing/events/camera_common/enable
  echo > /sys/kernel/debug/tracing/trace

1.驱动代码也检查了，也有这部分代码

2.加上debug部分，报错如下图片

$   sudo modprobe rtcpu_debug
  sudo -s
  echo 1 > /sys/kernel/debug/tracing/tracing_on
  echo 30720 > /sys/kernel/debug/tracing/buffer_size_kb
  echo 1 > /sys/kernel/debug/tracing/events/tegra_rtcpu/enable
  echo 1 > /sys/kernel/debug/tracing/events/freertos/enable
  echo 2 > /sys/kernel/debug/camrtc/log-level
  echo 1 > /sys/kernel/debug/tracing/events/camera_common/enable
  echo > /sys/kernel/debug/tracing/trace

root@ubuntu-desktop:/home/ubuntu# gst-launch-1.0 v4l2src device=/dev/video4 ! 'video/x-raw,width=1920,height=1080,format=YUY2'  ! videoconvert ! 'video/x-raw,format=I420' ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 -v
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0: sync = false
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
WARNING: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Signal lost
Additional debug info:
../sys/v4l2/gstv4l2src.c(556): gst_v4l2src_query_preferred_size (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
No input source was detected - video frames invalid
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
../libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.000635232
Setting pipeline to NULL ...
Freeing pipeline ...
root@ubuntu-desktop:/home/ubuntu#

[sorry.shao]

gst-launch-1.0 v4l2src device=/dev/video4 ! 'video/x-raw,width=1920,height=1080,format=YUY2'  ! videoconvert ! 'video/x-raw,format=I420' ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 -v

如果把format关键字去掉之后，能正常运行

[JerryChang]

hello sorry.shao,

it looks the issue from sensor side, please double check what’s the actual output formats.

[sorry.shao]

1. 我们用的是森云的相机，多款相机都有相似的问题。

2. 我们现在用如下指令出入，这个指令出图是正常的。

gst-launch-1.0 -ev v4l2src device=/dev/video4 ! rawvideoparse format=4 width=1920 height=1080 ! videoconvert ! fpsdisplaysink video-sink=xvimagesink sync=false

或者

gst-launch-1.0 -ev v4l2src device=/dev/video4 ! rawvideoparse format=5 width=1920 height=1080 ! videoconvert ! fpsdisplaysink video-sink=xvimagesink sync=false

3. 目前用的摄像头是 SG2-OX03CC-5200-GMSL2F，森云 SG2-OX03CC-5200-GMSL2F 相机的主要输出格式是 YUV422‑8bit（YUYV）彩色视频流。我们我们代码里面YUV422‑16bit（YUYV），不知道有没有影响

这几款也有相似的问题
森云SG2-OX03CC-5200-GMSL2F
森云IMX390C-5200-GMSL2
森云SG2-AR0233C-5200-GMSL2
森云SG1-OX01F10C-OmniVision
SG5-IMX490C-5300-GMSL2S
森云SG3-ISX031C-GMSL2

[JerryChang]

hello sorry.shao,

device tree settings must be identical with your actual hardware configuration.
it’s $ v4l2-ctl --list-formats-ext -d /dev/video0 to report the device tree settings you’ve done.
you should dig into your device tree settings to examine mode_type and pixel_phase configurations.

[sorry.shao]

$ v4l2-ctl --list-formats-ext -d /dev/video0
ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture

        [0]: 'YUYV' (YUYV 4:2:2)
                Size: Discrete 1920x1080
                        Interval: Discrete 0.033s (30.000 fps)
        [1]: 'YUYV' (YUYV 4:2:2)
                Size: Discrete 1920x1080
                        Interval: Discrete 0.033s (30.000 fps)
ubuntu@ubuntu-desktop:~$ v4l2-ctl --list-formats-ext -d /dev/video4
ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture

        [0]: 'UYVY' (UYVY 4:2:2)
                Size: Discrete 1920x1080
                        Interval: Discrete 0.033s (30.000 fps)
        [1]: 'NV16' (Y/CbCr 4:2:2)
                Size: Discrete 1920x1080
                        Interval: Discrete 0.033s (30.000 fps)
        [2]: 'UYVY' (UYVY 4:2:2)
                Size: Discrete 1920x1080
                        Interval: Discrete 0.033s (30.000 fps)

我看来一下我们设备树配置的也是yuv，具体如下
tegra234-p3737-camera-modules.zip (3.7 KB)

						mode0 {/*mode OX03C10_MODE_1920X1080_CROP_30FPS*/
							mclk_khz = "24000";
							status = "okay";
							num_lanes = "4";
							tegra_sinterface = "serial_b";
							vc_id = "3";
							discontinuous_clk = "no";
							dpcm_enable = "false";
							cil_settletime = "0";
							dynamic_pixel_bit_depth = "16";
							csi_pixel_bit_depth = "16";
							mode_type = "yuv";
							pixel_phase = "uyvy";

							active_w = "1920";
							active_h = "1080";
							readout_orientation = "0";
							line_length = "2200";
							inherent_gain = "1";
							mclk_multiplier = "6.16";
							pix_clk_hz = "74250000";
							serdes_pix_clk_hz = "374000000";

							gain_factor = "100";
							min_gain_val = "100"; /* dB */
							max_gain_val = "1600"; /* dB */
							step_gain_val = "1"; /* 0.1 */
							default_gain = "100";
							min_hdr_ratio = "1";
							max_hdr_ratio = "1";
							framerate_factor = "1000000";
							min_framerate = "30000000";
							max_framerate = "30000000";
							step_framerate = "30000000";
							default_framerate = "30000000";
							exposure_factor = "1000000";
							min_exp_time = "28"; /*us, 2 lines*/
							max_exp_time = "22000";
							step_exp_time = "1";
							default_exp_time = "22000";/* us */
							embedded_metadata_height = "0";
						};

[JerryChang]

hello sorry.shao,

let me have confirmation,
is it true video0’s format output with YUYV , whereas video4’s format output with UVYV?

here’re couple of incorrect settings and port bindings.
for instance, ox03c10_d@17,
you must set the vc-id property in the gmsl-link node for each sensor to match the vc_id property in the sensor mode device node

						mode0 {/*mode OX03C10_MODE_1920X1080_CROP_30FPS*/
							num_lanes = "4";
							vc_id = "3";

						ports {
							port@0 {
								ox03c10_ox03c10_out7: endpoint {
									vc-id = <0>;

and, you should have tegra_sinterface maps to port-index.
please update the property according to the CSI connections.

							tegra_sinterface = "serial_b";

									port-index = <4>;
									bus-width = <4>;

as you can see..
this is the YUV formats of your camera configuration,
please check whether video4’s format output with UVYV.
otherwise, please update the pixel_phase according to your sensor hardware settings.

							dynamic_pixel_bit_depth = "16";
							csi_pixel_bit_depth = "16";
							mode_type = "yuv";
							pixel_phase = "uyvy";

BTW,
there’s one minor issue,
it’s position property depend on the number of cameras in the system.
please use below for your 8-cam system.

enum {
    bottomleft,    //video0
    centerleft,    //video1
    centerright,   //video2
    topleft,       //video3
    bottomright,   //video4
    topright,      //video5
    bottomcenter,  //video6
    topcenter,     //video7

[sorry.shao]

有没有相关的demo，我们参考重新改一下

[JerryChang]

hello sorry.shao,

I’ve already point-out the code need to revise, please give them a try.