Deepstreamer app doesn't work YV12 video format

goutam.gupta · September 21, 2020, 7:06am

Hi,

I am not able to run deepstream app with a 3G-SDI camera which outputs video in YV12 format. Is there any way to pass the video format to deepstream app ?
Thanks.

Fiona.Chen · September 21, 2020, 7:21am

Can you provide platform and software information with the template?
• Hardware Platform (Jetson / GPU)
• DeepStream Version
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)

What kind of camera? CSI, USB or RTSP? How did you get the YV12 format video from camera with gstreamer?

goutam.gupta · September 21, 2020, 7:33am

Hi ,
Please find below the details:
Hardware Platform : TX2i
Deepstream version : 5.0
TensorRT version: 7.1.3.0
I am using 3G-SDI camera which is connected to mPCIE card. The device is recognised as /dev/video1. I am able to see the image by running following command:
gst-launch-1.0 v4l2src io-mode=2 device=/dev/video0 ! video/x-raw,width=1920,height=1080,format=YV12 ! xvimagesink sync=false&

The video format supported by this device is found by running the command:
jetson@jetson-desktop:~$ v4l2-ctl -d /dev/video1 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: ‘MPEG’ (compressed)
Name : H264 Encoded Stream
Size: Discrete 640x480
Interval: Discrete 0.017s (60.000 fps)
Size: Discrete 720x480
Interval: Discrete 0.017s (60.000 fps)
Size: Discrete 720x576
Interval: Discrete 0.020s (50.000 fps)
Size: Discrete 800x600
Interval: Discrete 0.017s (60.000 fps)
Size: Discrete 1024x768
Interval: Discrete 0.017s (60.000 fps)
Size: Discrete 1280x720
Interval: Discrete 0.020s (50.000 fps)
Interval: Discrete 0.017s (60.000 fps)
Size: Discrete 1280x800
Interval: Discrete 0.017s (60.000 fps)
Size: Discrete 1280x1024
Interval: Discrete 0.017s (60.000 fps)
Size: Discrete 1360x768
Interval: Discrete 0.017s (60.000 fps)
Size: Discrete 1440x900
Interval: Discrete 0.017s (60.000 fps)
Size: Discrete 1920x1080
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.042s (24.000 fps)
Interval: Discrete 0.040s (25.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.020s (50.000 fps)
Interval: Discrete 0.017s (60.000 fps)
Size: Discrete 1680x1050
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.042s (24.000 fps)
Interval: Discrete 0.040s (25.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.020s (50.000 fps)
Interval: Discrete 0.017s (60.000 fps)

Index       : 1
Type        : Video Capture
Pixel Format: 'YV12'
Name        : YVU 4:2:0 12bpp
	Size: Discrete 640x480
		Interval: Discrete 0.017s (60.000 fps)
	Size: Discrete 720x480
		Interval: Discrete 0.017s (60.000 fps)
	Size: Discrete 720x576
		Interval: Discrete 0.020s (50.000 fps)
	Size: Discrete 800x600
		Interval: Discrete 0.017s (60.000 fps)
	Size: Discrete 1024x768
		Interval: Discrete 0.017s (60.000 fps)
	Size: Discrete 1280x720
		Interval: Discrete 0.020s (50.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
	Size: Discrete 1280x800
		Interval: Discrete 0.017s (60.000 fps)
	Size: Discrete 1280x1024
		Interval: Discrete 0.017s (60.000 fps)
	Size: Discrete 1360x768
		Interval: Discrete 0.017s (60.000 fps)
	Size: Discrete 1440x900
		Interval: Discrete 0.017s (60.000 fps)
	Size: Discrete 1920x1080
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.042s (24.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.020s (50.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
	Size: Discrete 1680x1050
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.042s (24.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.020s (50.000 fps)
		Interval: Discrete 0.017s (60.000 fps)

Index       : 2
Type        : Video Capture
Pixel Format: 'YU12'
Name        : YUV 4:2:0 12bpp
	Size: Discrete 640x480
		Interval: Discrete 0.017s (60.000 fps)
	Size: Discrete 720x480
		Interval: Discrete 0.017s (60.000 fps)
	Size: Discrete 720x576
		Interval: Discrete 0.020s (50.000 fps)
	Size: Discrete 800x600
		Interval: Discrete 0.017s (60.000 fps)
	Size: Discrete 1024x768
		Interval: Discrete 0.017s (60.000 fps)
	Size: Discrete 1280x720
		Interval: Discrete 0.020s (50.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
	Size: Discrete 1280x800
		Interval: Discrete 0.017s (60.000 fps)
	Size: Discrete 1280x1024
		Interval: Discrete 0.017s (60.000 fps)
	Size: Discrete 1360x768
		Interval: Discrete 0.017s (60.000 fps)
	Size: Discrete 1440x900
		Interval: Discrete 0.017s (60.000 fps)
	Size: Discrete 1920x1080
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.042s (24.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.020s (50.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
	Size: Discrete 1680x1050
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.042s (24.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.020s (50.000 fps)
		Interval: Discrete 0.017s (60.000 fps)

Index       : 3
Type        : Video Capture
Pixel Format: 'NV12'
Name        : NV12 4:2:0 12bpp
	Size: Discrete 640x480
		Interval: Discrete 0.017s (60.000 fps)
	Size: Discrete 720x480
		Interval: Discrete 0.017s (60.000 fps)
	Size: Discrete 720x576
		Interval: Discrete 0.020s (50.000 fps)
	Size: Discrete 800x600
		Interval: Discrete 0.017s (60.000 fps)
	Size: Discrete 1024x768
		Interval: Discrete 0.017s (60.000 fps)
	Size: Discrete 1280x720
		Interval: Discrete 0.020s (50.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
	Size: Discrete 1280x800
		Interval: Discrete 0.017s (60.000 fps)
	Size: Discrete 1280x1024
		Interval: Discrete 0.017s (60.000 fps)
	Size: Discrete 1360x768
		Interval: Discrete 0.017s (60.000 fps)
	Size: Discrete 1440x900
		Interval: Discrete 0.017s (60.000 fps)
	Size: Discrete 1920x1080
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.042s (24.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.020s (50.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
	Size: Discrete 1680x1050
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.042s (24.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.020s (50.000 fps)
		Interval: Discrete 0.017s (60.000 fps)

This is the change made into deepstream_app_config file :

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=1
gie-kitti-output-dir=streamscl

[tiled-display]
enable=1
rows=1
columns=1
#width=1280
#height=720
width=1920
height=1080
gpu-id=0
nvbuf-memory-type=0

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=1
camera-width=1920
camera-height=1080
camera-fps-n=15
camera-fps-d=1
camera-v4l2-dev-node=1

[streammux]
gpu-id=0
batch-size=1
batched-push-timeout=-1

Set muxer output width and height

width=1920
height=1080
nvbuf-memory-type=0

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=1
source-id=0
gpu-id=0

[osd]
enable=1
gpu-id=0
border-width=3
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[primary-gie]
enable=1
gpu-id=0
batch-size=1
gie-unique-id=1
interval=0
labelfile-path=ssd_coco_labels.txt
model-engine-file=sample_ssd_relu6.uff_b1_gpu0_fp32.engine
config-file=config_infer_primary_ssd.txt
nvbuf-memory-type=0

I am able to successfully run the same app with USB camera which is /dev/video2.

Thanks
Goutam

Fiona.Chen · September 21, 2020, 7:38am

That’s good! Since nvvideoconvert only support I420，P010_10LE, NV12, BGRx, RGBA,or GRAY8 format input, you can use “videoconvert” to convert YV12 to I420 or NV12 format, and then use nvvideoconvert to convert the video to NVMM NV12 format for DeepStream.

goutam.gupta · September 21, 2020, 7:41am

Thanks. I am little new to this. Can you please elaborate more ? Which file do i need to make these changes? Any steps/instructions will be helpful.

Fiona.Chen · October 10, 2020, 9:13am

The camera supports more than one format (YV12, YU12, NV12) and deepstream-app can not set the format with config file. You may need to try to modify the deepstream-app source codes.

In /opt/nvidia/deepstream/deepstream-5.0/sources/apps/apps-common/src/deepstream_source_bin.c, the camera source is created in create_camera_source_bin() function.

You may try to change the following codes

caps1 = gst_caps_new_simple (“video/x-raw”,
“width”, G_TYPE_INT, config->source_width, “height”, G_TYPE_INT,
config->source_height, “framerate”, GST_TYPE_FRACTION,
config->source_fps_n, config->source_fps_d, NULL);

to the codes of

caps1 = gst_caps_new_simple (“video/x-raw”,
“width”, G_TYPE_INT, config->source_width, “height”, G_TYPE_INT,
config->source_height, “framerate”, GST_TYPE_FRACTION,
config->source_fps_n, config->source_fps_d, “format”, “YV12”, NULL);

And you need to rebuild deepstream-app and run the new deepstream-app to check whether it works.

Topic		Replies	Views
V4l2src & nvvideoconvert connection issue DeepStream SDK	10	784	September 30, 2023
USB Camera not working - Deepstream SDK 5 DeepStream SDK camera	9	2962	October 12, 2021
How to change source pixel format from YUYV to MJPG DeepStream SDK	11	8714	October 12, 2021
Camera not working while streaming with Deepstream 6.0 DeepStream SDK	5	1472	March 15, 2022
How to use deepstream-app with MJPEG format stream? 2nd try DeepStream SDK usb , deepstream	8	949	March 13, 2024
Setup nvconv dynamically for USB web camera DeepStream SDK	2	300	January 25, 2024
DeepStream: incorrect camera parameters provided, please provide supported resolution and frame rate DeepStream SDK camera	11	2336	May 18, 2022
Low image quality when using usb cam in Deepstream DeepStream SDK camera , jetson-inference , gstreamer	12	624	July 1, 2023
Deepstream 2.0 can not use USB camera DeepStream SDK	3	698	August 6, 2018
Low camera frame rate DeepStream SDK camera , nvbugs	30	6183	October 12, 2021

Deepstreamer app doesn't work YV12 video format

Set muxer output width and height

Related topics