ZED Stereo USB Camera as input for sample application

I am able to use the onboard camera as a sample input, however, I want to use the ZED Camera as a video input. It is a USB camera.

Make sure there are not other cameras being picked accidentally as a default. For example, if you connect a ZED to a Jetson TX2 you will see a “/dev/video0” for the carrier board camera, and the ZED camera will be “/dev/video1”. Either the ZED software or guvcview could be used if you name “/dev/video1” (adjust for your case…check “ls /dev/video*” before connecting the ZED, and then again after).

I am sorry I was not more specific. In the config file given, I changed the video input source to 1 which is the onboard camera. My question is more specifically what should I change it to in order to have it input from the ZED camera into the sample detection application? Thanks!

Sorry, that’s one I don’t know.

Hi mrkolaczek,
Can you open the ZED camera with ‘v4l2-ctl’?

Hi mrkolaczek,
How do you use the onboard video camera as the sample input?

Probably the answer is In the config file given, I changed the video input source to 1
Perhaps the config file is PGIE-FP16-CarType-CarMake-CarColor.txt
Self answered, thank you.
Regarding the initial issue, I can see ZED under :

ls /dev/vid*/dev/video0  /dev/video1  /dev/video2

but I do not have v4l2-ctl
However, while testing I can see another issue with getting the GUI versus non-gui.
Non gui execution below goes just fine:

nvgstiva-app -c ${HOME}/configs/PGIE-FP16-CarType-CarMake-CarColor.txt  -i ${HOME}/newanglevideo.mp4

but when using gui it wont load the specified file:

nvgstiva-app-gui -c ${HOME}/configs/PGIE-FP16-CarType-CarMake-CarColor.txt  -i ${HOME}/newanglevideo.mp4
** ERROR: <main:1781>: Unknown option -i

However, I managed to run the gui-app version with custom input video , it appears that -i parameter is depreciated in the existing implementation of the app. And the path to the file needs to be hard-coded into the configuration file, rather than passed with -i /path/file.mp4.
However, pertaining to the initial question, it appears that I can install the v4l2-ctl with the execution of the code below:

sudo apt-get install qv4l2 v4l-utils

Upd: now I am getting the below

Error from src_elem: Internal data flow error.
Debug info: gstbasesrc.c(2948): gst_base_src_loop (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin0/GstV4l2Src:src_elem:
streaming task paused, reason not-negotiated (-4)
v4l2-ctl --list-formats -d /dev/video1
ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'YUYV'
	Name        : YUYV 4:2:2

nvidia@tegra-ubuntu:~$ v4l2-ctl --list-formats -d /dev/video0
ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'BG10'
	Name        : 10-bit Bayer BGBG/GRGR

nvidia@tegra-ubuntu:~$ v4l2-ctl --list-formats -d /dev/video2
ioctl: VIDIOC_ENUM_FMT

I can get raw file with v4l2-ctl, but that seems a challenge to display the content of the raw file to verify if that actually works, or otherwise.

Not sure I really understand your problem, but you can display your ZED camera capture with

gst-launch-1.0 v4l2src device=/dev/video1 ! video/x-raw, width=2560, height=720, format=YUY2 , framerate=60/1 ! xvimagesink

no, I have approached to get a video from ZED using the v4l2-ctl code below:

v4l2-ctl --set-fmt-video=width=1280,height=720,pixelformat=BG10 --stream-mmap -d /dev/video1 --set-ctrl bypass_mode=0 --stream-count=300 --stream-to=ov5693.raw

It results in raw file that I do not know how to display.
However, the sequence you provided in the previous message displays the Zed stream. Thank you.
The issue is that the underlying deep stream sample app mechanism doesn’t appear to utilize the gst-launch-1.0 v4l2src. Though it is said to use v4l2src method.
Though I am just getting familiar with the tool and some my assumptions could be errorneous.
Actually I am not sure if I understand the problem as well and if it exists at all. I just tried to answer the DaneLLL question if v4l2-ctl does open ZED cam and that did move me to terra incognito.
The original issue I did approach looking into was if the sample app will run with ZED inputs.
Actually, if I am attaching a generic USB Webcam the same error occurs in case I set up in the sample app configuration file to use v4l2 input.

[application]
enable-perf-measurement=1
roi-marking=0
perf-measurement-interval-sec=5
gie-kitti-output-dir=/home/nvidia/kitti1
# 0 = standalone; 1 = server; 2 = client
app-mode=0

[source0]
enable=2
#Type - 1=CameraCSi 2=CameraV4L2 3=URI
type=2
camera-width=1920
camera-height=1080
camera-fps-n=30
camera-fps-d=1
camera-csi-sensor-id=0
camera-v4l2-dev-node=1
uri=file:///home/nvidia/sample_720p.mp4

[sink0]
enable=1
#Type - 1=FakeSink 2=OverlaySink 3=EglSink 4=XvImageSink 5=File
type=2
display-id=0
offset-x=0
offset-y=0
width=0
height=0
sync=1
overlay-index=1
source-id=0

[sink1]
enable=0
#Type - 1=FakeSink 2=OverlaySink 3=EglSink 4=XvImageSink 5=File
type=5
#1=mp4 2=mkv
container=2
#1=h264 2=h265
codec=1
bitrate=10000000
#1=cbr 2=vbr
rc-mode=2
iframeinterval=30
#1=baseline 2=main 3=high
profile=3
output-file=out.mp4
source-id=0
v4l2-ctl --list-devicesVIDIOC_QUERYCAP: failed: Inappropriate ioctl for device
VIDIOC_QUERYCAP: failed: Inappropriate ioctl for device
vi-output, ov5693 2-0036 (platform:15700000.vi:2):
	/dev/video0

Dummy video device (0x0000) (platform:v4l2loopback-000):
	/dev/video2
	/dev/v4l-subdev1
	/dev/v4l-subdev0

Webcam C110 (usb-3530000.xhci-2):
	/dev/video1

But any USB camera input will throw :

Error from src_elem: Internal data flow error.
Debug info: gstbasesrc.c(2948): gst_base_src_loop (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin0/GstV4l2Src:src_elem:
streaming task paused, reason not-negotiated (-4)

Update: another way to read from usb cam:

vlc v4l2:///dev/video1 --v4l2-chroma=h264

Perhaps to get imput coming from videocamera we can use URI method, instead of v4l2
source: https://stackoverflow.com/questions/49135496/how-to-read-h264-stream-as-a-file-from-the-usb-webcam-directly-in-c-c-without?rq=1

That command was for the onboard CSI camera (using /dev/video0).
I cannot say for C110, but for ZED I would capture 5s @60fps (more than 1GB) with:

v4l2-ctl -d /dev/video1 --set-fmt-video=width=2560,height=720,pixelformat=YUYV -p60 
v4l2-ctl -d /dev/video1 --stream-mmap --stream-count=300 --stream-to=zed.yuyv

Anyway, ZED has only this YUYV format, so it’s unlikely to be changed. You cannot use ctrl commands if the device doesn’t provide these controls. Use

qv4l2 -d /dev/video1

for checking the available controls and be sure you have the correct settings.

I have no knowledge of the underlying deep stream sample app mechanism, someone else may further comment the configuration.

Somehow nearly solved. When I want the app to get the input from USB or Zed camera I do

  1. start rtsp server
  2. specify in the app the urp pointing it to the rtsp server
    However the 1&2 steps do work by default for MIPI CSI camera, but when I try to rtsp /dev/video1 it wont work , neither within the app nor within the vlc:
/test-launch "v4l2src device=/dev/video1 ! omxh265enc ! rtph265pay name=pay0 pt=96"
stream ready at rtsp://127.0.0.1:8554/test
Framerate set to : 15 at NvxVideoEncoderSetParameterNvMMLiteOpen : Block : BlockType = 8 
===== MSENC =====
NvMMLiteBlockCreate : Block : BlockType = 8 
===== NVENC blits (mode: 1) into block linear surfaces =====
  1. I open in vlc “rtsp://127.0.0.1:8554/test”
    but it will not display

when I play the yuyv file it just shows all green. Does it display normally at your side?

vlc --demux rawvideo --rawvid-fps 25 --rawvid-width 1920 --rawvid-height 1080 --rawvid-chroma I420 zed.yuyv

It shows more interesting picture with

vlc --demux rawvideo --rawvid-fps 25 --rawvid-width 1920 --rawvid-height 1080 --rawvid-chroma YUYV zed.yuyv


it is like moves from top to bottom or vice versa while playing, and sort of triples instead of regular doubling

Seems that omxh265enc only accepts I420 or NV12 from NVMM memory.
You may find where is the gstreamer pipeline defined and insert nvvidconv for converting from YUY2 (YUYV) to I420 or NV12.

I suppose you would try instead

vlc --demux rawvideo --rawvid-fps 60 --rawvid-width 2560 --rawvid-height 720 --rawvid-chroma YUYV zed.yuyv

I don’t think you can use 1080p format at capture (you may convert with nvvidconv):

v4l2-ctl -d /dev/video1 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'YUYV'
	Name        : YUYV 4:2:2
		Size: Discrete 2560x720
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 1344x376
			Interval: Discrete 0.010s (100.000 fps)
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 3840x1080
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 4416x1242
			Interval: Discrete 0.067s (15.000 fps)

Yep, it works precisely.

may be you also know how to push Zed stream to rtsp somewhat like:
./test-launch “v4l2src device=/dev/video1 ! omxh265enc ! rtph265pay name=pay0 pt=96”
?

I don’t have test-launch installed now for checking, but you may try this pipeline :

./test-launch "v4l2src device=/dev/video1 ! 'video/x-raw, format=YUY2' ! nvvidconv ! 'video/x-raw(memory:NVMM), format=I420' ! omxh265enc ! rtph265pay name=pay0 pt=96"

Update: The pipeline I had proposed had quotes it shouldn’t have.
This works:

./test-launch "v4l2src device=/dev/video1 ! video/x-raw,format=YUY2 ! nvvidconv ! video/x-raw(memory:NVMM),width=2560,height=720,format=I420 ! omxh265enc ! rtph265pay name=pay0 pt=96"

I can decode the stream with:

gst-launch-1.0  rtspsrc location=rtsp://127.0.0.1:8554/test ! 'application/x-rtp, media=(string)video' ! decodebin ! videoconvert ! xvimagesink