Hello,
is any of the following commands are applied by the CPU core or in a dedicated HW core encoder/convertor/decoder inside the Nvidia:
gst-launch-1.0 v4lsrc device="/dev/video0" ! "video/x-raw, width=1280, height=720, format=RGB" ! videoconvert ! xvimagesink -e
or in
gst-launch-1.0 v4lsrc device="/dev/video0" ! "video/x-raw, width=1280, height=720, format=RGB" ! videoconvert ! omxh246enc ! ....
If the CPU core is utilized how should Ireduce this utilization?
Regards.
Hi igal,
For v4l2src, you can get better performance by configuring the format precisely.
With the usb camera I have for example. By executing ‘v4l2-ctl --list-formats-ext’, it shows
nvidia@tegra-ubuntu:~$ v4l2-ctl -d /dev/video1 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: 'UYVY'
Name : UYVY 4:2:2
Size: Discrete 1280x720
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1920x1080
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 3840x2160
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.133s (7.500 fps)
Size: Discrete 4208x3120
Interval: Discrete 0.111s (9.000 fps)
Interval: Discrete 0.222s (4.500 fps)
Size: Discrete 4096x2160
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.133s (7.500 fps)
Size: Discrete 640x480
Interval: Discrete 0.008s (120.000 fps)
Interval: Discrete 0.017s (60.000 fps)
Index : 1
Type : Video Capture
Pixel Format: 'MJPG' (compressed)
Name : Motion-JPEG
Size: Discrete 1280x720
Interval: Discrete 0.017s (60.000 fps)
Size: Discrete 1920x1080
Interval: Discrete 0.017s (60.000 fps)
Size: Discrete 3840x2160
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 4208x3120
Interval: Discrete 0.050s (20.000 fps)
Size: Discrete 4096x2160
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 640x480
Interval: Discrete 0.008s (120.000 fps)
For running 720p60 UYVY
Size: Discrete 1280x720
Interval: Discrete 0.017s (60.000 fps)
the preview command should be
$ gst-launch-1.0 v4l2src device=/dev/video1 ! video/x-raw,format=UYVY,width=1280,height=720,framerate=60/1 ! nvvidconv ! 'video/x-raw(memory:NVMM),format=NV12' ! nvoverlaysink
encoding command
$ gst-launch-1.0 v4l2src device=/dev/video1 num-buffers=300 ! video/x-raw,format=UYVY,width=1280,height=720,framerate=60/1 ! nvvidconv ! 'video/x-raw(memory:NVMM),format=NV12' ! omxh264enc ! qtmux ! filesink location=a.mp4
You can check CPU utilization via tegrastats.
DaneLLL, hi,
thanks for the reply.
This is what I get:
<b>v4l2-ctl -d /dev/video0 --list-formats-ext</b>
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: 'VYUY'
Name : VYUY 4:2:2
Size: Discrete 2592x1944
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 2592x1458
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1920x1080
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1280x720
Interval: Discrete 0.008s (120.000 fps)
Size: Discrete 2592x1944
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1920x1080
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 2592x1944
Interval: Discrete 0.033s (30.000 fps)
<b>gst-launch-1.0 v4l2src device="/dev/video0" ! "video/x-raw, width=1280, height=720, format=VYUY, framerate=60/1" ! nvvidconv ! 'video/x-raw(memory:NVMM), format=NV12' ! nvoverlaysink</b>
WARNING: erroneous pipeline: could not link v4l2src0 to nvvconv0
and no video is displayed.
any suggestions?
Hi igal,
VYUY is not suppoerted by gstreamer frameworks. You may try to run
$ gst-launch-1.0 v4l2src device=/dev/video1 ! video/x-raw,format=<b>I420</b>,width=1280,height=720,framerate=60/1 ! nvvidconv ! 'video/x-raw(memory:NVMM),format=NV12' ! nvoverlaysink
To see if v4l2src can handle VYUY->I20 conversion.
Here are the terminal printout:
gst-launch-1.0 v4l2src device="/dev/video0" ! "video/x-raw, width=1280, height=720, <b>format=I420</b>, framerate=60/1" ! nvvidconv ! 'video/x-raw(memory:NVMM), format=NV12' ! nvoverlaysink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Device '/dev/video0' cannot capture in the specified format
Additional debug info:
gstv4l2object.c(3482): gst_v4l2_object_set_format_full (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Tried to capture in YU12, but device returned format VYUY
Execution ended after 0:00:00.000230366
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
Hi igal,
Seems like VYUV is not well supported in gstreamer. You may go to http://gstreamer-devel.966125.n4.nabble.com/ to get help.
Besides, you can try 12_camera_v4l2_cuda in tegra_multimedia_api.
DaneLLL,
sorry, I’ve looked in the link that you’ve pointed me to… nothing there I could find useful.
What did you mean that gstreamer does not support VYUV, isn’t this format essential?
(BTW, any of the YUV combination is acceptable as opposed to RGB “family”).
Hi Igal,
You can check the capability of v4l2src:
nvidia@tegra-ubuntu:~$ gst-inspect-1.0 v4l2src
(ignore...)
Pad Templates:
SRC template: 'src'
Availability: Always
Capabilities:
image/jpeg
video/mpeg
mpegversion: 4
systemstream: false
video/mpeg
mpegversion: 2
video/mpegts
systemstream: true
video/x-bayer
format: { bggr, gbrg, grbg, rggb }
width: [ 1, 32768 ]
height: [ 1, 32768 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-dv
systemstream: true
video/x-h263
variant: itu
video/x-h264
stream-format: { byte-stream, avc }
alignment: au
video/x-pwc1
width: [ 1, 32768 ]
height: [ 1, 32768 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-pwc2
width: [ 1, 32768 ]
height: [ 1, 32768 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-raw
format: { RGB16, BGR, RGB, GRAY8, GRAY16_LE, GRAY16_BE, YVU9, YV12, YUY2, YVYU, UYVY, Y42B, Y41B, YUV9, NV12_64Z32, NV24, NV61, NV16, NV21, NV12, I420, BGRA, BGRx, ARGB, xRGB, BGR15, RGB15 }
width: [ 1, 32768 ]
height: [ 1, 32768 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-sonix
width: [ 1, 32768 ]
height: [ 1, 32768 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-vp8
video/x-wmv
wmvversion: 3
format: WVC1
(ignore...)
VYUY is not supported. You can get RGB but it should be format conversion via CPU in v4l2src. No chance to reduce CPU utilization.
Hi DaneLLL.
I have figured out how to configure to UYVY, and I have a streaming video from an external source.
Though I am not sure, yet, which option has the less CPU utilization, either getting RGB format from the external source or getting UYVY format from the external source (both options are available)?
I understand that both options are converted to NV12, prior to displaying/encoding.
Please elaborate.
Hi Igal,
configuring the format supported by the source is with less CPU utilization. You can compare the cases via tegrastats.
DaneLLL, hi,
sorry for my slow understanding, but what is the expected format that will make NVidia to produce less CPU utilization, that is, no software (but only hardware) conversions will be applied?
In order to be provided an RGB format I have to make my video source provider make extra work. So, I need to know if I should invest effort in this direction or the current UYVY to NV12 provides the minimal CPU utilization?
Please take into consideration that I have to compress the video (H.264/H.265) along the way.
Regards.
Hi Igal,
Gstreamer v4l2 element is from 3rdparty and not owned by NVIDIA. Suggest you try 12_camera_v4l2_cuda in tegra_multimedia_api. The sample optimize the v4l2 source case.
DaneLLL,
I need to use the gstreamer to encode and decode video (including audio). Is the tegra_multimedia_api is the most efficient way to apply the encoding and decoding?
Hi igal,
tegra_multimedia_api can bring better performance in leveraging TX2 HW engines, but more coding.
If you want to accelerate the development, gstreamer is simpler. Please run tegrastats to see CPU utilization of RGB and UYVY.
Since UYVY is supported by nvvidconv, it should consume less CPU utilization than RGB.
gst-launch-1.0 v4l2src device="/dev/video0" ! "video/x-raw, width=1280, height=720, <b>format=UYVY</b>, framerate=60/1" ! nvvidconv ! 'video/x-raw(memory:NVMM), format=NV12' ! nvoverlaysink