Hello!
We get images from two 13 MP sensors with fisheye-type lenses. We use gstreamer
capturing video in the resolution of 4224 * 3156 is fast and does not load the processor, we get 19 fps from each camera.
We need to scale the resolution to 1920 * 1080, using nvvidconv we get very bad results - about 7.5 fps.
If we immediately remove from the sensor 1920 * 1080 then without scaling everything is fine. How can I replace the nvvidconv module for scaling tasks? Maybe there is some kind of plugin that runs on the GPU quickly?
1 Like
Hi alexey,
For more information, please share both pipelines of 19 fps and 7.5 fps.
Hello Danel
pipe for two sensor capture:
gst-launch-1.0 v4l2src device=”/dev/video0” num-buffers=600 ! “video/x-raw, width=4224, height=3156, format UYVY ! fakesink” &
gst-launch-1.0 v4l2src device=”/dev/video0” num-buffers=600 ! “video/x-raw, width=4224, height=3156, format UYVY ! fakesink”
I get the following result
Execution ended after 0: 00: 31.795340909
Execution ended after 0: 00: 32.426287326
about 19 fps for this sensor is the maximum result at this resolution
Next, we checked the operation of the hardware compression module, we encoded two independent streams, for each sensor.
############################################################################################
gst-launch-1.0
v4l2src device=“/dev/video0” num-buffers=600
! “video/x-raw, width=1920, height=1080, format=UYVY”
! nvvidconv
! “video/x-raw(memory:NVMM), format=I420, width=1920, height=1080, framerate=72/1”
! omxh264enc
! h264parse
! qtmux
! fakesink &
############################################################################################
gst-launch-1.0
v4l2src device=“/dev/video1” num-buffers=600
! “video/x-raw, width=1920, height=1080, format=UYVY”
! nvvidconv
! “video/x-raw(memory:NVMM), format=I420, width=1920, height=1080, framerate=72/1”
! omxh264enc
! h264parse
! qtmux
! fakesink
result
Got EOS from element “pipeline0”.
Execution ended after 0:00:15.778541246
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …
Got EOS from element “pipeline0”.
Execution ended after 0:00:15.877253580
Setting pipeline to PAUSED …
Setting pipeline to READY …
NvRmChannelFreeSyncpt ioctl failed with 22
Setting pipeline to NULL …
Freeing pipeline …
so ~ 40 FPS is an excellent result for us
in the process of working codecs, two CPU cores were used for 90%
then we tried to give full resolution from the sensor and scaled it with the nvvidconv module up to 1080p resolution (on which the encoder should work well)
we expected to see a result no lower than 19 FPS.
############################################################################################
gst-launch-1.0
v4l2src device=“/dev/video0” num-buffers=600
! “video/x-raw, width=4224, height=3156, format=UYVY”
! nvvidconv
! “video/x-raw(memory:NVMM), format=I420, width=1920, height=1080, framerate=20/1”
! omxh264enc
! h264parse
! qtmux
! fakesink &
############################################################################################
gst-launch-1.0
v4l2src device=“/dev/video1” num-buffers=600
! “video/x-raw, width=4224, height=3156, format=UYVY”
! nvvidconv
! “video/x-raw(memory:NVMM), format=I420, width=1920, height=1080, framerate=20/1”
! omxh264enc
! h264parse
! qtmux
! fakesink
Got EOS from element “pipeline0”.
Execution ended after 0:01:20.785178897
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …
Got EOS from element “pipeline0”.
Execution ended after 0:01:21.048191480
Setting pipeline to PAUSED …
Setting pipeline to READY …
NvRmChannelFreeSyncpt ioctl failed with 22
Setting pipeline to NULL …
Freeing pipeline …
but in practice we saw only 7.5 FPS, this is an extremely small result, while we used the nvvidconv module which should perform scaling operations quickly
Hi alexey,
There is a memcpy in the pipeline:
! "video/x-raw, width=4224, height=3156, format=UYVY" \
! nvvidconv \
! "video/x-raw(memory:NVMM), format=I420, width=1920, height=1080, framerate=20/1" \
It copies CPU buffers to DMA buffers.
We suggest try tegra_multimedia_api, you can refer to 12_camera_v4l2_cuda
and the post https://devtalk.nvidia.com/default/topic/999493/jetson-tx1/nvidia-multimedia-apis-with-uyvy-sensor/post/5117049/#5117049
UPDATE: TK1 does not support it
Danell, can I access tegra multimedia api for Tegra k1? As far as I understood from the software download page, support for this API appeared from L4T version 24.2, but this version of linux is not supported for Tegra k1
[url]https://developer.nvidia.com/embedded/downloads#?search=multimedia%20api[/url]
My apology I did not notice it is TK1. No, tegra_multimedia_api cannot be accessed on TK1.
One more thing you can try is to set max performance Jetson/Performance - eLinux.org
Thanks Danell. I will try this option.