Gstreamer nvv4l2camerasrc plugin resolution problem

jerry.liu21023 · April 20, 2023, 9:03am

I use a 5M camera and try to bring up with FULL resolution(2592x1944) by Gstreamer.

This is my pipeline with nvv4l2camerasrc:

gst-launch-1.0 nvv4l2camerasrc bufapi-version=TRUE device=/dev/video0 ! \
'video/x-raw(memory:NVMM),format=UYVY,width=2592,height=1944' ! nvvideoconvert ! 'video/x-raw(memory:NVMM)' !  nv3dsink sync=false

However, I got the wrong image:

Then I modified the pipeline with v4l2src:

gst-launch-1.0 v4l2src device=/dev/video0 ! \
'video/x-raw,format=UYVY,width=2592,height=1944' ! nvvideoconvert ! 'video/x-raw(memory:NVMM)' !  nv3dsink sync=false

I can capture the correct image with the pipeline.

I think there is a potential problem in nvv4l2camerasrc when I use specific resolutions.

I also tried 2560x1440, 1920x1080, and 1280x720. They can work with nvv4l2camerasrc.

How could I fix the problem if I still want to use nvv4l2camerasrc to bring up the full resolution in my pipeline?

JerryChang · April 21, 2023, 2:45am

hello jerry.liu21023,

may I know what’s the sensor format dumps. $ v4l2-ctl -d /dev/video0 --list-formats-ext

jerry.liu21023 · April 21, 2023, 2:56am

Hi Jerry

$ v4l2-ctl -d /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
        Index       : 0
        Type        : Video Capture
        Pixel Format: 'UYVY'
        Name        : UYVY 4:2:2
                Size: Discrete 1280x720
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1920x1080
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 2560x1440
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 2592x1944
                        Interval: Discrete 0.033s (30.000 fps)

        Index       : 1
        Type        : Video Capture
        Pixel Format: 'NV16'
        Name        : Y/CbCr 4:2:2
                Size: Discrete 1280x720
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1920x1080
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 2560x1440
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 2592x1944
                        Interval: Discrete 0.033s (30.000 fps)

        Index       : 2
        Type        : Video Capture
        Pixel Format: 'UYVY'
        Name        : UYVY 4:2:2
                Size: Discrete 1280x720
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1920x1080
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 2560x1440
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 2592x1944
                        Interval: Discrete 0.033s (30.000 fps)

JerryChang · April 21, 2023, 6:02am

how about running gst pipeline with nvv4l2camerasrc as below.
$ gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! 'video/x-raw(memory:NVMM), format=(string)UYVY, width=(int)2592, height=(int)1944, framerate=(fraction)30/1, interlace-mode=progressive' ! nvvidconv ! nv3dsink -e

jerry.liu21023 · April 21, 2023, 7:56am

No, it cannot work.

$ gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! 'video/x-raw(memory:NVMM), format=(string)UYVY, width=(int)2592, height=(int)1944, framerate=(fraction)30/1, interlace-mode=progressive' ! nvvidconv ! nv3dsink -e
nvbuf_utils: Could not get EGL display connection
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock

(gst-launch-1.0:9173): GLib-CRITICAL **: 15:56:29.510: g_array_remove_index: assertion 'index_ < array->len' failed

(gst-launch-1.0:9173): GStreamer-CRITICAL **: 15:56:29.510: gst_caps_remove_structure: assertion 'idx <= gst_caps_get_size (caps)' failed

(gst-launch-1.0:9173): GStreamer-CRITICAL **: 15:56:29.510: gst_caps_remove_structure: assertion 'idx <= gst_caps_get_size (caps)' failed

(gst-launch-1.0:9173): GStreamer-CRITICAL **: 15:56:29.510: gst_structure_set_parent_refcount: assertion 'refcount != NULL' failed

(gst-launch-1.0:9173): GStreamer-CRITICAL **: 15:56:29.510: gst_caps_features_set_parent_refcount: assertion 'refcount != NULL' failed
free(): invalid pointer
Aborted (core dumped)

JerryChang · April 21, 2023, 8:16am

hello jerry.liu21023,

may I know which Jetpack release version you’re working with.
also, please share the failures from the kernel side. thanks

jerry.liu21023 · April 21, 2023, 8:46am

Hi Jerry,

We use JetPack 4.6.1

These are kernel messages after I run your pipeline.

[ 2935.891654] tevi-ap1302 30-003d: sensor_set_mode() , {3}, fmt_width=2592, fmt_height=1944
[ 2935.891666] tevi-ap1302 30-003d: sensor_start_streaming()
[ 2935.895498] [RCE] vi5_hwinit: firmware CL2018101701 protocol version 2.2
[ 2935.939406] tegra194-vi5 15c10000.vi: video_name: vi-output, corr_err: discarding frame 0, flags: 0, err_data 512
[ 2936.065842] tegra194-vi5 15c10000.vi: video_name: vi-output, corr_err: discarding frame 0, flags: 32, err_data 160
[ 2938.787451] tegra194-vi5 15c10000.vi: no reply from camera processor
[ 2938.787641] tegra194-vi5 15c10000.vi: video_name: vi-output, uncorr_err: request timed out after 2500 ms
[ 2938.787844] tegra194-vi5 15c10000.vi: err_rec: attempting to reset the capture channel
[ 2938.790832] tegra194-vi5 15c10000.vi: err_rec: successfully reset the capture channel
[ 2938.791085] tevi-ap1302 30-003d: sensor_stop_streaming()

JerryChang · April 24, 2023, 1:57am

hello jerry.liu21023,

it shows VI engine to discard frames, the error reported channel encountered an uncorrectable error and it must to be reset.
it might also due to frame got corrupted, those illegal data packet dropped by CSIMUX.

is it correct frame-rate reported by list-formats-ext?
please also examine by v4l standard ioctl to access the buffers. it’ll report the fps below the pipeline.
for example,
$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=2592,height=1944,pixelformat=UYVY --set-ctrl bypass_mode=0 --stream-mmap --stream-count=100

jerry.liu21023 · April 24, 2023, 5:53am

Hi Jerry,

The vi error only occurred the first time opening camera and it can stream after resetting the capture channel.

is it correct frame-rate reported by list-formats-ext ?

Yes, the resolution is correct but the real framerate varies depending on the influence of our camera’s setting.

There is the result and it can capture stream and get the fps:

$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=2592,height=1944,pixelformat=UYVY --set-ctrl bypass_mode=0 --stream-mmap --stream-count=100
<<<<<<<<<<<<<<<< 14.42 fps
<<<<<<<<<<<<<< 14.35 fps
<<<<<<<<<<<<<<< 14.37 fps
<<<<<<<<<<<<<< 14.35 fps
<<<<<<<<<<<<<< 14.37 fps
<<<<<<<<<^C

JerryChang · April 24, 2023, 6:13am

hello jerry.liu21023,

according to v4l standard ioctl to access the buffers, it’s reported 14-fps.
there’re some potential issue for frame-rate not reaching to 30, for example, it may due to the pixel clock settings you’ve given is too low. or, some exposure settings to limit the frame-rate capability…etc.

anyways, please check the camera stream functionality with the following pipeline.
for example,
$ gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! 'video/x-raw(memory:NVMM), format=(string)UYVY, width=(int)2592, height=(int)1944, framerate=(fraction)14/1, interlace-mode=progressive' ! nvvidconv ! nv3dsink -e

jerry.liu21023 · April 24, 2023, 9:50am

Hi Jerry,

I think it is not clock or exposure issues because I can use v4l2src to get the correct image from the streaming as I mention.

Correct Image by using v4l2src:

gst-launch-1.0 v4l2src device=/dev/video0 ! \
'video/x-raw, format=(string)UYVY, width=(int)2592, height=(int)1944, framerate=(fraction)30/1, interlace-mode=progressive' ! \
nvvidconv ! 'video/x-raw(memory:NVMM)' ! nv3dsink -e

Wrong Image by using nvv4l2camerasrc:

gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! \
'video/x-raw(memory:NVMM), format=(string)UYVY, width=(int)2592, height=(int)1944, framerate=(fraction)30/1, interlace-mode=progressive' ! \
nvvidconv ! 'video/x-raw(memory:NVMM)' ! nv3dsink -e

These two pipelines just are different from source element. I don’t think it is problem with our driver or camera setting, otherwise, I cannot get the correct image from v4l2src.

JerryChang · April 25, 2023, 12:37am

hello jerry.liu21023,

had you tried reduce the framerate property for setting to 14-fps?

jerry.liu21023 · April 25, 2023, 1:15am

Hi Jerry,

I modified driver to make camera reach up to 20 fps.

$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=2592,height=1944,pixelformat=UYVY --set-ctrl bypass_mode=0 --stream-mmap --stream-count=100
<<<<<<<<<<<<<<<<<<<<< 20.00 fps
<<<<<<<<<<<<<<<<<<<< 20.00 fps
<<<<<<<<<<<<<<<<<<<< 20.00 fps
<<<<<<<<<<<<<<<<<<<< 20.00 fps

However, it still is the same results as I mention.
As long as I use v4l2src, I can get the correct image, whereas if I use nvv4l2camerasrc, it will always be an incorrect image.

Correct Image by v4l2src:

gst-launch-1.0 v4l2src device=/dev/video0 ! \
'video/x-raw, format=(string)UYVY, width=(int)2592, height=(int)1944, framerate=(fraction)20/1, interlace-mode=progressive' ! \
nvvidconv ! 'video/x-raw(memory:NVMM)' ! nv3dsink -e

Incorrect Image by nvv4l2camerasrc:

gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! \
'video/x-raw(memory:NVMM), format=(string)UYVY, width=(int)2592, height=(int)1944, framerate=(fraction)20/1, interlace-mode=progressive' ! \
nvvidconv ! 'video/x-raw(memory:NVMM)' ! nv3dsink -e

JerryChang · April 25, 2023, 1:56am

hello jerry.liu21023,

thanks for sharing test results.
it may also due to the stride alignment, which should be followed VI’s 64 byte aligned to set the correct stride, by setting the width alignment to 64.
there’s CID controls to configure this, please try $ v4l2-ctl --set-ctrl preferred_stride=2624 before enabling the preview stream.

jerry.liu21023 · April 25, 2023, 2:06am

Hi Jerry,

I tried it but it doesn’t seem to have any effect.

$ v4l2-ctl --set-ctrl preferred_stride=2624
$ gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! 'video/x-raw(memory:NVMM), format=(string)UYVY, width=(int)2592, height=(int)1944, framerate=(fraction)20/1, interlace-mode=progressive' ! nvvidconv ! 'video/x-raw(memory:NVMM)' ! nv3dsink -e

JerryChang · April 25, 2023, 2:43am

please see-also, YUV camera(5.4M) preview issue

jerry.liu21023 · April 26, 2023, 7:02am

Hi Jerry,

Thanks for your reply!

So it looks like the constraint of data alignment for NvBuffer?

should the image line length be 256 bytes of alignment?

According to that, for our case, we should modify the width to 2560 or 2688

I tried to modify the width of the resolution for our camera:

2560x1944:

$ gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! 'video/x-raw(memory:NVMM), format=(string)UYVY, width=(int)2560, height=(int)1944, framerate=(fraction)20/1, interlace-mode=progressive' ! nvvidconv ! 'video/x-raw(memory:NVMM)' ! nv3dsink -e

2688x1944:

$ gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! 'video/x-raw(memory:NVMM), format=(string)UYVY, width=(int)2688, height=(int)1944, framerate=(fraction)20/1, interlace-mode=progressive' ! nvvidconv ! 'video/x-raw(memory:NVMM)' ! nv3dsink -e

Both pipelines can get the correct image.

Anyway, if we want to get a correct image of full-size resolution(2592x1944), we should capture the frame buffer to CPU buffer, then copy it to Nvbuffer like the below pipeline despite high CPU usage.

$ gst-launch-1.0 v4l2src device=/dev/video0 ! \
'video/x-raw,format=UYVY,width=2592,height=1944' ! nvvidconv ! 'video/x-raw(memory:NVMM),width=2592,height=1944' ! nv3dsink sync=false

JerryChang · April 26, 2023, 7:47am

hello jerry.liu21023,

that’s due to VI’s alignment, the buffer width should be alignment to 64.
you may modify the buffer size as 2688x1944 for capturing full-size resolution (2592x1944), and cropping redundant pixels by 2D engine.

system · May 17, 2023, 6:26am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Convert Image Jetson AGX Xavier camera	10	2170	October 18, 2021
Video scaling with GStreamer element 'videoscale' and 'nvdrmvideosink set-mode=1' Jetson TX2 gstreamer , video	18	8705	March 5, 2022
Gstreamer fails with nvivafilter and nvargus: "NvRmChannelSubmit: NvError_IoctlFailed with error code 22" Jetson Nano gstreamer , nvbugs	16	467	May 8, 2024
High CPU usage for video capturing Jetson Xavier NX gstreamer	18	3356	October 18, 2021
Camera's frame rate unstable Jetson AGX Xavier camera	22	4600	October 18, 2021
Gstreamer encode h.264 got frame drops,can not support 1080p@60fps Jetson Xavier NX camera , gstreamer	6	2079	November 17, 2021
Help with gstreamer speed improvement via using userptr buffer allocations Jetson TX2	10	4506	May 17, 2018
Problems with gstreamer video decoding pipeline Jetson AGX Xavier	11	5325	October 18, 2021
Streaming FLIR Boson video using Gstreamer 1.0 in Jetson TX2 Jetson TX2 gstreamer	8	2218	October 18, 2021
Gstreamer Pipeline issue : PosixMemMap:84 mmap failed Jetson Orin Nano gstreamer	15	682	April 25, 2024

Gstreamer nvv4l2camerasrc plugin resolution problem

Related topics