GStreamer warning: Embedded video playback halted; module v4l2src7 reported: Failed to allocate required memory

Robotics & Edge Computing Jetson & Embedded Systems Jetson Xavier NX
, ,
1

Hello,

I am trying to capture full resolution (3840x2160) images from my Spinel UC80MPD USB 2.0 camera on my Xavier NX and am experiencing some errors. Specifically, I’ve tried instantiating an OpenCV VideoCapture object in the following two ways:

cap = cv2.VideoCapture(0)
and
cap = cv2.VideoCapture('v4l2src device=/dev/video0 ! video/x-raw, width=(int)3840, height=(int)2160 ! videoconvert ! video/x-raw, format=(string)BGR ! appsink', cv2.CAP_GSTREAMER)

But, in both cases, I am encountering the following error messages:

[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (1757) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module v4l2src7 reported: Failed to allocate required memory.
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (886) open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created

If I instantiate the VideoCapture object at a resolution (supported by the camera) that is lower than 1920x1080, there are no issues whatsoever. This is strange because on the same Xavier NX I am able to capture full resolution (3264x2448) images using a Spinel UC80MPA without issue.

I’ve tested instantiating a VideoCapture object on my laptop with this new camera and have not had any warnings or errors, and am able to capture full-resolution images.

Does anyone have any thoughts on what the issue here might be and what steps I can take to fix it?

1 Like
3

Could you check if v4l2-ctl can capture 3264x2448?
Also does it YUV format? Can the gst-launch-1.0 v4l2src … working?

1 Like
4

This camera can not capture 3264x2448 (that’s the other camera that is working as expected). See below for the list of supported resolutions and formats for the camera in question:

[video4linux2,v4l2 @ 0x55870a9600] Compressed:       mjpeg :          Motion-JPEG : 3840x2160 2592x1944 2048x1536 1600x1200 1920x1080 1280x960 1280x720 800x600 640x480 320x240
[video4linux2,v4l2 @ 0x55870a9600] Raw       :     yuyv422 :           YUYV 4:2:2 : 3840x2160 2592x1944 2048x1536 1600x1200 1920x1080 1280x960 1280x720 800x600 640x480 320x240

Could you elaborate on what gst-launch-1.0 command I should check?

5

I tried this command:

gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw, width=3840, height=2160 ! videoconvert ! video/x-raw, format=BGR ! nvoverlaysink

Which yielded the following error:

Setting pipeline to PAUSED ...                                                                                                                                                            │··
Pipeline is live and does not need PREROLL ...                                                                                                                                            │··
Setting pipeline to PLAYING ...                                                                                                                                                           │··
New clock: GstSystemClock                                                                                                                                                                 │··
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed to allocate required memory.                                                                                       │··
Additional debug info:                                                                                                                                                                    │··
gstv4l2src.c(658): gst_v4l2src_decide_allocation (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:                                                                                          │··
Buffer pool activation failed                                                                                                                                                             │··
Execution ended after 0:00:00.104024708                                                                                                                                                   │··
Setting pipeline to PAUSED ...                                                                                                                                                            │··
Setting pipeline to READY ...                                                                                                                                                             │··
Setting pipeline to NULL ...                                                                                                                                                              │··
Freeing pipeline ...

Same memory allocation error.

7

I was able to solve this problem by instantiating the video capture object in the following way:

cap = cv2.VideoCapture(f'v4l2src device=/dev/video0 io-mode=2 ! image/jpeg, width=(int)3840, height=(int)2160 !  nvjpegdec ! video/x-raw, format=I420 ! appsink', cv2.CAP_GSTREAMER)

Upon further research into the specs of the camera (by running v4l2-ctl --info -d /dev/video0 --list-formats-ext) I was able to determine that, when capturing raw frames at full resolution, the max supported frame rate is 1 FPS (see below):

        Index       : 1                                                                                                                                                                                                                                                                                                                                                              │······
        Type        : Video Capture                                                                                                                                                                                                                                                                                                                                                  │······
        Pixel Format: 'YUYV'                                                                                                                                                                                                                                                                                                                                                         │······
        Name        : YUYV 4:2:2                                                                                                                                                                                                                                                                                                                                                     │······
                Size: Discrete 3840x2160                                                                                                                                                                                                                                                                                                                                             │······
                        Interval: Discrete 1.000s (1.000 fps)                                                                                                                                                                                                                                                                                                                        │······
                Size: Discrete 2592x1944                                                                                                                                                                                                                                                                                                                                             │······
                        Interval: Discrete 1.000s (1.000 fps)                                                                                                                                                                                                                                                                                                                        │······
                Size: Discrete 2048x1536                                                                                                                                                                                                                                                                                                                                             │······
                        Interval: Discrete 0.333s (3.000 fps)                                                                                                                                                                                                                                                                                                                        │······
                Size: Discrete 1600x1200                                                                                                                                                                                                                                                                                                                                             │······
                        Interval: Discrete 0.200s (5.000 fps)                                                                                                                                                                                                                                                                                                                        │······
                Size: Discrete 1920x1080                                                                                                                                                                                                                                                                                                                                             │······
                        Interval: Discrete 0.200s (5.000 fps)                                                                                                                                                                                                                                                                                                                        │······
                Size: Discrete 1280x960                                                                                                                                                                                                                                                                                                                                              │······
                        Interval: Discrete 0.200s (5.000 fps)                                                                                                                                                                                                                                                                                                                        │······
                Size: Discrete 1280x720                                                                                                                                                                                                                                                                                                                                              │······
                        Interval: Discrete 0.100s (10.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.200s (5.000 fps)                                                                                                                                                                                                                                                                                                                        │······
                Size: Discrete 800x600                                                                                                                                                                                                                                                                                                                                               │······
                        Interval: Discrete 0.050s (20.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.067s (15.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.100s (10.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.200s (5.000 fps)                                                                                                                                                                                                                                                                                                                        │······
                Size: Discrete 640x480                                                                                                                                                                                                                                                                                                                                               │······
                        Interval: Discrete 0.033s (30.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.040s (25.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.050s (20.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.067s (15.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.100s (10.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.200s (5.000 fps)                                                                                                                                                                                                                                                                                                                        │······
                Size: Discrete 320x240                                                                                                                                                                                                                                                                                                                                               │······
                        Interval: Discrete 0.033s (30.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.040s (25.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.050s (20.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.067s (15.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.100s (10.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.200s (5.000 fps)

However, using MJPEG, it is possible to capture full resolution images at 30 FPS (see below):

        Index       : 0                                                                                                                                                                                                                                                                                                                                                              │······
        Type        : Video Capture                                                                                                                                                                                                                                                                                                                                                  │······
        Pixel Format: 'MJPG' (compressed)                                                                                                                                                                                                                                                                                                                                            │······
        Name        : Motion-JPEG                                                                                                                                                                                                                                                                                                                                                    │······
                Size: Discrete 3840x2160                                                                                                                                                                                                                                                                                                                                             │······
                        Interval: Discrete 0.033s (30.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.040s (25.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.050s (20.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.067s (15.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.100s (10.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.200s (5.000 fps)                                                                                                                                                                                                                                                                                                                        │······
                Size: Discrete 2592x1944                                                                                                                                                                                                                                                                                                                                             │······
                        Interval: Discrete 0.033s (30.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.040s (25.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.050s (20.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.067s (15.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.100s (10.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.200s (5.000 fps)                                                                                                                                                                                                                                                                                                                        │······
                Size: Discrete 2048x1536                                                                                                                                                                                                                                                                                                                                             │······
                        Interval: Discrete 0.033s (30.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.040s (25.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.050s (20.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.067s (15.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.100s (10.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.200s (5.000 fps)                                                                                                                                                                                                                                                                                                                        │······
                Size: Discrete 1600x1200                                                                                                                                                                                                                                                                                                                                             │······
                        Interval: Discrete 0.033s (30.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.040s (25.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.050s (20.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.067s (15.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.100s (10.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.200s (5.000 fps)                                                                                                                                                                                                                                                                                                                        │······
                Size: Discrete 1920x1080                                                                                                                                                                                                                                                                                                                                             │······
                        Interval: Discrete 0.033s (30.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.040s (25.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.050s (20.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.067s (15.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.100s (10.000 fps)                                                                                                                                                                                                                                                                                                                       │······
                        Interval: Discrete 0.200s (5.000 fps)

So, even if I solve the memory allocation error, I wouldn’t be able to capture frames at the required frame rate. As such, initializing the capture object with MJPEG solved the problem for me.