Save v4l2 camera video as mp4 using Gstreamer, opencv, Python

Robotics & Edge Computing Jetson Systems Jetson AGX Orin
,
1

I am trying to save video from a camera to h264 mp4 file. I already made sure below Gstreamer pipeline works as expected.

gst-launch-1.0 -e v4l2src device=/dev/video0 ! "video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)30/1" ! nvvidconv ! "video/x-raw, format=(string)NV12, framerate=(fraction)30/1" ! nvvidconv ! "video/x-raw(memory:NVMM), format=(string)NV12, framerate=(fraction)30/1" ! queue ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=0.mp4

To reduce resource usage, I want to use NVIDIA hardware encoder as much as possble.

Now, I want to do the same thing on Python and opencv. The code I am trying is below:

import cv2

gst_cap = "v4l2src device=/dev/video0 ! nvvidconv ! appsink"

gst_writer = "appsrc ! nvvidconv ! video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)30/1 ! nvvidconv ! video/x-raw, format=(string)NV12, framerate=(fraction)30/1 ! nvvidconv ! video/x-raw(memory:NVMM), format=(string)NV12, framerate=(fraction)30/1 ! nvv4l2h264enc ! video/x-h264, stream-format=(string)byte-stream ! h264parse ! qtmux ! filesink location=test/0.mp4"

cap = cv2.VideoCapture(gst_cap)
out = cv2.VideoWriter(gst_writer, 0, 30, (1920, 1080), False)

try:
    while True:
        ret, frame = cap.read()
        if ret:
            out.write(frame)

except KeyboardInterrupt:
    cap.release()
    out.release()

This does not work. The output file is 0 byte. The log is below:

[ WARN:0@0.901] global cap_gstreamer.cpp:1728 open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
OpenCV: FFMPEG: tag 0x00000000/'????' is not supported with codec id 13 and format 'mp4 / MP4 (MPEG-4 Part 14)'
Opening in BLOCKING MODE 
NvMMLiteOpen : Block : BlockType = 4 
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4 
gst_nvvconv_transform: Transform not supported 
[ WARN:0@0.982] global cap_gstreamer.cpp:2784 handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module appsrc0 reported: Internal data stream error.
[ WARN:0@0.987] global cap_gstreamer.cpp:2676 writeFrame OpenCV | GStreamer warning: Error pushing buffer to GStreamer pipeline
[ WARN:0@1.002] global cap_gstreamer.cpp:2676 writeFrame OpenCV | GStreamer warning: Error pushing buffer to GStreamer pipeline
[ WARN:0@1.022] global cap_gstreamer.cpp:2676 writeFrame OpenCV | GStreamer warning: Error pushing buffer to GStreamer pipeline

Can you please guide me to the correct way?

Thank you,

3

Hi,
Please refer to the sample and give it a try:
Displaying to the screen with OpenCV and GStreamer - #9 by DaneLLL

In the sample it encodes to mkv. You may confirm mkv works first and then try mp4.