Save rgb video using opencv VideoWriter and Gstreamer

Robotics & Edge Computing Jetson Systems Jetson AGX Xavier
, ,
1

This is the code I use to create video writer:

        write_pipe = (
                'appsrc'
                + ' ! video/x-raw, format=BGR'
                + ' ! queue'
                + ' ! videoconvert'
                + ' ! video/x-raw,format=BGRx'
                + ' ! nvvidconv'
                + ' ! nvv4l2h264enc bitrate=14000000'
                + ' ! h264parse'
                + ' ! qtmux'
                + f' ! filesink location={path}'
        )
        return cv2.VideoWriter(
            write_pipe,
            cv2.CAP_GSTREAMER,
            fourcc=0,
            fps=fps,
            frameSize=resolution,
        )

As you can see it uses BGR format which is wrong, because the numpy arrays I have contain data represented as RGB. So the resulting video has blue and red channels swapped. I tried modifying the definition to be:

write_pipe = (
                'appsrc'
                + ' ! video/x-raw, format=RGB'
                + ' ! queue'
                + ' ! videoconvert'
                + ' ! video/x-raw,format=RGBA'
                + ' ! nvvidconv'
                + ' ! nvv4l2h264enc bitrate=14000000'
                + ' ! h264parse'
                + ' ! qtmux'
                + f' ! filesink location={path}'
        )

but this does not work and I get error:

[ WARN:0@2.973] global /mnt/system/opencv/opencv-4.6.0/modules/videoio/src/cap_gstreamer.cpp (2293) writeFrame OpenCV | GStreamer warning: Error pushing buffer to GStreamer pipeline

How to solve this?

2

If you look at opencv 4.6.0 videoio gstreamer backend supported formats, you’ll see that only BGR is supported for 8 bits 3 channels.
Furthermore, VideoWriter doesn’t support 4 bytes formats.

So you would have to use BGR format as source of writer.

Here you would have 2 options:

1. Convert your frame into BGR:

frameBGR = cv2.cvtColor(frame, cv2.COLOR_RGB2BGR)

This would take CPU time.

2. Cheat gstreamer. This is a bit weird for gstreamer coherency, though for your case I’d try:

write_pipe = (
                'appsrc'
                + ' ! video/x-raw, format=BGR'
                + ' ! queue'
                + ' ! multipartmux ! multipartdemux single-stream=1'
                + ' ! video/x-raw, format=RGB'
                + ' ! videoconvert'
                + ' ! video/x-raw,format=RGBA'
                + ' ! nvvidconv'
                + ' ! nvv4l2h264enc bitrate=14000000'
                + ' ! h264parse'
                + ' ! qtmux'
                + f' ! filesink location={path}'
3

@Honey_Patouceul thank you.
Would you be able to explain what you mean by “bit weird for gstreamer coherency”?

4

Well, many gstreamer elements would determine what processing to do depending on the SINK (input) and SRC (output) caps.
My proposal breaks this, using multpartmux/multipartdemux for changing the caps without changing buffers…this is not clean.
Though, in your case, appsrc is sending RGB buffer with BGR caps, so this trick is just intended to restore coherency.

5

@Honey_Patouceul thank you

6

Final note:
If your case has fixed resolution (should be the case with an opencv VideoWriter), you may save the multipart mux/demux overhead using capssetter instead:

 ! capssetter replace=true caps=video/x-raw,format=RGB,width=<your_width>,height=<your_height>, ... specify any non standard framerate, pixel-aspect-ratio, ...
7

@Honey_Patouceul Excellent, thank you. I ended up using capssetter.

8

Happy to help a polite user :-)