How to speed up Video Recording with Python cv2.VideoWriter on Jetson Nano

[puffvayne]

I am trying to do video recording on Jetson Nano. I found that when the cv2.VideoWriter using write() the program becomes slow. How can I speed up the cv2.VideoWriter? Maybe use Gstreamer or GPU or nvidia-component can speed up the FPS.

My Code:

import os
import time
import cv2

width = 2560
height = 1440
framerate = 30
video_path = 'fps_test.mp4'

gs_pipeline = f"v4l2src device=/dev/video0 io-mode=2 " \
              f"! image/jpeg, width={width}, height={height}, framerate={framerate}/1, format=MJPG " \
              f"! nvv4l2decoder mjpeg=1 " \
              f"! nvvidconv flip-method=4  " \
              f"! video/x-raw, format=BGRx " \
              f"! videoconvert " \
              f"! video/x-raw, format=BGR " \
              f"! appsink drop=1"

gs_pipeline = gs_pipeline
print(f"gst-launch-1.0 {gs_pipeline}\n")

v_cap = v_writer = None


def main():
    global v_cap, v_writer
    codec = cv2.VideoWriter_fourcc(*'mp4v')
    v_writer = cv2.VideoWriter(video_path, codec, 20, (width, height))
    v_cap = cv2.VideoCapture(gs_pipeline, cv2.CAP_GSTREAMER)

    prev_frame_fetched_time = 0
    while v_cap.isOpened():
        ret_val, frame = v_cap.read()
        if not ret_val:
            break

        v_writer.write(frame)

        curr_frame_fetched_time = time.time()
        curr_fps = 1 / (curr_frame_fetched_time - prev_frame_fetched_time)
        prev_frame_fetched_time = curr_frame_fetched_time

        print(f"FPS = {curr_fps:.3f}")

    v_cap.release()
    cv2.destroyAllWindows()


if __name__ == '__main__':
    try:
        main()
    except KeyboardInterrupt:  # If CTRL+C is pressed, exit cleanly:
        pass
    finally:
        v_cap.release()
        cv2.destroyAllWindows()
        if os.path.exists(video_path):
            os.remove(video_path)

Console:

FPS = 0.000
FPS = 2.438
FPS = 3.606
FPS = 2.805
FPS = 2.741
FPS = 3.265
FPS = 3.603
FPS = 3.524
FPS = 3.290

[Honey_Patouceul]

Your VideoWriter to mp4v is probably using FFMPEG backend using CPU MP4V encoding.
On Jetson, you may use some HW encoder, but for Nano you may only have H264, H265, VP8 or VP9…
So you may try h264 into qtmux(mp4-like) container using gstreamer backend with;

gst_writer_str = 'appsrc ! video/x-raw,format=BGR,width={width},height={height},framerate={framerate}/1 ! queue ! videoconvert ! video/x-raw,format=BGRx ! nvvidconv ! nvv4l2h264enc insert-vui=1 ! h264parse ! qtmux ! filesink location=test_h264.mp4'
v_writer = cv2.VideoWriter(gst_writer_str, cv2.CAP_GSTREAMER, 0, float({framerate}), (int({width}), int({height}))
if not v_writer.isOpened():
   ... Error

# if you get here loop reading frames from capture and send these into writer... 

Note that you don’t need opencv if just transcoding, only if you want to process frames in between. A pure gstreamer pipline may better perform:

gst-launch-1.0 v4l2src device=/dev/video0 io-mode=2 ! image/jpeg, width={width}, height={height}, framerate={framerate}/1, format=MJPG ! nvv4l2decoder mjpeg=1 ! nvvidconv flip-method=4 ! nvv4l2h264enc insert-vui=1 ! h264parse ! qtmux ! filesink location=test_h264.mp4

Also note that your timing may not be accurate, you may cut into:

• time waiting for a frame to be available (this may vary with available time after frame processing if any)
• time for actually reading frame (should no vary so much)
• time to process frame if any (depends on what you do with frames)
• time to send frame to writer (should not vary so much)

[puffvayne]

1. first with the given pipline with python cv2, no file is created
2. I think we don’t need to specify the resolution and framerate again
3. What does insert-vui=1 do?

[Honey_Patouceul]

Sorry, I had forgotten the BGR->BGRx conversion in writer above, I’ve edited it. So you would try:

gst_writer_str = 'appsrc ! video/x-raw,format=BGR,width=2560,height=1440,framerate=30/1 ! queue ! videoconvert ! video/x-raw,format=BGRx ! nvvidconv ! nvv4l2h264enc insert-vui=1 ! h264parse ! qtmux ! filesink location=test_h264.mp4'
v_writer = cv2.VideoWriter(gst_writer_str, cv2.CAP_GSTREAMER, 0, float(30), (int(2560), int(1440)))
if not v_writer.isOpened():
     ...Error case