Frame writing delay after creating cv2.VideoWriter() object

Hello.
In our project we write several minutes length videos to RAM. Simplified code for recording video looks like this:

import datetime
import cv2
import time


def get_gstreamer_output_pipeline(video_file_name, path_to_video_folder):
    return ('appsrc ! '
            'video/x-raw, format=BGR ! '
            'queue ! '
            'videoconvert ! '
            'video/x-raw,format=RGBA ! '
            'nvvidconv ! '
            'nvv4l2h264enc ! '
            'h264parse ! '
            'qtmux ! '
            'filesink location=%s%s ' % (path_to_video_folder, video_file_name))


def main():
    path_to_RAM_videos_folder = '/run/user/120/videos/'

    gstreamer_pipeline = (
        'nvarguscamerasrc exposuretimerange="33000000 33000000" gainrange="2 2" ispdigitalgainrange="4 4" wbmode=0 ! '
        'video/x-raw(memory:NVMM), width=(int)1080, height=(int)720, format=(string)NV12, framerate=(fraction)30/1 ! '
        'nvvidconv ! '
        'video/x-raw, format=(string)BGRx ! '
        'videoconvert ! '
        'video/x-raw, format=(string)BGR ! '
        'appsink')

    video_capturer = cv2.VideoCapture(gstreamer_pipeline, cv2.CAP_GSTREAMER)

    current_date = datetime.datetime.now()
    video_name = current_date.strftime('%Y%m%d_%H%M%S.avi')
    gst_out = get_gstreamer_output_pipeline(video_name, path_to_RAM_videos_folder)
    out = cv2.VideoWriter(gst_out, cv2.CAP_GSTREAMER, 0, 30.0, (int(1080), int(720)))

    while (video_capturer.isOpened()):
        tic_1 = time.time()
        if (datetime.datetime.now() - current_date).total_seconds() > N * 60:
            out.release()

            current_date = datetime.datetime.now()
            video_name = current_date.strftime('%Y%m%d_%H%M%S.avi')
            gst_out = get_gstreamer_output_pipeline(video_name, path_to_RAM_videos_folder)
            out = cv2.VideoWriter(gst_out, cv2.CAP_GSTREAMER, 0, 30.0, (int(1080), int(720)))
        toc_1 = time.time()

        # Reading frame from camera:
        tic_2 = time.time()
        _, frame = video_capturer.read()
        toc_2 = time.time()

        tic_3 = time.time()
        # Some code.
        toc_3 = time.time()

        # Writing frame to video file:
        tic_4 = time.time()
        out.write(frame)
        toc_4 = time.time()

        # Checking time delays:
        if (toc_1 - tic_1 > 0.04 or
                toc_2 - tic_2 > 0.04 or
                toc_3 - tic_3 > 0.04 or
                toc_4 - tic_4 > 0.04):
            time_expenses = 'Main thread t1:' + str(round(toc_1 - tic_1, 4)) + \
                            '|t2:' + str(round(toc_2 - tic_2, 4)) + \
                            '|t3:' + str(round(toc_3 - tic_3, 4)) + \
                            '|t4:' + str(round(toc_4 - tic_4, 4))

        # Writing to the log file:
        logger_file.debug(time_expenses)

    tic_1 = toc_1 = tic_2 = toc_2 = tic_3 = toc_3 = tic_4 = toc_4 = 0

The following lines rarely appear in the log file (1-2 times a day):

[2021-08-23 16:09:10] DEBUG  Main thread t1:0.0647|t2:0.0009|t3:0.017|t4:0.0008 
[2021-08-23 16:09:12] DEBUG  Main thread t1:0.0|t2:0.0009|t3:0.021|t4:1.7238

t1 in first row is time spended for creating VideoWriter() object, this is normal. However, in the second line, there is a delay for recording the frame (t4). I. e., the first one or more frames after the object created are recorded in a short time, and the next frame delays the cycle for a few seconds. And such cases appeares 1-2 times a day.

The problem will not be repeated if I start recording short videos in isolation for a long time (without all application’s functional). Our python application takes 165/400% of CPU time and nvargus-daemon takes 20/400% CPU (output of “top” command).

How can we find reason of such delays and how to solve it? Thank you.

Hi,
It is possible to see delay if you run OpenCV on Jetson Nano. There is discussion in
[Gstreamer] nvvidconv, BGR as INPUT - #2 by DaneLLL

We would suggest use gstreamer or jetson_multiemdia_api on Jetson Nano. If you have to use OpenCv, please execute sudo nvpmodel -m 0 and sudo jetson_clocks, to run CPU cores at max clocks.