Problems using gstreamer pipeline in openCV for recording video

Autonomous Machines Jetson & Embedded Systems Jetson Nano
,
1

Hi.
I am having trouble with using gstreamer piplelines with python openCV.
I want to record my video with 4032*3040 windows without any auto exposure or gain controls.
Even though I used capture_width=4032 and capture_height=3040, openCV records the video with the same size of display window.

I also like to record the video without external gain controls. If I cover the sensors, I wish the video goes darker. But even though exposuretime and gain is limited, the video do not work as I intended.

My video and the code is attached below.

External Image

import cv2
import datetime
FRAMERATE = 30
def gstreamer_pipeline(
capture_width=4032,
capture_height=3040,
display_width=1280,
display_height=720,
framerate=FRAMERATE,
flip_method=0,
):
return (
"nvarguscamerasrc ! "
"video/x-raw(memory:NVMM), "
"width=(int)%d, height=(int)%d, "
"format=(string)NV12, framerate=(fraction)%d/1, "
"exposuretimerange = "34000 34000", aeLock=true, "
“awblock=true, ispdigitalgainrange="14.72 14.72", gainrange="14.72 14.72" !”
"nvvidconv flip-method=%d ! "
“video/x-raw, width=(int)%d, height=(int)%d, format=(string)BGRx ! "
“videoconvert ! "
“video/x-raw, format=(string)BGR ! appsink”
% (
capture_width,
capture_height,
framerate,
flip_method,
display_width,
display_height,
)
)
def show_camera():
# To flip the image, modify the flip_method parameter (0 and 2 are the most common)
print(gstreamer_pipeline(flip_method=0))
cap = cv2.VideoCapture(gstreamer_pipeline(flip_method=0), cv2.CAP_GSTREAMER)
fourcc = cv2.VideoWriter_fourcc(*‘XVID’)
if cap.isOpened():
window_handle = cv2.namedWindow(“CSI Camera”, cv2.WINDOW_AUTOSIZE)
now = datetime.datetime.now().strftime(”%d_%H-%M-%S”)
record = False
# Window
while cv2.getWindowProperty(“CSI Camera”, 0) >= 0:
ret_val, img = cap.read()
cv2.imshow(“CSI Camera”, img)
# This also acts as
keyCode = cv2.waitKey(30) & 0xFF
# Stop the program on the ESC key
if keyCode == 27:
break
# START record on the “1” key
elif keyCode == 49:
print(“START RECORD”)
record = True
video=cv2.VideoWriter(str(now) + “.avi”, fourcc, FRAMERATE, (img.shape[1], img.shape[0]))
# START record on the “2” key
elif keyCode == 50:
print(“STOP RECORD”)
record = False
video.release()
if record == True:
video.write(img)
cap.release()
cv2.destroyAllWindows()
else:
print(“Unable to open camera”)
if name == “main”:
show_camera()

Thank you for your help.!

3

Hi,
The buffer sending to appsink is in 1280x720. You need to set it to 4032x3040.

4

Thanks. DaneLLL

I got the points.
Is there a way to record 4032*3040 and show it in small sized window simulatneously?

5

Hi,
If you don’t need to do processing to 4032x3040 buffers. You can use tee like:

import sys
import cv2

def read_cam():
    cap = cv2.VideoCapture("nvarguscamerasrc ! video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080,format=(string)NV12, framerate=(fraction)30/1 ! tee name=t t. ! queue ! nvv4l2h264enc ! h264parse ! matroskamux ! filesink location=a.mkv t. ! queue ! nvvidconv ! video/x-raw, format=(string)I420, width=640, height=480 ! appsink sync=0 ")
    if cap.isOpened():
        cv2.namedWindow("demo", cv2.WINDOW_AUTOSIZE)
        while True:
            ret_val, img = cap.read();
            img2 = cv2.cvtColor(img, cv2.COLOR_YUV2BGR_I420);
            cv2.imshow('demo',img2)
            cv2.waitKey(10)
    else:
     print("camera open failed")

    cv2.destroyAllWindows()


if __name__ == '__main__':
    read_cam()
1 Like
6

Thank you for suggestions.

I need to capture 4032*3040, so I modified a little bit from your codes including bitrates.
It works well but when I move it to window platform, it’s framerate decreased to a half (15 fps).
And I am yet having with troubles in auto exposure problems.
I turned on aeLock, awblock and limited exposure, gain, ISPgain to a specific range.
But video still grow blighter when I cover the sensor.
Is there a way to solve this issue?

7

Hi,
For image quality, could you check with the vendor? If you use the camera from our camera partners, you may report issues to them to get further suggestion.

8

I used Raspberry pi HQ camera with IMX477 drivers from Ridgerun

9

Hi flashsquirrel,

Have you contacted with Ridgerun Support: https://www.ridgerun.com/support ?

1 Like
10

hello flashsquirrel,

according to your video clips, it’s AE behavior to adjust the gain/exposure time automatically.
could you please enable the sensor stream with argus_camera application, you may have a try to enable ae_lock for your use-case.
thanks

1 Like
11

Hi @JerryChang and @kayccc

I enabled ae_lock. But AE still behaves.
I will try to contact Ridgerun support.

Thank you for your suggestions and effort.
It was really helped.