Frame lost while saving videos from camera using jetson nano B01

[lolo.oubari.ihssane]

i’m wondering if someone could help me, i’m trying to display and save videos from my flir hadron camera ,i’m using gstreamer in python3 opencv and it succefully displays and saves the IR videos from the BOSON, but i’m losing frames in the RGB videos which leads to slow displaying and fast playback and tried to add various command to the gstreamer piplines such as videorate to control the fps but nothing help .
here is the RGB code for displaying and saving:

import cv2

camset=‘v4l2src io-mode=4 device=/dev/video0 do-timestamp=true ! video/x-raw,format=UYVY,width=1920,height=1080,framerate=30/1 ! queue ! videoconvert ! videorate ! video/x-raw,format=BGR ! appsink drop=1’

cap=cv2.VideoCapture(capset,cv2.CAP_GSTREAMER)
if cap.isOpened() is not True:
print(“Cannot open camera. Exiting.”)
quit()
actual_width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
actual_height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
actual_fps = float(cap.get(cv2.CAP_PROP_FPS));
print(‘Capture opened framing %d x %d @ %f’ % (actual_width,actual_height,actual_fps))

gst_writer_str = ‘appsrc ! video/x-raw,format=BGR,width=1920,height=1080,framerate=30/1 ! queue ! videoconvert ! video/x-raw,format=BGRx ! nvvidconv ! nvv4l2h265enc maxperf-enable=1 preset-level=4 control-rate=1 bitrate=14000000 ! h265parse ! qtmux ! filesink location=streaming.mp4’
fourcc = 0 #RAW
out = cv2.VideoWriter(gst_writer_str, cv2.CAP_GSTREAMER, fourcc, actual_fps, (actual_width, actual_height), True)
if not out.isOpened():
print(“Cannot open writer. Exiting.”)
quit()

while (True):
ret,frame = cap.read()
cv2.imshow(‘camera’,frame)
out.write(frame)
if cv2.waitKey(1) & 0XFF == ord(‘q’):
break
out.release()
cap.release()
cv2.destroyAllWindows()

[DaneLLL]

Hi,
Please try matroskamux plugin. Here is a sample for reference:
Displaying to the screen with OpenCV and GStreamer - #9 by DaneLLL

For using qtmux, you would need to send End-of-Stream signal in termination to get a valid mp4 file.

[lolo.oubari.ihssane]

Hey , i’ve change that thank you but i still have the same problem with frame lost while displayin from the camera and we’ve modified the script a little to figure exactly the part that slowing the display and it was the reading from the camera ret,frame = cap.read()
we’ve also noticed that the jetson nano is only using CPU and now i’m trying to install opencv with Cuda and cudnn to be able to use the GPU to process the frames . do you have any idea about if i need to add any lines in my code to force the using of GPU ?
also thank you for response i appreciate it

[DaneLLL]

Hi,
Please run sudo jetson_clocks to enable CPU cores always at maximum clock. This is optimal throughput of CPU cores on Jetson Nano. OpenCV mainly runs on CPU and performance may be capped by CPU capability. If running at maximum clock still cannot achieve target performance, we would suggest use gstreamer or jetson_multiemdia_api to developer your use-case. The two frameworks use hardware DMA buffer and additional memory copy is eliminated to get maximum throughput.

[lolo.oubari.ihssane]

Hey, i really appreciate your help. I’ve tried the sudo jetson_clocks but it didn’t help. it’s an opencv problem because when i execute the same gstreamer pipline in the terminal the display is fluide but in the python3 script i’m losing frame in the read() part i truly have no idea how to solve this problem to stop losing frames. i have to work on opencv because there’s going to be some AI running for the videos idon’t have a choice

[DaneLLL]

Hi,
If your use-case is to use CUDA filters in OpenCV, we suggest use jetson_multimedia_api + mapping NvBuffer to gpuMat. Please check this patch:
LibArgus EGLStream to nvivafilter - #14 by DaneLLL

And please check VPI functions:
VPI - Vision Programming Interface: Main Page

We support some CV functions in VPI, if your use-case is included, please try jetson_multimedia_api + VPI.