OpenCV - gstreamer use 100% CPU and 0% GPU

Robotics & Edge Computing Jetson & Embedded Systems Jetson Nano
,
1

I have compiled OpenCV with support gstreamer support using this script: JEP/install_opencv4.5.0_Jetson.sh at master · AastaNV/JEP · GitHub

When I execute the following code:

import cv2

mainVideoResolution = (2592, 1944)
thumbnailVideoResolution = (640, 480)

cap = cv2.VideoCapture("nvarguscamerasrc ! video/x-raw(memory:NVMM), width=(int)2592, height=(int)1944,format=(string)NV12, framerate=(fraction)30/1 ! nvvidconv ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! queue ! appsink drop=1", cv2.CAP_GSTREAMER)
print(cap)
if not cap.isOpened():
   print('Failed to open camera')
   exit

mainVideo = cv2.VideoWriter("appsrc ! queue ! videoconvert ! video/x-raw,format=BGRx ! nvvidconv ! nvv4l2h264enc insert-sps-pps=1 insert-vui=1 ! rtph264pay ! udpsink host=127.0.0.1 port=5004", cv2.CAP_GSTREAMER, 0, 30.0, mainVideoResolution) 
if not mainVideo.isOpened():
   print('Failed to open mainVideo')
   cap.release()
   exit

thumbnailVideo = cv2.VideoWriter("appsrc ! queue ! videoconvert ! video/x-raw,format=BGRx ! nvvidconv ! video/x-raw(memory:NVMM),width=320,height=240 ! nvv4l2h264enc insert-sps-pps=1 insert-vui=1 ! rtph264pay ! udpsink host=127.0.0.1 port=5005", cv2.CAP_GSTREAMER, 0, 30.0, thumbnailVideoResolution) 
if not thumbnailVideo.isOpened():
   print('Failed to open rtpudp82')
   thumbnailVideo.release()
   cap.release()
   exit


while True:
	ret_val, img = cap.read()
	if not ret_val:
		break

	mainVideo.write(img)
	thumbnailVideo.write(img)



mainVideo.release()
thumbnailVideo.release()
cap.release()

The CPU usage is almost 100% but GPU usage is about 0%. Why that happens if I use gstreamer which supports hardware acceleration.
If execute similar code:

gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM), format=NV12, width=2592, height=1944' ! tee name=t ! queue ! nvv4l2h264enc insert-sps-pps=true ! h264parse !  rtph264pay pt=96 ! udpsink host=127.0.0.1 port=5004 sync=false t. ! queue ! nvvidconv ! 'video/x-raw(memory:NVMM), width=640, height=480' ! nvv4l2h264enc insert-sps-pps=true ! h264parse !  rtph264pay pt=96 ! udpsink host=127.0.0.1 port=5005 sync=false

form bash I have CPU usage at about 45% and still 0% usage of GPU?

I check the resource usage using jtop.

  1. Why there is no GPU usage during using gstreamber both from Python OpenCV and from bash?
  2. Why there is that big difference when I use gstreamer with openCV compared to using pure gstreamer?
3

Hi,
In OpenCV, main format is BGR which is not supported by most hardware engines in Jetson, and have to utilize significant CPU usage. It limits performance. we would suggest run a gstreamer pipeline and map NVMM buffer to cv::gpu::gpuMat. Pleaae refer to the sample:
Nano not using GPU with gstreamer/python. Slow FPS, dropped frames - #8 by DaneLLL

4

@DaneLLL thank you for fast replay.

If I understand correctly that any use of OpenCV (even with “correct”/HW supported format) will cause high CPU usage?

5

Hi,

Yes, if you construct gstreamer pipelines with appsink or appsrc, the buffer has to be in BGR format and it takes high CPU usage. Optimal solution is to map RGBA CUDA buffer to cv::gpu::gpuMat and utilize CUDA filter.