Nano not using GPU with gstreamer/python. Slow FPS, dropped frames

I am using the a typical pipeline (see below) to feed my Opencv/Python program frames. My CPU is running at 100% , while my GPU is only 25%. Is there anything I can do to put more work on the GPU?

def gstreamer_pipeline(
capture_width=3264,
capture_height=1848,
display_width=3264,
display_height=1848,
framerate=28,
flip_method=0,
):
return (
"nvarguscamerasrc ! "
"video/x-raw(memory:NVMM), "
"width=(int)%d, height=(int)%d, "
"format=(string)NV12, framerate=(fraction)%d/1 ! "
"nvvidconv flip-method=%d ! "
"video/x-raw, width=(int)%d, height=(int)%d, format=(string)BGRx ! "
"videoconvert ! "
“video/x-raw, format=(string)BGR ! appsink”
% (
capture_width,
capture_height,
framerate,
flip_method,
display_width,
display_height,
)
)

Hi,
The optimal-performance solution is to pass NVMM[video/x-raw(memory:NVMM)] buffers in the pipelines. However, with python, it only accepts CPU[video/x-raw,format=BGR] buffers. This requires copying NVMM buffers to CPU buffers and takes certain CPU loading.

On Jetson Nano with ‘sudo jetson_clcoks’ being executed, it might be good for 1920x1080p30. But the resolution of your usecase is close to 4K, which shall exceed hardware capability.

Suggest you try pure gstreamer or tegra_multimedia_api.

That makes sense.

Is there any way for me to be able to do opencv processing on the device with the CSI camera? I am trying to make a live-streaming system. I need to do a little bit of panning/zooming and text overlays on the stream

Hi,
There is a sample of gstreamer + cuda::gpuMat. Please refer to
https://devtalk.nvidia.com/default/topic/1022543/jetson-tx2/gstreamer-nvmm-lt-gt-opencv-gpumat/post/5311027/#5311027

Can I use gpuMat with C++ only? or can I run gpuMat with python?

Hi,
Not sure but it looks like python uses appsink to get CPU[video/x-raw] buffer. It may not be supported to use gpuMat.

Ok. Thanks for the help. I’m going to use pure gstreamer as you suggested.

Hi,
Attached a sample for reference. Please follow the steps:
0. Install CUDA, tegra_multimedia_api samples through sdkmanager

  1. Down the script https://github.com/AastaNV/JEP/blob/master/script/install_opencv4.1.1_Jetson.sh
  2. Add the option and execute the script to build OCV4.1.1
-D OPENCV_GENERATE_PKGCONFIG=YES
  1. Execute
$ sudo ldconfig -v
  1. Build and run the sample
~/gst_cv_gpumat$ CUDA_VER=10.0 make
Compiling: gst_cv_gpumat.cpp
g++ -I/usr/src/tegra_multimedia_api/include -I/usr/local/cuda-10.0/include `pkg-config --cflags gstreamer-1.0 opencv4)` -c gst_cv_gpumat.cpp -o gst_cv_gpumat.o
Linking: gst_cv_gpumat
g++ -o gst_cv_gpumat gst_cv_gpumat.o -I/usr/src/tegra_multimedia_api/include -I/usr/local/cuda-10.0/include `pkg-config --cflags gstreamer-1.0 opencv4)` -Wall -std=c++11 -L/usr/lib/aarch64-linux-gnu/tegra/ -lEGL -lGLESv2 -L/usr/lib/aarch64-linux-gnu/tegra/ -lcuda -lnvbuf_utils -L/usr/local/cuda-10.0/lib64/ -lcudart  `pkg-config --libs gstreamer-1.0 opencv4`
~/gst_cv_gpumat$ ./gst_cv_gpumat

gst_cv_gpumat.zip (3.05 KB)

1 Like