Hi, I’m trying to use the cv2.dnn module from opencv 4.5.3 on my Jetson Nano 4GB. I compiled OpenCV with cuda enabled, and has you can see the GPU usage while running a live webcam demo of the yolov3 integration on opencv, it dosen’t seems to use the GPU (and I have maybe 3 frames per minute…)
Here you can see a general picture showing the JTOP tools showing used ressources:
And here you can see what return the command print(cv2.getBuildInformation()) on my python3 installation, after being tipped this two command: sudo nvpmodel -m 0 and sudo jetson_clocks.sh
General configuration for OpenCV 4.5.3 =====================================
Version control: unknown
Extra modules:
Location (extra): /home/simonus/opencv_contrib/modules
Version control (extra): unknown
Platform:
Timestamp: 2021-08-28T09:01:14Z
Host: Linux 4.9.201-tegra aarch64
CMake: 3.10.2
CMake generator: Unix Makefiles
CMake build tool: /usr/bin/make
Configuration: RELEASE
CPU/HW features:
Baseline: NEON FP16
required: NEON
C/C++:
Built as dynamic libs?: YES
C++ standard: 11
C++ Compiler: /usr/bin/c++ (ver 7.5.0)
C++ flags (Release): -fsigned-char -ffast-math -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -fvisibility=hidden -fvisibility-inlines-hidden -fopenmp -O3 -DNDEBUG -DNDEBUG
C++ flags (Debug): -fsigned-char -ffast-math -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -fvisibility=hidden -fvisibility-inlines-hidden -fopenmp -g -O0 -DDEBUG -D_DEBUG
C Compiler: /usr/bin/cc
C flags (Release): -fsigned-char -ffast-math -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -fvisibility=hidden -fopenmp -O3 -DNDEBUG -DNDEBUG
C flags (Debug): -fsigned-char -ffast-math -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -fvisibility=hidden -fopenmp -g -O0 -DDEBUG -D_DEBUG
Linker flags (Release): -Wl,--gc-sections -Wl,--as-needed
Linker flags (Debug): -Wl,--gc-sections -Wl,--as-needed
ccache: NO
Precompiled headers: NO
Extra dependencies: m pthread cudart_static dl rt nppc nppial nppicc nppicom nppidei nppif nppig nppim nppist nppisu nppitc npps cublas cudnn cufft -L/usr/local/cuda/lib64 -L/usr/lib/aarch64-linux-gnu
3rdparty dependencies:
OpenCV modules:
To be built: alphamat aruco barcode bgsegm bioinspired calib3d ccalib core cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev datasets dnn dnn_objdetect dnn_superres dpm face features2d flann freetype fuzzy gapi hdf hfs highgui img_hash imgcodecs imgproc intensity_transform line_descriptor mcc ml objdetect optflow phase_unwrapping photo plot python2 python3 quality rapid reg rgbd saliency sfm shape stereo stitching structured_light superres surface_matching text tracking ts video videoio videostab wechat_qrcode xfeatures2d ximgproc xobjdetect xphoto
Disabled: world
Disabled by dependency: -
Unavailable: cvv java julia matlab ovis viz
Applications: perf_tests apps
Documentation: NO
Non-free algorithms: YES
GUI:
GTK+: YES (ver 3.22.30)
GThread : YES (ver 2.56.4)
GtkGlExt: NO
OpenGL support: NO
VTK support: NO
Media I/O:
ZLib: /usr/lib/aarch64-linux-gnu/libz.so (ver 1.2.11)
JPEG: /usr/lib/aarch64-linux-gnu/libjpeg.so (ver 80)
WEBP: build (ver encoder: 0x020f)
PNG: /usr/lib/aarch64-linux-gnu/libpng.so (ver 1.6.34)
TIFF: build (ver 42 - 4.2.0)
JPEG 2000: build (ver 2.4.0)
OpenEXR: build (ver 2.3.0)
HDR: YES
SUNRASTER: YES
PXM: YES
PFM: YES
Video I/O:
DC1394: YES (2.2.5)
FFMPEG: YES
avcodec: YES (57.107.100)
avformat: YES (57.83.100)
avutil: YES (55.78.100)
swscale: YES (4.8.100)
avresample: YES (3.7.0)
GStreamer: YES (1.14.5)
v4l/v4l2: YES (linux/videodev2.h)
Parallel framework: TBB (ver 2020.2 interface 11102)
Trace: YES (with Intel ITT)
Other third-party libraries:
Lapack: NO
Eigen: YES (ver 3.3.4)
Custom HAL: YES (carotene (ver 0.0.1))
Protobuf: build (3.5.1)
NVIDIA CUDA: YES (ver 10.2, CUFFT CUBLAS FAST_MATH)
NVIDIA GPU arch: 53
NVIDIA PTX archs:
cuDNN: YES (ver 8.0.0)
Python 2:
Interpreter: /usr/bin/python2.7 (ver 2.7.17)
Libraries: /usr/lib/aarch64-linux-gnu/libpython2.7.so (ver 2.7.17)
numpy: /usr/lib/python2.7/dist-packages/numpy/core/include (ver 1.13.3)
install path: lib/python2.7/dist-packages/cv2/python-2.7
Python 3:
Interpreter: /usr/bin/python3 (ver 3.6.9)
Libraries: /usr/lib/aarch64-linux-gnu/libpython3.6m.so (ver 3.6.9)
numpy: /usr/local/lib/python3.6/dist-packages/numpy/core/include (ver 1.19.4)
install path: lib/python3.6/dist-packages/cv2/python-3.6
Python (for build): /usr/bin/python2.7
Java:
ant: NO
JNI: NO
Java wrappers: NO
Java tests: NO
Install to: /usr
-----------------------------------------------------------------
As you can see, it return that my installation have CUDA support, don’t know why it’s not using it… Somebody can help?
Here is the python code i’m using:
#import pyrealsense2.pyrealsense2 as rs
import pyrealsense2 as rs
import cv2
import numpy as np
# Load Yolo
print("LOADING YOLO")
net = cv2.dnn.readNet("yolov3.weights", "yolov3.cfg")
#save all the names in file o the list classes
classes = []
with open("coco.names", "r") as f:
classes = [line.strip() for line in f.readlines()]
#get layers of the network
layer_names = net.getLayerNames()
#Determine the output layer names from the YOLO model
output_layers = [layer_names[i[0] - 1] for i in net.getUnconnectedOutLayers()]
print("YOLO LOADED")
# Configure depth and color streams
pipeline = rs.pipeline()
config = rs.config()
config.enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30)
config.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)
# Start streaming
pipeline.start(config)
align = rs.align(rs.stream.color)
while True:
# Capture frame-by-frame
frames = pipeline.wait_for_frames()
aligned_frames = align.process(frames)
depth = aligned_frames.get_depth_frame()
img = aligned_frames.get_color_frame()
img = np.asanyarray(img.get_data())
img = cv2.resize(img, None, fx=0.4, fy=0.4)
depth = np.asanyarray(depth.get_data())
depth = cv2.resize(depth, None, fx=0.4, fy=0.4)
height, width, channels = img.shape
# USing blob function of opencv to preprocess image
blob = cv2.dnn.blobFromImage(img, 1 / 255.0, (416, 416),
swapRB=True, crop=False)
#Detecting objects
net.setInput(blob)
outs = net.forward(output_layers)
# Showing informations on the screen
class_ids = []
confidences = []
boxes = []
for out in outs:
for detection in out:
scores = detection[5:]
class_id = np.argmax(scores)
confidence = scores[class_id]
if confidence > 0.5:
# Object detected
center_x = int(detection[0] * width)
center_y = int(detection[1] * height)
w = int(detection[2] * width)
h = int(detection[3] * height)
# Rectangle coordinates
x = int(center_x - w / 2)
y = int(center_y - h / 2)
boxes.append([x, y, w, h])
confidences.append(float(confidence))
class_ids.append(class_id)
#We use NMS function in opencv to perform Non-maximum Suppression
#we give it score threshold and nms threshold as arguments.
indexes = cv2.dnn.NMSBoxes(boxes, confidences, 0.5, 0.4)
font = cv2.FONT_HERSHEY_PLAIN
colors = np.random.uniform(0, 255, size=(len(classes), 3))
for i in range(len(boxes)):
if i in indexes:
x, y, w, h = boxes[i]
label = str(classes[class_ids[i]])
color = colors[class_ids[i]]
cv2.rectangle(img, (x, y), (x + w, y + h), color, 2)
cv2.putText(img, label, (x, y + 30), font, 2, color, 3)
if(label=="orange"):
print(label," detecté")
distance=(depth[int(y+h/2),int(x+w/2)])
horizontal_angle=37*(x+w/2)/(width/2)
vertical_angle=31*(y+h/2)/(height/2)
cv2.imshow("Image",cv2.resize(img, (800,600)))
if cv2.waitKey(1) & 0xFF == ord('q'):
break
pipeline.stop()
cv2.destroyAllWindows()