Face Recognition Running Slow on Jetpack 4.4

The simple python face recognition program below will run at:

15 fps on the Jetson Nano on Jetpack 4.2
15 fps on Jetson Nano on Jetpack 4.3
30 fps on the Jetson Xaview NX on Jetpack 4.4
BUT only
3-5 fps on Jetson Nano on Jetpack 4.4

Wondering if it might be a similar problem to:

as the performance hit is about the same.

This is the code, which loads training data, and then recognizes, boxes and labels known faces:

‘’’
import face_recognition
import cv2
import os
import pickle
import time
print(cv2.version)
fpsReport=0
scaleFactor=.25

Encodings=
Names=

with open(‘train.pkl’,‘rb’) as f:
Names=pickle.load(f)
Encodings=pickle.load(f)
font=cv2.FONT_HERSHEY_SIMPLEX

cam= cv2.VideoCapture(‘/dev/video1’)

timeStamp=time.time()
while True:

_,frame=cam.read()
frameSmall=cv2.resize(frame,(0,0),fx=scaleFactor,fy=scaleFactor)
frameRGB=cv2.cvtColor(frameSmall,cv2.COLOR_BGR2RGB)
facePositions=face_recognition.face_locations(frameRGB,model='cnn')
allEncodings=face_recognition.face_encodings(frameRGB,facePositions)
for (top,right,bottom,left),face_encoding in zip(facePositions,allEncodings):
    name='Unkown Person'
    matches=face_recognition.compare_faces(Encodings,face_encoding)
    if True in matches:
        first_match_index=matches.index(True)
        name=Names[first_match_index]
    top=int(top/scaleFactor)
    right=int(right/scaleFactor)
    bottom=int(bottom/scaleFactor)
    left=int(left/scaleFactor)
    cv2.rectangle(frame,(left,top),(right, bottom),(0,0,255),2)
    cv2.putText(frame,name,(left,top-6),font,.75,(0,0,255),2)
dt=time.time()-timeStamp
fps=1/dt
fpsReport=.90*fpsReport + .1*fps
#print('fps is:',round(fpsReport,1))
timeStamp=time.time()
cv2.rectangle(frame,(0,0),(100,40),(0,0,255),-1)
cv2.putText(frame,str(round(fpsReport,1))+ 'fps',(0,25),font,.75,(0,255,255,2))
cv2.imshow('Picture',frame)
cv2.moveWindow('Picture',0,0)
if cv2.waitKey(1)==ord('q'):
    break

cam.release()
cv2.destroyAllWindows()
‘’’

Any advice would be appreciated.
Thanks!
Paul McWhorter
www.toptechboy.com

face_recognition internally uses Dlib a lot - can you check if Dlib was built with with CUDA support enabled (python3 setup.py install --set DLIB_USE_CUDA=1)?

Thank you for your help, Typing your suggested command, ( python3 setup.py install --set DLIB_USE_CUDA=1 ), I get the following

‘’’
$ sudo python3 setup.py install --set DLIB_USE_CUDA=1
running install
running bdist_egg
running egg_info
writing dlib.egg-info/PKG-INFO
writing dependency_links to dlib.egg-info/dependency_links.txt
writing top-level names to dlib.egg-info/top_level.txt
package init file ‘dlib/init.py’ not found (or not a regular file)
reading manifest file ‘dlib.egg-info/SOURCES.txt’
reading manifest template ‘MANIFEST.in’
writing manifest file ‘dlib.egg-info/SOURCES.txt’
installing library code to build/bdist.linux-aarch64/egg
running install_lib
running build_py
running build_ext
Building extension for Python 3.6.9 (default, Apr 18 2020, 01:56:04)
Invoking CMake setup: ‘cmake /home/pjm/Downloads/dlib-19.17/tools/python -DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/home/pjm/Downloads/dlib-19.17/build/lib.linux-aarch64-3.6 -DPYTHON_EXECUTABLE=/usr/bin/python3 -DDLIB_USE_CUDA=1 -DCMAKE_BUILD_TYPE=Release’
– pybind11 v2.2.2
– Using CMake version: 3.10.2
– Compiling dlib version: 19.17.0
– Searching for BLAS and LAPACK
– Searching for BLAS and LAPACK
– Checking for module ‘cblas’
– No package ‘cblas’ found
– Found OpenBLAS library
– Using OpenBLAS’s built in LAPACK
– Looking for cuDNN install…
– Found cuDNN: /usr/lib/aarch64-linux-gnu/libcudnn.so
– Enabling CUDA support for dlib. DLIB WILL USE CUDA
– C++11 activated.
– Configuring done
– Generating done
– Build files have been written to: /home/pjm/Downloads/dlib-19.17/build/temp.linux-aarch64-3.6
Invoking CMake build: ‘cmake --build . --config Release – -j2’
[ 78%] Built target dlib
[100%] Built target dlib_python
creating build/bdist.linux-aarch64/egg
copying build/lib.linux-aarch64-3.6/dlib.cpython-36m-aarch64-linux-gnu.so → build/bdist.linux-aarch64/egg
creating stub loader for dlib.cpython-36m-aarch64-linux-gnu.so
byte-compiling build/bdist.linux-aarch64/egg/dlib.py to dlib.cpython-36.pyc
creating build/bdist.linux-aarch64/egg/EGG-INFO
copying dlib.egg-info/PKG-INFO → build/bdist.linux-aarch64/egg/EGG-INFO
copying dlib.egg-info/SOURCES.txt → build/bdist.linux-aarch64/egg/EGG-INFO
copying dlib.egg-info/dependency_links.txt → build/bdist.linux-aarch64/egg/EGG-INFO
copying dlib.egg-info/not-zip-safe → build/bdist.linux-aarch64/egg/EGG-INFO
copying dlib.egg-info/top_level.txt → build/bdist.linux-aarch64/egg/EGG-INFO
writing build/bdist.linux-aarch64/egg/EGG-INFO/native_libs.txt
creating ‘dist/dlib-19.17.0-py3.6-linux-aarch64.egg’ and adding ‘build/bdist.linux-aarch64/egg’ to it
removing ‘build/bdist.linux-aarch64/egg’ (and everything under it)
Processing dlib-19.17.0-py3.6-linux-aarch64.egg
removing ‘/usr/local/lib/python3.6/dist-packages/dlib-19.17.0-py3.6-linux-aarch64.egg’ (and everything under it)
creating /usr/local/lib/python3.6/dist-packages/dlib-19.17.0-py3.6-linux-aarch64.egg
Extracting dlib-19.17.0-py3.6-linux-aarch64.egg to /usr/local/lib/python3.6/dist-packages
dlib 19.17.0 is already the active version in easy-install.pth

Installed /usr/local/lib/python3.6/dist-packages/dlib-19.17.0-py3.6-linux-aarch64.egg
Processing dependencies for dlib==19.17.0
Finished processing dependencies for dlib==19.17.0
‘’’

Looks good. Now you can check if performance/framerate is better. If not I suppose that cuDNN for JP4.4 is the culprit…

Still at 3 fps. Again, same setup on jetpack 4.3 runs at 15 fps. Appreciate your help.
Paul McWhorter

1 Like

Hi,

Thanks for the information.

As you already know, we also see some performance regression on darknet.
This really help us to narrow down the issue.

We are still checking this. Will keep you update.
Thanks.

Hello i got the same issue.
Is there any solution for face recognition fps?
I have to use jetpack 4.4 for others dependencies and i got a bottleneck with face recognition into a real-time application in c++.

Thanks

1 Like

hello,
do you have any solution?

thank you

Result of sudo python3 setup.py install --set DLIB_USE_CUDA=1 has cuda turned off

yveske@Jetson-Nano:~/dlib-19.17$ sudo python3 setup.py install --set DLIB_USE_CUDA=1
running install
running bdist_egg
running egg_info
writing dlib.egg-info/PKG-INFO
writing dependency_links to dlib.egg-info/dependency_links.txt
writing top-level names to dlib.egg-info/top_level.txt
package init file ‘dlib/init.py’ not found (or not a regular file)
reading manifest file ‘dlib.egg-info/SOURCES.txt’
reading manifest template ‘MANIFEST.in’
writing manifest file ‘dlib.egg-info/SOURCES.txt’
installing library code to build/bdist.linux-aarch64/egg
running install_lib
running build_py
running build_ext
Building extension for Python 3.6.9 (default, Jul 17 2020, 12:50:27)
Invoking CMake setup: ‘cmake /home/yveske/dlib-19.17/tools/python -DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/home/yveske/dlib-19.17/build/lib.linux-aarch64-3.6 -DPYTHON_EXECUTABLE=/usr/bin/python3 -DDLIB_USE_CUDA=1 -DCMAKE_BUILD_TYPE=Release’
– pybind11 v2.2.2
– Using CMake version: 3.10.2
– Compiling dlib version: 19.17.0
– Searching for BLAS and LAPACK
– Searching for BLAS and LAPACK
– Checking for module ‘cblas’
– No package ‘cblas’ found
– Found OpenBLAS library
– Using OpenBLAS’s built in LAPACK
– Looking for cuDNN install…
– Found cuDNN: /usr/lib/aarch64-linux-gnu/libcudnn.so
– Disabling CUDA support for dlib. DLIB WILL NOT USE CUDA
– C++11 activated.
– Configuring done
– Generating done
– Build files have been written to: /home/yveske/dlib-19.17/build/temp.linux-aarch64-3.6
Invoking CMake build: ‘cmake --build . --config Release – -j2’
[ 77%] Built target dlib
[100%] Built target dlib_python
creating build/bdist.linux-aarch64/egg
copying build/lib.linux-aarch64-3.6/dlib.cpython-36m-aarch64-linux-gnu.so → build/bdist.linux-aarch64/egg
creating stub loader for dlib.cpython-36m-aarch64-linux-gnu.so
byte-compiling build/bdist.linux-aarch64/egg/dlib.py to dlib.cpython-36.pyc
creating build/bdist.linux-aarch64/egg/EGG-INFO
copying dlib.egg-info/PKG-INFO → build/bdist.linux-aarch64/egg/EGG-INFO
copying dlib.egg-info/SOURCES.txt → build/bdist.linux-aarch64/egg/EGG-INFO
copying dlib.egg-info/dependency_links.txt → build/bdist.linux-aarch64/egg/EGG-INFO
copying dlib.egg-info/not-zip-safe → build/bdist.linux-aarch64/egg/EGG-INFO
copying dlib.egg-info/top_level.txt → build/bdist.linux-aarch64/egg/EGG-INFO
writing build/bdist.linux-aarch64/egg/EGG-INFO/native_libs.txt
creating ‘dist/dlib-19.17.0-py3.6-linux-aarch64.egg’ and adding ‘build/bdist.linux-aarch64/egg’ to it
removing ‘build/bdist.linux-aarch64/egg’ (and everything under it)
Processing dlib-19.17.0-py3.6-linux-aarch64.egg
removing ‘/usr/local/lib/python3.6/dist-packages/dlib-19.17.0-py3.6-linux-aarch64.egg’ (and everything under it)
creating /usr/local/lib/python3.6/dist-packages/dlib-19.17.0-py3.6-linux-aarch64.egg
Extracting dlib-19.17.0-py3.6-linux-aarch64.egg to /usr/local/lib/python3.6/dist-packages
dlib 19.17.0 is already the active version in easy-install.pth

Installed /usr/local/lib/python3.6/dist-packages/dlib-19.17.0-py3.6-linux-aarch64.egg
Processing dependencies for dlib==19.17.0
Finished processing dependencies for dlib==19.17.0

How do I proceed ?

TIA
Yveske

Assuming you are on JetPack 4.4, then DLIB version 19.17 is too old and incompatible with JP4.4.
DLIB 19.21.99 compiles fine on my Xavier AGX (JetPack 4.4, CMake version: 3.16.4)

...
-- Found CUDA: /usr/local/cuda-10.2 (found suitable version "10.2", minimum required is "7.5")
-- Looking for cuDNN install...
-- Found cuDNN: /usr/lib/aarch64-linux-gnu/libcudnn.so
-- Building a CUDA test project to see if your compiler is compatible with CUDA...
-- Building a cuDNN test project to check if you have the right version of cuDNN installed...
-- Enabling CUDA support for dlib. DLIB WILL USE CUDA
...
1 Like

Would like to give you a million thumbs up, worked like a charm !!!
I’m following the lessons of Paul McWhorter and face recognition went from 0.9 fps to 7.6 fps !!!
Can’t thank you enough !!!

Hi,

Good to know it works now.
You can also find some information for dlib+JetPack4.4 here:

Thanks.

Hi guys,

How did you got it working?
I flashed my nano with new jetpack 4.4,
installed dlib first with pip3,
then uninstalled it (I’ve uninstalled it couple of times to make shure there is only one dlib installation),
builed dlib ‘19.21.0’ from source
and still I got 3fps while trying to do dlib.get_frontal_face_detector in python3 script using camera picture.

When I just comment detector everything works smooth.

dlib.DLIB_USE_CUDA gives me True but when I run the script, jtop shows me GPU performance close to nothing.

Please help, I spent already more than a week on this :(

Same issue on JetPack 4.5.1
Opencv 4.5.2 with CUDA
Dlib 19.22.0 With CUDA

2.5 fps on Xavier Nx

Any advice ?

Installing/compiling OpenCV and Dlib with CUDA support does not automagically speed up your application. You most likely need to apply changes to your program code in order to utilize GPU instead of CPU.

The code i use is here any andvice is welcome )
https://forums.developer.nvidia.com/t/xavier-nx-opencv-and-face-recognition-very-slow-and-no-gpu/178153

You should call face-recognition like this:
face_recognition.face_locations(image, model="cnn")

You may also try to reduce the image size before passing it to the face recogntion algorithm. Most model are trained on small image sizes like 300x300 while you seem to be using 1280x720.

Following blog is worth a look as well, it is refering to outdated dlib-version, though:

Thanks )

face_recognition.face_locations(image, model="cnn")

allow me to go from 2.5fps to 6fps (640x480) and in 330x330 up 15fps
That really better )