Face Recognition Running Slow on Jetpack 4.4

The simple python face recognition program below will run at:

15 fps on the Jetson Nano on Jetpack 4.2
15 fps on Jetson Nano on Jetpack 4.3
30 fps on the Jetson Xaview NX on Jetpack 4.4
BUT only
3-5 fps on Jetson Nano on Jetpack 4.4

Wondering if it might be a similar problem to:


as the performance hit is about the same.

This is the code, which loads training data, and then recognizes, boxes and labels known faces:

‘’’
import face_recognition
import cv2
import os
import pickle
import time
print(cv2.version)
fpsReport=0
scaleFactor=.25

Encodings=
Names=

with open(‘train.pkl’,‘rb’) as f:
Names=pickle.load(f)
Encodings=pickle.load(f)
font=cv2.FONT_HERSHEY_SIMPLEX

cam= cv2.VideoCapture(’/dev/video1’)

timeStamp=time.time()
while True:

_,frame=cam.read()
frameSmall=cv2.resize(frame,(0,0),fx=scaleFactor,fy=scaleFactor)
frameRGB=cv2.cvtColor(frameSmall,cv2.COLOR_BGR2RGB)
facePositions=face_recognition.face_locations(frameRGB,model='cnn')
allEncodings=face_recognition.face_encodings(frameRGB,facePositions)
for (top,right,bottom,left),face_encoding in zip(facePositions,allEncodings):
    name='Unkown Person'
    matches=face_recognition.compare_faces(Encodings,face_encoding)
    if True in matches:
        first_match_index=matches.index(True)
        name=Names[first_match_index]
    top=int(top/scaleFactor)
    right=int(right/scaleFactor)
    bottom=int(bottom/scaleFactor)
    left=int(left/scaleFactor)
    cv2.rectangle(frame,(left,top),(right, bottom),(0,0,255),2)
    cv2.putText(frame,name,(left,top-6),font,.75,(0,0,255),2)
dt=time.time()-timeStamp
fps=1/dt
fpsReport=.90*fpsReport + .1*fps
#print('fps is:',round(fpsReport,1))
timeStamp=time.time()
cv2.rectangle(frame,(0,0),(100,40),(0,0,255),-1)
cv2.putText(frame,str(round(fpsReport,1))+ 'fps',(0,25),font,.75,(0,255,255,2))
cv2.imshow('Picture',frame)
cv2.moveWindow('Picture',0,0)
if cv2.waitKey(1)==ord('q'):
    break

cam.release()
cv2.destroyAllWindows()
‘’’

Any advice would be appreciated.
Thanks!
Paul McWhorter
www.toptechboy.com

face_recognition internally uses Dlib a lot - can you check if Dlib was built with with CUDA support enabled (python3 setup.py install --set DLIB_USE_CUDA=1)?

Thank you for your help, Typing your suggested command, ( python3 setup.py install --set DLIB_USE_CUDA=1 ), I get the following

‘’’
$ sudo python3 setup.py install --set DLIB_USE_CUDA=1
running install
running bdist_egg
running egg_info
writing dlib.egg-info/PKG-INFO
writing dependency_links to dlib.egg-info/dependency_links.txt
writing top-level names to dlib.egg-info/top_level.txt
package init file ‘dlib/init.py’ not found (or not a regular file)
reading manifest file ‘dlib.egg-info/SOURCES.txt’
reading manifest template ‘MANIFEST.in’
writing manifest file ‘dlib.egg-info/SOURCES.txt’
installing library code to build/bdist.linux-aarch64/egg
running install_lib
running build_py
running build_ext
Building extension for Python 3.6.9 (default, Apr 18 2020, 01:56:04)
Invoking CMake setup: ‘cmake /home/pjm/Downloads/dlib-19.17/tools/python -DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/home/pjm/Downloads/dlib-19.17/build/lib.linux-aarch64-3.6 -DPYTHON_EXECUTABLE=/usr/bin/python3 -DDLIB_USE_CUDA=1 -DCMAKE_BUILD_TYPE=Release’
– pybind11 v2.2.2
– Using CMake version: 3.10.2
– Compiling dlib version: 19.17.0
– Searching for BLAS and LAPACK
– Searching for BLAS and LAPACK
– Checking for module ‘cblas’
– No package ‘cblas’ found
– Found OpenBLAS library
– Using OpenBLAS’s built in LAPACK
– Looking for cuDNN install…
– Found cuDNN: /usr/lib/aarch64-linux-gnu/libcudnn.so
– Enabling CUDA support for dlib. DLIB WILL USE CUDA
– C++11 activated.
– Configuring done
– Generating done
– Build files have been written to: /home/pjm/Downloads/dlib-19.17/build/temp.linux-aarch64-3.6
Invoking CMake build: ‘cmake --build . --config Release – -j2’
[ 78%] Built target dlib
[100%] Built target dlib_python
creating build/bdist.linux-aarch64/egg
copying build/lib.linux-aarch64-3.6/dlib.cpython-36m-aarch64-linux-gnu.so -> build/bdist.linux-aarch64/egg
creating stub loader for dlib.cpython-36m-aarch64-linux-gnu.so
byte-compiling build/bdist.linux-aarch64/egg/dlib.py to dlib.cpython-36.pyc
creating build/bdist.linux-aarch64/egg/EGG-INFO
copying dlib.egg-info/PKG-INFO -> build/bdist.linux-aarch64/egg/EGG-INFO
copying dlib.egg-info/SOURCES.txt -> build/bdist.linux-aarch64/egg/EGG-INFO
copying dlib.egg-info/dependency_links.txt -> build/bdist.linux-aarch64/egg/EGG-INFO
copying dlib.egg-info/not-zip-safe -> build/bdist.linux-aarch64/egg/EGG-INFO
copying dlib.egg-info/top_level.txt -> build/bdist.linux-aarch64/egg/EGG-INFO
writing build/bdist.linux-aarch64/egg/EGG-INFO/native_libs.txt
creating ‘dist/dlib-19.17.0-py3.6-linux-aarch64.egg’ and adding ‘build/bdist.linux-aarch64/egg’ to it
removing ‘build/bdist.linux-aarch64/egg’ (and everything under it)
Processing dlib-19.17.0-py3.6-linux-aarch64.egg
removing ‘/usr/local/lib/python3.6/dist-packages/dlib-19.17.0-py3.6-linux-aarch64.egg’ (and everything under it)
creating /usr/local/lib/python3.6/dist-packages/dlib-19.17.0-py3.6-linux-aarch64.egg
Extracting dlib-19.17.0-py3.6-linux-aarch64.egg to /usr/local/lib/python3.6/dist-packages
dlib 19.17.0 is already the active version in easy-install.pth

Installed /usr/local/lib/python3.6/dist-packages/dlib-19.17.0-py3.6-linux-aarch64.egg
Processing dependencies for dlib==19.17.0
Finished processing dependencies for dlib==19.17.0
‘’’

Looks good. Now you can check if performance/framerate is better. If not I suppose that cuDNN for JP4.4 is the culprit…

Still at 3 fps. Again, same setup on jetpack 4.3 runs at 15 fps. Appreciate your help.
Paul McWhorter

1 Like

Hi,

Thanks for the information.

As you already know, we also see some performance regression on darknet.
This really help us to narrow down the issue.

We are still checking this. Will keep you update.
Thanks.

Hello i got the same issue.
Is there any solution for face recognition fps?
I have to use jetpack 4.4 for others dependencies and i got a bottleneck with face recognition into a real-time application in c++.

Thanks

1 Like

hello,
do you have any solution?

thank you

Result of sudo python3 setup.py install --set DLIB_USE_CUDA=1 has cuda turned off

yveske@Jetson-Nano:~/dlib-19.17$ sudo python3 setup.py install --set DLIB_USE_CUDA=1
running install
running bdist_egg
running egg_info
writing dlib.egg-info/PKG-INFO
writing dependency_links to dlib.egg-info/dependency_links.txt
writing top-level names to dlib.egg-info/top_level.txt
package init file ‘dlib/init.py’ not found (or not a regular file)
reading manifest file ‘dlib.egg-info/SOURCES.txt’
reading manifest template ‘MANIFEST.in’
writing manifest file ‘dlib.egg-info/SOURCES.txt’
installing library code to build/bdist.linux-aarch64/egg
running install_lib
running build_py
running build_ext
Building extension for Python 3.6.9 (default, Jul 17 2020, 12:50:27)
Invoking CMake setup: ‘cmake /home/yveske/dlib-19.17/tools/python -DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/home/yveske/dlib-19.17/build/lib.linux-aarch64-3.6 -DPYTHON_EXECUTABLE=/usr/bin/python3 -DDLIB_USE_CUDA=1 -DCMAKE_BUILD_TYPE=Release’
– pybind11 v2.2.2
– Using CMake version: 3.10.2
– Compiling dlib version: 19.17.0
– Searching for BLAS and LAPACK
– Searching for BLAS and LAPACK
– Checking for module ‘cblas’
– No package ‘cblas’ found
– Found OpenBLAS library
– Using OpenBLAS’s built in LAPACK
– Looking for cuDNN install…
– Found cuDNN: /usr/lib/aarch64-linux-gnu/libcudnn.so
– Disabling CUDA support for dlib. DLIB WILL NOT USE CUDA
– C++11 activated.
– Configuring done
– Generating done
– Build files have been written to: /home/yveske/dlib-19.17/build/temp.linux-aarch64-3.6
Invoking CMake build: ‘cmake --build . --config Release – -j2’
[ 77%] Built target dlib
[100%] Built target dlib_python
creating build/bdist.linux-aarch64/egg
copying build/lib.linux-aarch64-3.6/dlib.cpython-36m-aarch64-linux-gnu.so -> build/bdist.linux-aarch64/egg
creating stub loader for dlib.cpython-36m-aarch64-linux-gnu.so
byte-compiling build/bdist.linux-aarch64/egg/dlib.py to dlib.cpython-36.pyc
creating build/bdist.linux-aarch64/egg/EGG-INFO
copying dlib.egg-info/PKG-INFO -> build/bdist.linux-aarch64/egg/EGG-INFO
copying dlib.egg-info/SOURCES.txt -> build/bdist.linux-aarch64/egg/EGG-INFO
copying dlib.egg-info/dependency_links.txt -> build/bdist.linux-aarch64/egg/EGG-INFO
copying dlib.egg-info/not-zip-safe -> build/bdist.linux-aarch64/egg/EGG-INFO
copying dlib.egg-info/top_level.txt -> build/bdist.linux-aarch64/egg/EGG-INFO
writing build/bdist.linux-aarch64/egg/EGG-INFO/native_libs.txt
creating ‘dist/dlib-19.17.0-py3.6-linux-aarch64.egg’ and adding ‘build/bdist.linux-aarch64/egg’ to it
removing ‘build/bdist.linux-aarch64/egg’ (and everything under it)
Processing dlib-19.17.0-py3.6-linux-aarch64.egg
removing ‘/usr/local/lib/python3.6/dist-packages/dlib-19.17.0-py3.6-linux-aarch64.egg’ (and everything under it)
creating /usr/local/lib/python3.6/dist-packages/dlib-19.17.0-py3.6-linux-aarch64.egg
Extracting dlib-19.17.0-py3.6-linux-aarch64.egg to /usr/local/lib/python3.6/dist-packages
dlib 19.17.0 is already the active version in easy-install.pth

Installed /usr/local/lib/python3.6/dist-packages/dlib-19.17.0-py3.6-linux-aarch64.egg
Processing dependencies for dlib==19.17.0
Finished processing dependencies for dlib==19.17.0

How do I proceed ?

TIA
Yveske

Assuming you are on JetPack 4.4, then DLIB version 19.17 is too old and incompatible with JP4.4.
DLIB 19.21.99 compiles fine on my Xavier AGX (JetPack 4.4, CMake version: 3.16.4)

...
-- Found CUDA: /usr/local/cuda-10.2 (found suitable version "10.2", minimum required is "7.5")
-- Looking for cuDNN install...
-- Found cuDNN: /usr/lib/aarch64-linux-gnu/libcudnn.so
-- Building a CUDA test project to see if your compiler is compatible with CUDA...
-- Building a cuDNN test project to check if you have the right version of cuDNN installed...
-- Enabling CUDA support for dlib. DLIB WILL USE CUDA
...
1 Like

Would like to give you a million thumbs up, worked like a charm !!!
I’m following the lessons of Paul McWhorter and face recognition went from 0.9 fps to 7.6 fps !!!
Can’t thank you enough !!!

Hi,

Good to know it works now.
You can also find some information for dlib+JetPack4.4 here:

Thanks.