Simple accelerated face recognition

Hello,
What is the best way to do a simple accelerated face recognition with Jetson NX?

I have a trained a model with dlib/face_recognition on a desktop computer but I can’t install face_recognition on the Jetson. Not sure if it feasible as per Installing dlib with python3.6 in Jetson Nano

But also I don’t know if it will be accelerated…

I have also tried to install OpenCV with cuda but it failed (I need to try one more thing… ie ./build_opencv.sh master as described in https://github.com/mdegans/nano_build_opencv/issues/36)

I don’t understand why opencv is installed on the image but not accelerated… I am losing a lot of time to get going.

So I would welcome any advice for doing simple accelerated face recognition on Jetson NX (ideally leveraging the model that I already have)
Thanks!

1 Like

Hi,

We didn’t notice that dlib cannot be installed on the JetsonNX.
Could you share which kind of error do you meet with us first?

Thanks.

Hi,
So I have been able to compile OpenCV and it now says Cuda YES in jtop but it is still very slow…

I have also been able to install dlib, not sure why it finally worked.

So I am able to run some python script that do face recognition using dlib an hog but it is very slow and the GPU does not seem to be used a lot in jtop.
cnn model is even slower
I have used 720p input mode in Gstreamer and 360p output, but even at this low res I probably have around 1fps
Any idea?
Thanks,

Hi,

There is an issue in building dlib with cuDNN v8.0 library.
The reason is that the convolution API changed and make dlib to turn off the CUDA/cuDNN support.

This issue is fixed in the latest source from GitHub.
Would you mind to give it a try to see if helps first?

Thanks.

Hi,
Thanks for your message. Yes happy to try and revert.
Can you let me know how (and if) I can check if my current dlib build is correct? Beyond checking GPU usage…
Thanks,

Youri

Also can you let me know what I should do precisely?
Can I run pip3 install dlib?
Or do I need do build from git source? as described here:

Is it normal that the fix described in this post is not applied if I am not mistaken in the current git version?
Thank you

fyi I gave it a try (building from git) as I thought the fix you refered to was different. fyi I get following error when launching cmake:

– Looking for cuDNN install…
– Found cuDNN: /usr/lib/aarch64-linux-gnu/libcudnn.so
– Building a CUDA test project to see if your compiler is compatible with CUDA…
– Checking if you have the right version of cuDNN installed.
– *** Found cuDNN, but it looks like the wrong version so dlib will not use it. ***
– *** Dlib requires cuDNN V5.0 OR GREATER. Since cuDNN is not found DLIB WILL NOT USE CUDA. ***
– *** If you have cuDNN then set CMAKE_PREFIX_PATH to include cuDNN’s folder. ***
– Disabling CUDA support for dlib. DLIB WILL NOT USE CUDA

Screenshot from 2020-08-05 22-41-04

In case you’re interested in exploring a different model

This is an excellent post / implementation of face detection with TensorRT

https://jkjung-avt.github.io/tensorrt-mtcnn/

I’ve had personal success using this with Jetson (tested with NX).

This may be a good option for face recognition:

It also includes an MTCNN face detection implementation.

If you need higher performance you may be able to apply torch2trt to the backbone neural networks of these models.

I’m curious if this works for you, please let me know if you have any questions or run into any issues.

Best,
John

Thanks! I will try to have a look, I don’t know anything about these models and pytorch in general.
I like the idea about subsequent frame acceleration, I am very surprised that so many implementation are analysing video frames independently…
I am quite happy with the very easy implementation with face_recognition but quite slow.

I agree, intuitively it seems like more models could exploit the information computed from previous frames.

Cool, I haven’t actually tried the facenet-pytorch models myself, but I’m fairly experienced with PyTorch + TensorRT.

I’m maintaining https://github.com/NVIDIA-AI-IOT/torch2trt so interested to hear if it works here. Let me know if you get around to trying it. I’m happy to answer any questions around PyTorch + TensorRT, or help how I can getting this running.

Very curious to see what improvements you can get!

Best,
John

@jaybdub I started to look into it but not sure how to properly install mmcv for NX, do you know where I can find instructions? I understand there is a lite and a full version. Thanks!

@AastaLLL can you kindly help me with the dlib install and issues described above?
What has been solved? Should We still comment out the line to get it work?
What went wrong in my install above?
Thanks!

@yandssiegel
Do you have any issues to install dlib and face_recognition on Jetson Xavier NX until now?

Hello,
I struggled but finally managed
However it is slow and I don’t see the GPU being really used in jtop, see above.
So @AastaLLL mentioned an issue that has been fixed (don’t know if it is the same as the ‘comment out’ previous fix).
Anyway I tried to install again to see if it improves and did not work. Cf above.
Thks,

It was caught just before on my Jetson Xavier NX,
dlib and face_recognition applied, and GPU is working.
CNN instead of hog is used.

I installed dlib from source code, but face_recognition from pip3.

python3.6.9
dlib: 19.21.99
face_recognition: 1.2.3
cv2: 4.1.1

Thanks, what kind of fps do you get with a 640/480 image size and simple face recognition (I guess it also depned on the nb of known faces)
Do you use gstreamer for input?
Thks,

Hello,
Could someone help with this? It is not solved.
I have very poor result, less than 2fps, for simple dlib face recognition with a box drawing…
I don’t know if my install is correct (dlib? gstreamer? opencv??)
Or if I should be satisfied with this performance…

@AastaLLL mentioned a fix for dlib but I don’t know which fix he is referring to (cf above)

This post seems to be related but not closed either:

Thank you

Hi,

The fix is adding the cuDNNv8 new API support.
And it is available in the newest dlib-19.21 release in Aug 8.

So you can build it from source directly:

Install dependencies

$ sudo apt-get install python3-pip
$ sudo apt-get install libjpeg-dev

Build dlib from source

$ wget http://dlib.net/files/dlib-19.21.tar.bz2
$ tar jxvf dlib-19.21.tar.bz2
$ cd dlib-19.21/
$ mkdir build
$ cd build/
$ cmake ..
$ cmake --build .
$ cd ../
$ sudo python3 setup.py install
$ sudo pip3 install face_recognition

Testing

Please create Images folder and store some testing images in the folder.

test.py

import cv2
import os
import numpy as np
import dlib

face_locations = []
face_encodings = []

### Path where images are present for testing
imagefolderpath = "Images/"

### Model for face detection
face_detector = dlib.get_frontal_face_detector()

for image in os.listdir(imagefolderpath):
    image = cv2.imread(os.path.join(imagefolderpath,image),1)

    t = time.time()
    faces = face_detector(image,0)
    for face in faces:
        x,y,w,h = face.left(),face.top(),face.right(),face.bottom()
        face_locations.append((x,y,h,w))
    face_encodings = face_recognition.face_encodings(image, known_face_locations = face_locations, num_jitters = 1)

    for (left, top, bottom, right) in face_locations:
        cv2.rectangle(image, (left,top), (right, bottom), (0, 0, 255), 2)
        cv2.imshow('Image', image)
        cv2.waitKey(0)
        cv2.destroyAllWindows()
python3 test.py

You should be able to see CUDA and cuDNN are both enabled in the dlib-19.21.

...
-- Found CUDA: /usr/local/cuda (found suitable version "10.2", minimum required is "7.5") 
-- Looking for cuDNN install...
-- Found cuDNN: /usr/lib/aarch64-linux-gnu/libcudnn.so
-- Building a CUDA test project to see if your compiler is compatible with CUDA...
-- Building a cuDNN test project to check if you have the right version of cuDNN installed...
-- Enabling CUDA support for dlib.  DLIB WILL USE CUDA
-- C++11 activated.
-- Configuring done
-- Generating done
-- Build files have been written to: /home/nvidia/dlib-19.21/build

Thanks.

Thanks, I have managed to perform the step above. It seemed to have worked fine.

I was also able to run the example (after adding import face_recognition which was missing and removing import numpy and t=time.time() which is not used)

I have launched a live stream image recognition script and it is better it seems but not amazing (around 9fps)
I use gstreamer with a raspberry pi v2 cam. Can you recommend a gstreamer pipeline?
Thank you

Hi yandssiegel,

Regarding the gstreamer related issue, please help to open a new topic. Thanks