Multiple faces in same video not recognized properly

Hi,

I took the code here:

https://github.com/AastaNV/Face-Recognition

We trained it for our office colleagues’ faces, and were seeing decent performance.

But, as soon as I create 2 parallel camera streams (simply creating 2 instances for 2 different cameras), and put the tensorrt inferencing inside a for loop, I see that it cannot anymore recognise more than 1 person in the same camera stream.

What works:
Single camera with any number of people
Multiple camera with each camera having only one face

What doesn’t work:
Multiple camera with any one camera having more than one face detected.
“lab” variable shows “-1” for all the recognition labels except the first one.

I have tried it on saved video file to be sure, and for same instance, single source setup is able to classify all the detections.

Also, when I print the confidence scores from “lab” variable, in the scenario where multiple faces are detected, their sum seems to be adding up to nearly 100%. It seems that the network is applying some sort of softmax across all the detections, even if they are non-overlapping.

Regards,
Anup

Hi,

This sample doesn’t support multiple input stream.

There is a face tracking table in the recognition plugin implementation.
If you have more than one input source, please duplicate the table to the corresponding amount.
https://github.com/AastaNV/Face-Recognition/blob/master/pluginImplement.cpp#L382

Thanks.

Hi,

We just observed that if there are multiple ROIs in same stream also, there seems to be sort of a softmax applied on the “lab” variable. Only one ROI seems to be passed on to TensorRT according to the logs too. If there are 3 faces, and one is 98% confident, we see other confidence scores total to only 2%. Have you tested on images with multiple subjects?

Hi,

Due to the pipeline design, only one ROI can be passed into the classification network.
Here is the source to choose an ROI region:
https://github.com/AastaNV/Face-Recognition/blob/master/pluginImplement.cpp#L345

In this sample, we didn’t output the classification confidence.
You can modify the code here to get the information:
https://github.com/AastaNV/Face-Recognition/blob/master/pluginImplement.cpp#L401

Thanks.