Multiple faces in same video not recognized properly

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

Hi,

I took the code here:

We trained it for our office colleagues’ faces, and were seeing decent performance.

But, as soon as I create 2 parallel camera streams (simply creating 2 instances for 2 different cameras), and put the tensorrt inferencing inside a for loop, I see that it cannot anymore recognise more than 1 person in the same camera stream.

What works:
Single camera with any number of people
Multiple camera with each camera having only one face

What doesn’t work:
Multiple camera with any one camera having more than one face detected.
“lab” variable shows “-1” for all the recognition labels except the first one.

I have tried it on saved video file to be sure, and for same instance, single source setup is able to classify all the detections.

Also, when I print the confidence scores from “lab” variable, in the scenario where multiple faces are detected, their sum seems to be adding up to nearly 100%. It seems that the network is applying some sort of softmax across all the detections, even if they are non-overlapping.

Regards,
Anup

2

Hi,

This sample doesn’t support multiple input stream.

There is a face tracking table in the recognition plugin implementation.
If you have more than one input source, please duplicate the table to the corresponding amount.
[url]https://github.com/AastaNV/Face-Recognition/blob/master/pluginImplement.cpp#L382[/url]

Thanks.

3

Hi,

We just observed that if there are multiple ROIs in same stream also, there seems to be sort of a softmax applied on the “lab” variable. Only one ROI seems to be passed on to TensorRT according to the logs too. If there are 3 faces, and one is 98% confident, we see other confidence scores total to only 2%. Have you tested on images with multiple subjects?

4

Hi,

Due to the pipeline design, only one ROI can be passed into the classification network.
Here is the source to choose an ROI region:
[url]https://github.com/AastaNV/Face-Recognition/blob/master/pluginImplement.cpp#L345[/url]

In this sample, we didn’t output the classification confidence.
You can modify the code here to get the information:
[url]https://github.com/AastaNV/Face-Recognition/blob/master/pluginImplement.cpp#L401[/url]

Thanks.