Face recognition in deepstream_tao_apps

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Hello

Device: Jetson Xavier NX
Firmware: JP5.1

I’m using a pipeline containing a sequence of emotion and gaze detectors (based on an emotion detector application where the gaze detector was added to the pipeline after the emotion detector).

Is it possible to recognize different people?
For example, one person went out of view of the camera, then returned back (or, for example, if people switched places), will the ID binding to the face of a certain person be preserved?

I tried to output NvDsObjectMeta fields in the static GstPadProbeReturn pgie_pad_buffer_probe (GstPad * pad, GstPadProbeInfo * info, gpointer u_data) callback in deepstream_emotion_app.cpp and in the void EmotionAlgorithm::OutputThread(void) method in emotion_impl.cpp:

//...
for(NvDsMetaList * l_obj = frame_meta->obj_meta_list; l_obj != NULL; l_obj = l_obj->next)
{
    NvDsObjectMeta *obj_meta = (NvDsObjectMeta *)l_obj->data;
    if(!obj_meta)
    {
        g_print("No obj meta\n");
        break;
    }
    
    g_print("%s: ---NvDsObjectMeta info---\n", __PRETTY_FUNCTION__);
    g_print("obj_meta->class_id            = %d\n", obj_meta->class_id);
    g_print("obj_meta->object_id           = %ld\n", obj_meta->object_id);
    g_print("obj_meta->obj_label           = %s\n", obj_meta->obj_label);
    g_print("obj_meta->unique_component_id = %d\n", obj_meta->unique_component_id);
    g_print("obj_meta->rect_params         = {%f,%f,%f,%f})\n----------------------\n",
        obj_meta->rect_params.left,
        obj_meta->rect_params.top,
        obj_meta->rect_params.width,
        obj_meta->rect_params.height);
    //...
}
//...

As a result, I got an output like:

void EmotionAlgorithm::OutputThread(): ---NvDsObjectMeta info---
obj_meta->class_id            = 0
obj_meta->object_id           = -1
obj_meta->obj_label           = Face
obj_meta->unique_component_id = 1
obj_meta->rect_params         = {545.970703,194.758850,361.450195,361.450195})
----------------------
Frame Number = 373 Face Count = 2
GstPadProbeReturn pgie_pad_buffer_probe(GstPad*, GstPadProbeInfo*, gpointer): ---NvDsObjectMeta info---
obj_meta->class_id            = 0
obj_meta->object_id           = -1
obj_meta->obj_label           = Face
obj_meta->unique_component_id = 1
obj_meta->rect_params         = {1024.824219,490.591949,115.467903,148.661926})
----------------------
2

Hi,
This looks more related to Deepstream. We are moving this post to the Deepstream forum to get better help.
Thank you.

4

it is a tracker question, it depends on the time length of leaving and tracker method. please refer to tracker
object_id holds a unique ID for tracking the object.

5

Hello
Initially, I asked the question in the context of using emotion and gaze detectors, i.e. as a kind of black box that takes input data and produces a result.
In the topic header, I may not have sufficiently disclosed the essence of my question. I’ll clarify my question from the topic header - keep the binding to people’s faces and distinguish: which person is smiling, which one is sad, which one is shocked, etc. Is it possible inside the code of the detectors to distinguish the faces of different people?

6

yes, you can add a tracker after pgie(facenet), tracker will assign an unique id for every object(face), then you can know which person is smiling or other expressions.

7

Hello
I have tried running several sample applications.
To get acquainted with the work of these, I tried to run several examples (for example /opt/nvidia/deepstream/deepstream-6.2/sources/apps/sample_apps/deepstream-test2).
Unfortunately, the application on the test video shows a lot of false positives (there are two people on the test video, but, nevertheless, the number of people increases when the frame changes and the detector finds “ghosts” - it draws extra frames either intersecting with real people in the frame, or somewhere on the wall, etc.).

videos

Screencast from 08.06.2023.tar.xz (81.2 MB)

8

There is no update from you for a period, assuming this is not an issue any more.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

Thanks for the sharing, it is only a sample of model using, the model is not well trained, you can try other model with good accuracy.

9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.