Reidentify the tracked ID

[orkhan]

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)

• GPU
  • DeepStream Version
• 6.2
  • JetPack Version (valid for Jetson only)
  • TensorRT Version
• 8.5
  • NVIDIA GPU Driver Version (valid for GPU only)
  • Issue Type( questions, new requirements, bugs)
• questions

Hi, I have a face recognition deepstreamer pipeline which includes detector → tracker (NVDeepSort) → featurizer plugins.

My issue is related to nvtracker plugin, sometimes detector detects the face which is not clearly visible. Tracker is assigning the ID to the detected object and passes it to featurizer (recognizer). As the face is not clear recognizer fails to recognize the face but the tracker keeps the same ID and dont send data for recognition again.

I want to run featurizer for unrecognized tracked ID for N times (frames) and try to identify the person, if it is still fails just label it UNKNOWN, or if one time I managed to recognize assign the label.

How can I achieve this?

Thanks in advance.

[kesong]

Can you show picture or video to show “the tracker keeps the same ID and dont send data for recognition again”?

[orkhan]

@orkhan

l.

How can I achieve this?

Thanks in advance.

As far as I know that is the normal behavior of any tracker, not to send same data to the bigger model that comes after the tracker. So, once tracker assigns ID it will just use that ID until ID is lost. The moment it loses the track, it sends data to bigger model to recognize again the face.

I just want to have control over tracker ID, to drop it when I want. Can I achieve this?

[kesong]

Do you mean “classifier-async-mode” in nvinfer? Gst-nvinfer — DeepStream 6.3 Release documentation

[orkhan]

I think I will use my own tracker algorithm which is feature based tracker and then it applies some post-processing logic to pass to recognition model.

Tracker here is the classifier plugin, which get the features for each detected face. Now I want to send to the next plugin user_meta which will contain custom struct containing frame_dets, tracked_dets, added_ids.

How can I parse this information in C++ .so file to the next plugin.

Below is the example of detector .so file:

extern "C" bool NvDsInferParseCustomSCRFD(
    std::vector<NvDsInferLayerInfo> const& outputLayersInfo,
    NvDsInferNetworkInfo const& networkInfo,
    NvDsInferParseDetectionParams const& detectionParams,
    std::vector<NvDsInferParseObjectInfo>& objectList)
{
    std::vector<NvDsInferParseObjectInfo> objects;
    const NvDsInferLayerInfo &n_bboxes   = outputLayersInfo[0];
    const NvDsInferLayerInfo &boxes      = outputLayersInfo[1]; // (num_boxes, 4)
    const NvDsInferLayerInfo &scores     = outputLayersInfo[2]; // (num_boxes, )
    const NvDsInferLayerInfo &classes    = outputLayersInfo[3]; // (num_boxes, )
    const NvDsInferLayerInfo &landmarks  = outputLayersInfo[4]; // (num_boxes, )

    int num_bboxes = *(const int*)(n_bboxes.buffer);
    // std::cout << "Got from plugin " << std::to_string(num_bboxes) << " faces" << std::endl;

    assert(boxes.inferDims.numDims == 2);
    assert(scores.inferDims.numDims == 1);
    assert(classes.inferDims.numDims == 1);

    std::vector<NvDsInferParseObjectInfo> outObjs =
        decodeSCRFDTensor(
            (const float*)(boxes.buffer), (const float*)(scores.buffer), (const float*)(classes.buffer), (const float*)(landmarks.buffer), num_bboxes, detectionParams,
            networkInfo.width, networkInfo.height);

    objects.insert(objects.end(), outObjs.begin(), outObjs.end());

    objectList = objects;

    return true;
}

[kesong]

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

NvDeepSORT tracker on DeepStream 6.3 can output Re-ID features Do you want to use it? Gst-nvtracker — DeepStream 6.3 Release documentation