Issue using binary classifier in deepstream

How do I differentiate between positive and negative class from a binary classifier in python code ?

I have a primary detector and a secondary binary classifier, I want the differentiate the boxes that were classified as true (got a confidence higher than the threshold) from the boxes that didn’t (had a confidence less than threshold)
threshold is specified in the secondary config file as: classifier-threshold=0.51
I’m using python, I tried to differentiate them using

current_obj_classification.result_class_id but it gives any number between 0 and 10k+ (instead of just [0-1] or [0-none],
current_obj_classification.result_label I expected the label number but it gave me an array of length 128 but my last layer has a shape of (1),
current_obj_classification.result_prob I expected a float between [0,1] but it gave large values (+ve and -ve)

# where 
    classifier_base_meta = obj_meta.classifier_meta_list.data
    obj_class_meta = pyds.NvDsClassifierMeta.cast(classifier_base_meta)
    obj_meta_label_info = obj_class_meta.label_info_list
    current_obj_classification = pyds.NvDsLabelInfo.cast(obj_meta_label_info)

• Hardware Platform GPU
• DeepStream Version 5
• TensorRT Version 7.0
• NVIDIA GPU Driver Version 450.51.05
• Issue Type question

2 Likes

@bcao

Hello again, I made a custom parsing function for my classifier
I assigned the values where the sample did
but when I print this in my python code:

print(f"result_class_id {current_obj_classification.result_class_id}, "
                             f"label_id:{current_obj_classification.label_id} "
                             f"number of labels {obj_class_meta.num_labels} "
                             # f"\nlabel {current_obj_classification.result_label}, "
                             f"result label length {len(current_obj_classification.result_label)}\n"
                             f"result probability {current_obj_classification.result_prob}"
                             f"\n\n--------------------------------------------")

I get

result_class_id 0, label_id:32748 number of labels 1 result label length 128
result probability 0.0

--------------------------------------------
result_class_id 0, label_id:0 number of labels 1 result label length 128
result probability 2.0037627128951483e-16

--------------------------------------------
result_class_id 0, label_id:0 number of labels 1 result label length 128
result probability 8918679552.0

--------------------------------------------
result_class_id 0, label_id:0 number of labels 1 result label length 128
 result probability 0.0
 --------------------------------------------

this is the function I call in my config
custom-lib-path=libnvds_infercustomparser.so
parse-classifier-func-name=NvDsInferClassiferParseCustomSoftmax

(the model itself has a softmax (None,1) but I want to express two classes with it like it shows here, does that affect the output? )

    bool NvDsInferClassiferParseCustomSoftmax (std::vector<NvDsInferLayerInfo> const &outputLayersInfo,
            NvDsInferNetworkInfo  const &networkInfo,
            float classifierThreshold,
            std::vector<NvDsInferAttribute> &attrList,
            std::string &descString)
    {
        /* Get the number of attributes supported by the classifier. */
        unsigned int numAttributes = outputLayersInfo.size();

        /* Iterate through all the output coverage layers of the classifier.
        */
    //    std::cout << "number of attributes "<< numAttributes << std::endl; 
        for (unsigned int l = 0; l < numAttributes; l++)
        {
            /* outputCoverageBuffer for classifiers is usually a softmax layer.
             * The layer is an array of probabilities of the object belonging
             * to each class with each probability being in the range [0,1] and
             * sum all probabilities will be 1.
             */
            NvDsInferDimsCHW dims;
            getDimsCHWFromDims(dims, outputLayersInfo[l].inferDims);
            unsigned int numClasses = dims.c;

            float *outputCoverageBuffer = (float *) outputLayersInfo[l].buffer;
            // float maxProbability = 0;
            bool attrFound = false;
            NvDsInferAttribute attr;
            
            /*  Unlike the original function: this function is made for a classifier with one probability and 2 classes 
            * 0 emergency 1 non_emergency, so instead of looping over classes we'll just compare the confidenece to the 
            * threshold where (number> thresh) => none emergency | (number < thresh) => emergency */
            float probability = outputCoverageBuffer[0]; //[c];
            if (probability > classifierThreshold)
            {
                attrFound = true;
                attr.attributeIndex = l; //l;
                attr.attributeValue = 1; //c;
                attr.attributeConfidence = (probability-classifierThreshold) / (1-classifierThreshold);
            } 
            else
            { // <=  thresh
                attrFound = true;
                attr.attributeIndex = l;
                attr.attributeValue = 0;
                attr.attributeConfidence = (classifierThreshold-probability)/classifierThreshold;
            }
     
            // std::cout << "***************************" << std::endl;
            // std::cout << "Layer name " << outputLayersInfo[l].layerName  << "\nattribute index " << attr.attributeIndex << 
            // " attribute value " << attr.attributeValue <<
            // " calculated confidence " << attr.attributeConfidence << " original confidence " << probability << std::endl;
            // std::cout << "***************************" << std::endl << std::endl;

            if (attrFound)
            {
                if (labels.size() > attr.attributeIndex &&
                        attr.attributeValue < labels[attr.attributeIndex].size())
                    attr.attributeLabel =
                        labels[attr.attributeIndex][attr.attributeValue].c_str();
                else
                    attr.attributeLabel = nullptr;
                attrList.push_back(attr);
                if (attr.attributeLabel)
                    descString.append(attr.attributeLabel).append(" ");
            }
        }

        return true;
    }

The rest of the file at nvdsinfer_customclassifierparser.cpp remains unchanged

#include <cstring>
#include <iostream>
#include "nvdsinfer_custom_impl.h"

/* This is a sample classifier output parsing function from softmax layers for
 * the vehicle type classifier model provided with the SDK. */

/* C-linkage to prevent name-mangling */
extern "C"
bool NvDsInferClassiferParseCustomSoftmax (std::vector<NvDsInferLayerInfo> const &outputLayersInfo,
        NvDsInferNetworkInfo  const &networkInfo,
        float classifierThreshold,
        std::vector<NvDsInferAttribute> &attrList,
        std::string &descString);

static std::vector < std::vector< std:: string > > labels { {
    "coupe1", "largevehicle1", "sedan1", "suv1", "truck1", "van1"} };

extern "C"
bool NvDsInferClassiferParseCustomSoftmax (std::vector<NvDsInferLayerInfo> const &outputLayersInfo,
        NvDsInferNetworkInfo  const &networkInfo,
        float classifierThreshold,
        std::vector<NvDsInferAttribute> &attrList,
        std::string &descString)
{
    /* Get the number of attributes supported by the classifier. */
    unsigned int numAttributes = outputLayersInfo.size();

    /* Iterate through all the output coverage layers of the classifier.
    */
//    std::cout << "number of attributes "<< numAttributes << std::endl; 
    for (unsigned int l = 0; l < numAttributes; l++)
    {
        /* outputCoverageBuffer for classifiers is usually a softmax layer.
         * The layer is an array of probabilities of the object belonging
         * to each class with each probability being in the range [0,1] and
         * sum all probabilities will be 1.
         */
        NvDsInferDimsCHW dims;
        getDimsCHWFromDims(dims, outputLayersInfo[l].inferDims);
        unsigned int numClasses = dims.c;

        float *outputCoverageBuffer = (float *) outputLayersInfo[l].buffer;
        // float maxProbability = 0;
        bool attrFound = false;
        NvDsInferAttribute attr;
        
        /*  Unlike the original function: this function is made for a classifier with one probability and 2 classes 
        * 0 emergency 1 non_emergency, so instead of looping over classes we'll just compare the confidenece to the 
        * threshold where (number> thresh) => none emergency | (number < thresh) => emergency */
        float probability = outputCoverageBuffer[0]; //[c];
        if (probability > classifierThreshold)
        {
            attrFound = true;
            attr.attributeIndex = l; //l;
            attr.attributeValue = 1; //c;
            attr.attributeConfidence = (probability-classifierThreshold) / (1-classifierThreshold);
        } 
        else
        { // <=  thresh
            attrFound = true;
            attr.attributeIndex = l;
            attr.attributeValue = 0;
            attr.attributeConfidence = (classifierThreshold-probability)/classifierThreshold;
        }
 
        // std::cout << "***************************" << std::endl;
        // std::cout << "Layer name " << outputLayersInfo[l].layerName  << "\nattribute index " << attr.attributeIndex << 
        // " attribute value " << attr.attributeValue <<
        // " calculated confidence " << attr.attributeConfidence << " original confidence " << probability << std::endl;
        // std::cout << "***************************" << std::endl << std::endl;

        if (attrFound)
        {
            if (labels.size() > attr.attributeIndex &&
                    attr.attributeValue < labels[attr.attributeIndex].size())
                attr.attributeLabel =
                    labels[attr.attributeIndex][attr.attributeValue].c_str();
            else
                attr.attributeLabel = nullptr;
            attrList.push_back(attr);
            if (attr.attributeLabel)
                descString.append(attr.attributeLabel).append(" ");
        }
    }

    return true;
}

/* Check that the custom function has been defined correctly */
CHECK_CUSTOM_CLASSIFIER_PARSE_FUNC_PROTOTYPE(NvDsInferClassiferParseCustomSoftmax);

1 Like

Thanks, will check and get back to you.

2 Likes

Thank you,
I sent you a private message with the rest of the files, in case they could help you reproduce the issue

@aya95

1 Like

Hello @bcao ^^
Thank you for helping, any news ?

1 Like

so the issue that cause the numbers to be different from the parsing function output was that I didn’t iterate the label info list and the classifier meta list as I only needed the first -and the only- object in each of them

but I had to add the part that tries to access the next element
classifier_meta_list = classifier_meta_list.next
and breaks when it gets the exception

it started working as intended when I added that part for both classifier_meta_list and label_info_list

Hi @mai.algendy
I custom a parsing function for my model classifier, but I got core dump. I have no idea where I’m wrong.
I post my issue in here: Core dumped when using custom classifier parsing function
Any help, pls?