Service Maker C++ API ClassifierMetadata not finished?

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU): dGPU and Jetson Orin
• DeepStream Version: 8.0
• JetPack Version (valid for Jetson only): Latest
• TensorRT Version: Shipped with Deepstream Versions
• NVIDIA GPU Driver Version (valid for GPU only): 580.126.09
• Issue Type( questions, new requirements, bugs): BUG
• How to reproduce the issue ? I am attempting to migrate my C Deepstream application to using the Service Maker C++ API but it doesn’t look like a lot of the features are finished. Specifically looking at the ClassifierMetadata in a probe I am creating. The ClassifierMetadata only appears to expose labels and not confidence score from the SGIE.
**• Requirement details: The ClassifierMetadata should also include confidence score.
**
I am attempting to write a probe to show the SGIE Classification Label and Confidence score but I am only seeing the label being exposed in the ClassifierMetadata class:

#pragma once

#include "buffer_probe.hpp"

using namespace deepstream;

class SGIELabelOverlayProbe : public BufferProbe::IBatchMetadataObserver {
    public:
    
    probeReturn handleData(BufferProbe& probe, const BatchMetadata& batch) override {
        
        // Iterate through all frames in the batch
        batch.iterate([](const FrameMetadata& frame) {
            
            // Iterate through all objects in the frame
            frame.iterate([](const ObjectMetadata& obj) {
                
                // We will build a new label string here
                std::stringstream new_label;
                
                // Start with the Primary GIE label (e.g., "Car")
                // Note: getLabel() typically returns the PGIE class label
                new_label << obj.label(); 

                // Iterate through Classifier Metadata (SGIE results) attached to this object
                obj.iterate([](const ClassifierMetadata& classifier) {
                    
                    
                    // Iterate through the labels generated by this classifier
                    classifier.iterate([](const LabelMetadata& label) {
                        // Append the SGIE label (e.g., "Color: Red")
                        // You can format this string however you want
                        // label.label() returns the string text of the classification
                        new_label << " [" << label.label() << "]"; 
                    });
                    
                });

                // Update the object's display text to include the SGIE info.
                // This ensures the OSD plugin draws the full text above the bounding box.
                // Note: The setLabel method updates the text_params.display_text used by OSD.
                const_cast<ObjectMetadata&>(obj).setLabel(new_label.str());
            });
        });

        return probeReturn::Probe_Ok;
    }
};

The ClassifierMetadata class in the metadata.hpp looks like so:

/** @brief Meta generated from a classifier */
class ClassifierMetadata : public Metadata {
public:
  typedef std::unique_ptr<AbstractIterator<ClassifierMetadata>> Iterator;

  /**
   * @brief Constructor through opaque data pointer
   *
   * By default an empty classifier metadata object is created.
   *
   */
  ClassifierMetadata(void* data = nullptr);
  /** @brief Destructor */
  virtual ~ClassifierMetadata();

  /** @brief Number of the lables generated by the classifier */
  unsigned int nLabels() const;
  /** @brief Identifying the unique component that generates the metadata */
  unsigned int uniqueComponentId() const;
  /** @brief Get the nth label */
  std::string getLabel(unsigned int nth) const;

};

Why isn’t confidence score exposed along with the label?

The confidence of the label is not implemented now. We will consider to add it.

Thanks for the speedy reply! Is there a workaround you can recommend to fetch confidence score from the SGIE classifier in a probe with ServiceMaker C++ API?

Really need this confidence score in production if we are to migrate to the ServiceMaker C++ API from the C Deepstream application. It was easy to fetch SGIE classifier confidence with a probe in C code.

Unfortunately, there is no API to get the confidence value through Service Maker.

Anyway to workaround it with a C plugin probe or something like that?

The only way to work around is to implement a customized “ClassifierMetadata1” class(derived from ‘Metadata’ class ), and use this class to get what you want.

Thanks! Not seeing the source file for “metadata.hpp”, is that open source? Wouldn’t I need to see the source file to see how to properly access the buffer data?

I see in this file /opt/nvidia/deepstream/deepstream-8.0/service-maker/cmake# cat nvds_service_makerConfig.cmake:

####################################################################################################
# SPDX-FileCopyrightText: Copyright (c) 2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: LicenseRef-NvidiaProprietary
#
# NVIDIA CORPORATION, its affiliates and licensors retain all intellectual
# property and proprietary rights in and to this material, related
# documentation and any modifications thereto. Any use, reproduction,
# disclosure or distribution of this material and related documentation
# without an express license agreement from NVIDIA CORPORATION or
# its affiliates is strictly prohibited.
####################################################################################################

find_path(nvds_service_maker_INCLUDE_DIR includes HINTS /opt/nvidia/deepstream/deepstream/service-maker)
find_library(nvds_service_maker_LIBRARY NAMES nvds_service_maker HINTS /opt/nvidia/deepstream/deepstream/lib)
find_library(nvds_plugin_LIBRARY NAMES nvds_service_maker_utils HINTS /opt/nvidia/deepstream/deepstream/lib)

find_package(PkgConfig REQUIRED)
pkg_check_modules(GLIB REQUIRED glib-2.0)

if (nvds_service_maker_INCLUDE_DIR AND nvds_service_maker_LIBRARY AND nvds_plugin_LIBRARY)
    set(nvds_service_maker_FOUND TRUE)
    set(DEEPSTREAM_SDK_INC_DIR "/opt/nvidia/deepstream/deepstream/sources/includes")
    add_library(nvds_service_maker SHARED IMPORTED)
    target_include_directories(nvds_service_maker INTERFACE "${nvds_service_maker_INCLUDE_DIR}/includes" ${DEEPSTREAM_SDK_INC_DIR})
    set_target_properties(nvds_service_maker PROPERTIES IMPORTED_LOCATION "${nvds_service_maker_LIBRARY}")
    target_compile_features(nvds_service_maker INTERFACE cxx_std_17)
    add_library(nvds_service_maker_utils STATIC IMPORTED)
    set_target_properties(nvds_service_maker_utils PROPERTIES IMPORTED_LOCATION "${nvds_plugin_LIBRARY}")
else ()
    set(nvds_service_maker_FOUND FALSE

Looks like only the includes are available and point to a shared library that has the actual implementation. Anyway you could give me an example header for your recommendation to “implement a custom “ClassifierMetadata1” class(derived from Metadata class)”?

The ‘ClassifierMetadata’ class implementation is not open source. You need to implement your own.