Custom pgie output field for object detection

[young2theMax]

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) AGX Orin
• DeepStream Version 6.2
• JetPack Version (valid for Jetson only) 5.0.2
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs) Questions
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

Hi, I have a object detection model that makes the following outputs.
[c_x, c_y, score, index, expectedVolume].
In the pipeline, PGIE → custom-lib → tracker, I put the probe function after tracker and need to use expectedVolume.

In object detection, the applications use NvDsInferParseObjectInfo but there is no additional field that can be used in this application.

How can I put new field in NvDsInferParseObjectInfo?
Or is there any other way to maintain the value?

[yuweiw]

You can consider using metadata to carry this value, please refer to our source code:sources\apps\sample_apps\deepstream-gst-metadata-test\deepstream_gst_metadata.c.

[young2theMax]

Is it possible to use usermeta in custom-parser that comes right after pgie?
Cause when I try to use batchmeta in custom-parser function, I get this error.

static void checkFunc_ ## customParseFunc (NvDsInferParseCustomFunc func = customParseFunc)
| ^
| |
| bool ()(const std::vector&, const NvDsInferNetworkInfo&, const NvDsInferParseDetectionParams&, NvDsBatchMeta&, std::vector&) {aka bool ()(const std::vector&, const NvDsInferNetworkInfo&, const NvDsInferParseDetectionParams&, _NvDsBatchMeta&, std::vector&)}

[yuweiw]

@young2theMax

custom-lib

Is your custom-lib a GStreamer-based plugin? You can just use the metadata for Gstreamer.

[young2theMax]

The custom parser uses nvdsinfer_custom_impl.h and made as so file.

/**
 * Type definition for the custom bounding box parsing function.
 *
 * @param[in]  outputLayersInfo A vector containing information on the output
 *                              layers of the model.
 * @param[in]  networkInfo      Network information.
 * @param[in]  detectionParams  Detection parameters required for parsing
 *                              objects.
 * @param[out] objectList       A reference to a vector in which the function
 *                              is to add parsed objects.
 */
typedef bool (* NvDsInferParseCustomFunc) (
        std::vector<NvDsInferLayerInfo> const &outputLayersInfo,
        NvDsInferNetworkInfo  const &networkInfo,
        NvDsInferParseDetectionParams const &detectionParams,
        std::vector<NvDsInferObjectDetectionInfo> &objectList);

/**
 * Validates a custom parser function definition. Must be called
 * after defining the function.
 */
#define CHECK_CUSTOM_PARSE_FUNC_PROTOTYPE(customParseFunc) \
    static void checkFunc_ ## customParseFunc (NvDsInferParseCustomFunc func = customParseFunc) \
        { checkFunc_ ## customParseFunc (); }; \
    extern "C" bool customParseFunc (std::vector<NvDsInferLayerInfo> const &outputLayersInfo, \
           NvDsInferNetworkInfo  const &networkInfo, \
           NvDsInferParseDetectionParams const &detectionParams, \
           std::vector<NvDsInferObjectDetectionInfo> &objectList);

Here I added NvDsBatchMeta *batchMeta.
When I try to use it, I get segmentation fault.
Here is the snippet of my code.

extern "C" bool CustomParsing(
    std::vector<NvDsInferLayerInfo> const &outputLayersInfo,
    NvDsInferNetworkInfo const &networkInfo,
    NvDsInferParseDetectionParams const &detectionParams,
    std::vector<NvDsInferParseObjectInfo> &objectList,
    NvDsBatchMeta *batchMeta) {
  bool parsingSuccess = CustomParsingMain(outputLayersInfo, networkInfo,
                                              detectionParams, objectList);
  
  if (parsingSuccess && batchMeta != nullptr) {
    NvDsFrameMeta *firstFrameMeta = nullptr;
    if (batchMeta != nullptr && batchMeta->frame_meta_list != nullptr) {
        firstFrameMeta = (NvDsFrameMeta *)(batchMeta->frame_meta_list->data);
    }
  }
  
  return parsingSuccess;                                            
}

CHECK_CUSTOM_PARSE_FUNC_PROTOTYPE(CustomParsing);

But here, raise segmentation fault error.

  if (parsingSuccess && batchMeta != nullptr) {
    NvDsFrameMeta *firstFrameMeta = nullptr;
    if (batchMeta != nullptr && batchMeta->frame_meta_list != nullptr) {
        firstFrameMeta = (NvDsFrameMeta *)(batchMeta->frame_meta_list->data);
    }
  }

[yuweiw]

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

OK. So the custom-lib is just a postprocess. We do not currently support your expansion of the existing structure. If you have new requirements, it is still recommended that you implement the parsing yourself in the probe function. We have many simaliar demo to analyze the output in the probe function, like deepstream_bodypose2d_app.cpp.