Strange behavior of the pre-cluster-threshold parameter

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)
• DeepStream Version
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

In my config file, cluster-mode is set to 4 (namely no clustering) since the decode and NMS operations are customized in nvdsparsebbox file. I think when no clustering mode is configured, the pre-cluster-threshold is not working. However, as long as this parameter is set, the bbox parsing will be affected. It has no effects at all only when it is commented out.

Can you show us the complete configuration file?

Thank you for your attention. Complete configuration is as follows:
[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
#0=RGB, 1=BGR
model-color-format=0
labelfile-path=labels.txt
onnx-file = model.onnx
model-engine-file = model.onnx_b1_gpu0_fp32.engine
uff-input-dims=3;800;1440;0

0=FP32, 1=INT8, 2=FP16 mode

network-mode=0
num-detected-classes=1
gie-unique-id=1
is-classifier=0
output-blob-names=output
maintain-aspect-ratio=1
parse-bbox-func-name=NvDsInferParseCustom
custom-lib-path=…/…/DS5.1/nvdsinfer_person/libnvdsinfer_custom_impl.so

0=Group Rectangles, 1=DBSCAN, 2=NMS, 3= DBSCAN+NMS Hybrid, 4 = None(No clustering)

cluster-mode=4

[class-attrs-all]
##nms-iou-threshold=0.5

detection threshold prior to clustering operation

##pre-cluster-threshold=0.2

The pre-cluster-threshold parameter is not used in the nvdsparsebbox cpp file, where bbox confidence_threshold (0.1) and nms confidence_threshold (0.7) are pre-defined.

Update:

Serveral debuggings show that objectList (type std::vector&) in nvdsparsebbox cpp file is pre-cluster-threshold independent, which means that nvinfer is correct. But the pre-cluster-threshold affects the num_obj_meta in batch_meta->frame_meta_list (batch_size=1) from gst_buffer_get_nvds_batch_meta function in osd_sink_pad_buffer_probe.

Execution flow of our deepstream:
src → nvstreammux → nvinfer → nvtracker → nvstreamdemux → queue → nvvidconv → nvosd → fakesink

Silimar to this question, setting pre-cluster-threshold to 0 or commentting out won’t affect performance

I find the solution.
The key is the function DetectPostprocessor::fillDetectionOutput in nvdsinfer_context_impl_output_parsing.cpp. The network-type not specified in config file is default to 0, namely Detector. Therefore nvdsinfer use DetectPostprocessor::fillDetectionOutput to perform post processing. DetectPostprocessor::fillDetectionOutput then invoke preClusteringThreshold (where the pre-cluster-threshold parameter is used) before whatever the luster model is.

Some suggestions:
@Fiona.Chen, I don’t think it is a reasonable design. Instead, it is misleading. The cluster-mode=4 means that no clustering is used. Therefore pre-cluster-threshold should not be working for this mode. Current design makes it applied for all clustering modes.

cluster-mode=4 is specified for instance segmentation case

https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_plugin_gst-nvinfer.html#gst-nvinfer-file-configuration-specifications

I have read the documentation before and test it in my debuggings. I add some debug infos in fillDetectionOutput implementation of class DetectPostprocessor and InstanceSegmentPostprocessor in nvdsinfer_context_impl_output_parsing.cpp and recompile the libnvds_infer.so. The result show that the configuration of network-type=0 (Detector) and cluster-mode=4(Instance segmentation) will invoke Detect’s Postprocessor rather than InstanceSegment’s Postprocessor. However, both the implementations perform preClusteringThreshold.

The codes showing the result are as follows:
In InstanceSegmentPostprocessor::fillDetectionOutput

preClusteringThreshold(m_DetectionParams, m_InstanceMaskList);
switch (m_ClusterMode)
{
    case NVDSINFER_CLUSTER_NONE:
        fillUnclusteredOutput(output);
        break;
    default:
        printError("Invalid cluster mode for instance mask detection");
        return NVDSINFER_OUTPUT_PARSING_FAILED;
}

In DetectPostprocessor::fillDetectionOutput

preClusteringThreshold(m_DetectionParams, m_ObjectList);
......
switch (m_ClusterMode)
{
    case NVDSINFER_CLUSTER_NMS:
        clusterAndFillDetectionOutputNMS(output);
        break;

    case NVDSINFER_CLUSTER_DBSCAN:
        clusterAndFillDetectionOutputDBSCAN(output);
        break;

    case NVDSINFER_CLUSTER_GROUP_RECTANGLES:
        clusterAndFillDetectionOutputCV(output);
        break;

    case NVDSINFER_CLUSTER_DBSCAN_NMS_HYBRID:
        clusterAndFillDetectionOutputHybrid(output);
        break;

    case NVDSINFER_CLUSTER_NONE:
        fillUnclusteredOutput(output);
        break;

    default:
        break;
}

cluster-mode=4 is for both detector and instance segmentation cases. What confused to me is that why preClusteringThreshold is necessary even if no clustering is used.

The postprocessor is chosen by “network-type”.

preClusteringThreshold provides more choices for the users to control the post-processing. If you don’t want any postprocessing, just set preClusteringThreshold value as 0.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.