What is the role of cluster-mode in pgie config file?

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU): Jetson AGX Xavier
• DeepStream Version: 5.0
• JetPack Version (valid for Jetson only): 4.4
• TensorRT Version: 10.2
• Issue Type( questions, new requirements, bugs): question

I want to match the values in NMS Config component in yolov3 experiment spec file to deepstream pgie config but I’m not sure how to go about it.

My main question is I don’t know what is the role of cluster-mode (i.e., what clustering is used for in nvinfer) ? My understand is the clustering used by deepstream are to group nearby bboxes together before NMS is applied, is that correct?

What I’ve done:

  1. Read the doc for Gst-nvinfer already and still can’t find the answer to my question.
  2. Followed the suggested options here and still have no affirmative answer to what clustering is used for in nvinfer.
3

which yolov3 are you using?
There are two yolov3 in DS, one is TLT yolov3 under /opt/nvidia/deepstream/deepstream/samples/configs/tlt_pretrained_models, the other is yolov3 under /opt/nvidia/deepstream/deepstream/sources/objectDetector_Yolo

4

Hi @mchi,

I used a yolov3 model trained via TLT.

5

please check if Setting a Confidence Algorithm - #13 by mchi helps your question.

6

Hi @mchi,

I read the linked answer but it doesn’t address my question. I want to ask what is the purpose of clustering in nvinfer, my understand is nvinfer uses the clustering method specidied by cluster-mode to group nearby bboxes together before NMS is applied, is that corrrect?

7

Hi @hyperlight
Shared the screenshot below again. The cluster-mode itself includes NMS.

image
image1386×164 26.5 KB

And, as you can find in code below, if you select cluster-mode with NMS, it will do NMS. If your model include NMS layer, you can select the culster-model without NMS.

File: /opt/nvidia/deepstream/deepstream-5.0/sources/libs/nvdsinfer/nvdsinfer_context_impl_output_parsing.cpp

NvDsInferStatus
DetectPostprocessor::fillDetectionOutput(
    const std::vector<NvDsInferLayerInfo>& outputLayers,
    NvDsInferDetectionOutput& output)
{
  ...
    switch (m_ClusterMode)
    {
        case NVDSINFER_CLUSTER_NMS:
            clusterAndFillDetectionOutputNMS(output);
            break;

        case NVDSINFER_CLUSTER_DBSCAN:
            clusterAndFillDetectionOutputDBSCAN(output);
            break;

        case NVDSINFER_CLUSTER_GROUP_RECTANGLES:
            clusterAndFillDetectionOutputCV(output);
            break;

        case NVDSINFER_CLUSTER_DBSCAN_NMS_HYBRID:
            clusterAndFillDetectionOutputHybrid(output);
            break;

        case NVDSINFER_CLUSTER_NONE:
            fillUnclusteredOutput(output);
            break;

        default:
            break;
    }
    ...
}