# 0 MAP using NMS in custom detectnet

**URL:** <https://forums.developer.nvidia.com/t/0-map-using-nms-in-custom-detectnet/205443>\
**Category:** TAO Toolkit\
**Created:** [March 8, 2022, 3:37pm UTC](https://forums.developer.nvidia.com/t/0-map-using-nms-in-custom-detectnet/205443 "2022-03-08T15:37:45Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![KGerry](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/kgerry/32/173604_2.png) [@KGerry](https://forums.developer.nvidia.com/u/KGerry)\
**Post date:** [March 8, 2022, 3:37pm UTC](https://forums.developer.nvidia.com/t/0-map-using-nms-in-custom-detectnet/205443/1 "2022-03-08T15:37:45Z")

</div>

Please provide the following information when requesting support.

• Hardware (V100/Xavier/)  
• Network Type (Detectnet\_v2)  
• TLT Version (latest)  
• Training spec file(If have, please share here)  
• How to reproduce the issue ? (This is for errors. Please share the command line and the detailed log here.)

I watn to try out NMS as custering config in my detectnet\_v2 object detector. Using the default specification files (that include DBSCAN settings) I can get a pretty good result with a MAP 0f 60%, which is pretty ood considering the complexity of the objects.

The documentation states I should use the following values;  
nms\_iou\_threshold float 0.2  
nms\_confidence\_threshold float 0. (0. is this a typo?)

When replacing the DBSCAN lines with the following values:  
clustering\_algorithm: NMS  
nms\_iou\_threshold: 0.5  
nms\_confidence\_threshold: 0.4

The MAP is 0.x and no detection at all

Can someone tell me what values I should use for NMS? Do I need to put in topK as well?

---

<div class="post-metadata">

**Author:** ![Morganh](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/morganh/32/9748_2.png) [@Morganh](https://forums.developer.nvidia.com/u/Morganh)\
**Post date:** [March 8, 2022, 5:16pm UTC](https://forums.developer.nvidia.com/t/0-map-using-nms-in-custom-detectnet/205443/2 "2022-03-08T17:16:37Z")

</div>

Could you try more nms\_iou\_threshold and nms\_confidence\_threshold?

---

<div class="post-metadata">

**Author:** ![KGerry](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/kgerry/32/173604_2.png) [@KGerry](https://forums.developer.nvidia.com/u/KGerry)\
**Post date:** [March 9, 2022, 4:13pm UTC](https://forums.developer.nvidia.com/t/0-map-using-nms-in-custom-detectnet/205443/3 "2022-03-09T16:13:03Z")

</div>

@Morganh I did bump it up, but killed the process after 10 epochs because MAP remained 0.x  
I tried with different values, but no change

Here’s the postprocessing that worked **good** with **DBSCAN** :

```auto
> postprocessing_config {
> target_class_config {
> key: " *******"
> value {      
> clustering_config {
> clustering_algorithm: DBSCAN
> dbscan_confidence_threshold: 0.9
> coverage_threshold: 0.005
> dbscan_eps: 0.15
> dbscan_min_samples: 0.05
> minimum_bounding_box_height: 10

```

Here’s the postprocessing that I’m trying now that gives **MAP 0.x** with **NMS** :

```auto
> postprocessing_config {
> target_class_config {
> key: " *******"
> value {
> clustering_config {
> clustering_algorithm: NMS
> nms_iou_threshold: 0.6
> nms_confidence_threshold: 0.6
> minimum_bounding_box_height: 10

```

Here’s the Eval config

```auto
> evaluation_config {
> validation_period_during_training: 10
> first_validation_epoch: 10
> minimum_detection_ground_truth_overlap {
> key: " *******"
> value: 0.4
> }
> evaluation_box_config {
> key: " *******"
> value {
> minimum_height: 10
> maximum_height: 9999
> minimum_width: 10
> maximum_width: 9999
> }
> }
> average_precision_mode: INTEGRATE
> }

```

---

<div class="post-metadata">

**Author:** ![Morganh](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/morganh/32/9748_2.png) [@Morganh](https://forums.developer.nvidia.com/u/Morganh)\
**Post date:** [March 9, 2022, 4:19pm UTC](https://forums.developer.nvidia.com/t/0-map-using-nms-in-custom-detectnet/205443/4 "2022-03-09T16:19:23Z")

</div>

I recall that you already train a tlt model with a MAP 0f 60%.  
Then, you can run “tao evaluate xxx” with this tlt model.  
Try to use NMS with different thresholds.

---

<div class="post-metadata">

**Author:** ![KGerry](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/kgerry/32/173604_2.png) [@KGerry](https://forums.developer.nvidia.com/u/KGerry)\
**Post date:** [March 9, 2022, 4:30pm UTC](https://forums.developer.nvidia.com/t/0-map-using-nms-in-custom-detectnet/205443/5 "2022-03-09T16:30:32Z")

</div>

@Morganh Because most Deepstream apps use NMS clustering in favor of DBSCAN.  
Yes, our model had a MAP of 60%, but that was with DBSCAN, but we want to try if NMS gives better results. We tried many many hours, using all different NMS settings.

The documentation states:  
nms\_iou\_threshold float 0.2  
nms\_confidence\_threshold float 0.

which clearly isn’t going to work for our model. Can you suggest some (default) values?  
Is our evaluator good, or does it need changes to evaluate NMS?

---

<div class="post-metadata">

**Author:** ![Morganh](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/morganh/32/9748_2.png) [@Morganh](https://forums.developer.nvidia.com/u/Morganh)\
**Post date:** [March 9, 2022, 4:51pm UTC](https://forums.developer.nvidia.com/t/0-map-using-nms-in-custom-detectnet/205443/6 "2022-03-09T16:51:35Z")

</div>

> [@KGerry](#):
>
> killed the process after 10 epochs because MAP remained 0.x

You need not run training. Just need to run evaluation against the tlt model.

[https://docs.nvidia.com/tao/tao-toolkit/text/object\_detection/detectnet\_v2.html#evaluating-the-model](https://docs.nvidia.com/tao/tao-toolkit/text/object_detection/detectnet_v2.html#evaluating-the-model)

```auto
tao detectnet_v2 evaluate -e <experiment_spec_with_different_nms_threshold>
                               -m <your_60%_mAP_tlt_model>
                               -k <key>

```

---

<div class="post-metadata">

**Author:** ![KGerry](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/kgerry/32/173604_2.png) [@KGerry](https://forums.developer.nvidia.com/u/KGerry)\
**Post date:** [March 10, 2022, 11:28am UTC](https://forums.developer.nvidia.com/t/0-map-using-nms-in-custom-detectnet/205443/7 "2022-03-10T11:28:18Z")

</div>

So the clustering config (either NMS or DBSCAN) has nothing to do with the **training** , but is used in **evaluating** and every model I trained can do inference with either NMS or BDSCAN?

---

<div class="post-metadata">

**Author:** ![Morganh](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/morganh/32/9748_2.png) [@Morganh](https://forums.developer.nvidia.com/u/Morganh)\
**Post date:** [March 10, 2022, 4:46pm UTC](https://forums.developer.nvidia.com/t/0-map-using-nms-in-custom-detectnet/205443/8 "2022-03-10T16:46:09Z")

</div>

Yes.

---

<div class="post-metadata">

**Author:** ![system](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/system/32/68080_2.png) [@system](https://forums.developer.nvidia.com/u/system)\
**Post date:** [March 24, 2022, 4:46pm UTC](https://forums.developer.nvidia.com/t/0-map-using-nms-in-custom-detectnet/205443/9 "2022-03-24T16:46:23Z")

</div>

Closed on March 24, 2022, 4:46pm UTC

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.