TAO Validation Metrics

[p.vahidinia]

Hi.
How does TAO calculate the mAP and other validation metrics?

[Morganh]

You can refer to an old version of tlt 2.0 user guide.
https://docs.nvidia.com/tao/archive/tlt-20/tlt-user-guide/text/evaluating_model.html#

The classification app computes evaluation loss, Top-k accuracy, precision and recall as metrics. Meanwhile, tlt-evaluate for DetectNet_v2, FasterRCNN, Retinanet, DSSD, YOLOV3, and SSD computes the Average Precision per class and the mean Average Precision metrics as defined in the Pascal VOC challenge. Both sample and integrate modes are supported to calculate average precision. The former was used in VOC challenges before 2010 while the latter was used from 2010 onwards. The SAMPLE mode uses an 11-point method to compute the AP, while the INTEGRATE mode uses a more fine-grained integration method and gets a more accurate number of AP. MaskRCNN reports COCO’s detection evaluation metrics. AP50 in COCO metrics is comparable to mAP in Pascal VOC metrics.

[p.vahidinia]

@Morganh

AP50 in COC

Thank you.