Performance measurment

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

Hello,

I’m building my own performance metrics scripts to assess the models. The idea is to include performance metrics (e.g. FPS/latency/etc) as well as the precision metrics from the model. As a sanity check, I’m trying to replicate the results (mAP) obtained from tlt-evaluate for a DetectNet_v2 model with ResNet-18 backbone.

However, I’ve realized there are different configuration parameters on the evaluation configuration file used in TLT and the DeepStream deployed engine. More specifically, I’m trying to understand the relation between the evaluation_config used in the training/evaluation spec. file for TLT, and the pgie.txt configuration file for DeepStream. Is there any way I can guarantee the same post-processing configuration for both cases?

For example, how can I reproduce the following post-processing configuration:

clustering_config {
        coverage_threshold: 0.005
        dbscan_eps: 0.15
        dbscan_min_samples: 0.05
        minimum_bounding_box_height: 20

Using the DeepStream configuration parameters (minBoxes, cluster-mode, etc)

Thanks,

3

In deepstream, for dbscan, you can search some parameters or function.
For example,

clusterAndFillDetectionOutputDBSCAN(output);
define CONFIG_GROUP_INFER_CLASS_ATTRS_EPS “eps”
define CONFIG_GROUP_INFER_CLASS_ATTRS_MIN_BOXES “minBoxes”
define CONFIG_GROUP_INFER_CLASS_ATTRS_DBSCAN_MIN_SCORE “dbscan-min-score”