Not clear documentation about TAO inference output

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

• Network Type Detectnet_v2

Hi, I did not find good documentation about tao inference. I know that it generates some .txt in kitti format (15 values) and 1 extra value which its supposed to be the confidence right? but there are values from 0.* to 131. 8 for example when I use “aggregate_cov” and I dont understand that part.
It supposed to be values between 0-1 or between 0-100?

In the spec file of inference there is “aggregate_cov” and “mean_cov” for confidence _model. When I tried “mean_cov” there was a warning like:

/usr/local/lib/python3.6/dist-packages/iva/detectnet_v2/postprocessor/utilities.py:352: RuntimeWarning: invalid value encountered in double_scalars

And it generates only empties .txt

So my last question, how can I get the confidence for each bbox?

3

It is aggregate_cov instead of aggregate_conv.

And the description is mentioned in DetectNet_v2 - NVIDIA Docs

Algorithm to compute the final confidence of the clustered bboxes. In the aggregate_cov mode, the final confidence of a detection is the sum of the confidences of all the candidate bboxes in a cluster. In mean_cov mode, the final confidence is the mean confidence of all the bboxes in the cluster.

4

Thanks for replying.
Yeah, before writting this post I had read the documentation. I dont want the sum which is from 0.* to 131.* (in my case)
I will rephrase my question. Is there any way to get a confidence between 0-1 or 0-100 for each bbox?

5

I was thinking about normalizing the aggregate_cov maybe, i dont know if it is correct.

6

Please use mean_cov mode. It ranges from 0.0 to 1.0.

7

Yeah, but as I said at the beginning it returns this empty txts

8

It is not expected. Did you ever try running with official jupyter notebook? TAO Toolkit Quick Start Guide — TAO Toolkit 3.22.05 documentation
There is an example inference spec file.

9

Yep, the same thing, using aggregate_cov it works perfect, but for mean_cov occurs that error

Screenshot from 2022-03-09 01-07-44
Screenshot from 2022-03-09 01-07-441557×844 21.7 KB

my inference spec file:

inferencer_config{
  # defining target class names for the experiment.
  # Note: This must be mentioned in order of the networks classes.
  target_classes: "car"
  target_classes: "two-wheeler"
  target_classes: "person"
  target_classes: "stop_sign"
  # Inference dimensions.
  image_width: 960
  image_height: 544
  # Must match what the model was trained for.
  image_channels: 3
  batch_size: 24
  gpu_index: 0
  # model handler config
  tlt_config{
    model: "/trafficcamnet_activeLearning/experiments_AL/model_nvactivelearning_1_100.tlt"
  }
}
bbox_handler_config{
  kitti_dump: true
  disable_overlay: false
  overlay_linewidth: 2
  classwise_bbox_handler_config{
    key:"car"
    value: {
      confidence_model: "aggregate_cov"
      output_map: "car"
      bbox_color{
        R: 0
        G: 255
        B: 0
      }
      clustering_config{
        clustering_algorithm: DBSCAN
        coverage_threshold: 0.005
        dbscan_eps: 0.3
        dbscan_min_samples: 0.05
        dbscan_confidence_threshold: 0.9
        minimum_bounding_box_height: 4
      }
    }
  }
  classwise_bbox_handler_config{
    key:"two-wheeler"
    value: {
      confidence_model: "aggregate_cov"
      output_map: "two-wheeler"
      bbox_color{
        R: 0
        G: 255
        B: 255
      }
      clustering_config{
        clustering_algorithm: DBSCAN
        coverage_threshold: 0.005
        dbscan_eps: 0.3
        dbscan_min_samples: 0.05
        dbscan_confidence_threshold: 0.9
        minimum_bounding_box_height: 4
      }
    }
  }
  classwise_bbox_handler_config{
    key:"person"
    value: {
      confidence_model: "aggregate_cov"
      output_map: "person"
      bbox_color{
        R: 255
        G: 0
        B: 0
      }
      clustering_config{
        clustering_algorithm: DBSCAN
        coverage_threshold: 0.005
        dbscan_eps: 0.3
        dbscan_min_samples: 0.05
        dbscan_confidence_threshold: 0.9
        minimum_bounding_box_height: 4
      }
    }
  }
  classwise_bbox_handler_config{
    key:"stop_sign"
    value: {
      confidence_model: "aggregate_cov"
      output_map: "stop_sign"
      bbox_color{
        R: 0
        G: 0
        B: 255
      }
      clustering_config{
        clustering_algorithm: DBSCAN
        coverage_threshold: 0.005
        dbscan_eps: 0.3
        dbscan_min_samples: 0.05
        dbscan_confidence_threshold: 0.9
        minimum_bounding_box_height: 4
      }
    }
  }
    classwise_bbox_handler_config{
    key:"default"
    value: {
      confidence_model: "aggregate_cov"
      bbox_color{
        R: 255
        G: 255
        B: 0
      }
      clustering_config{
        clustering_algorithm: DBSCAN
        dbscan_confidence_threshold: 0.9
        coverage_threshold: 0.005
        dbscan_eps: 0.3
        dbscan_min_samples: 0.05
        minimum_bounding_box_height: 4
      }
    }
  }
}
10

It looks like a problem of division by zero that you didnt solve…

11

Can you share the spec when you set to “mean_cov” mode?

12

Yep, I just changed that word.

inferencer_config{
  # defining target class names for the experiment.
  # Note: This must be mentioned in order of the networks classes.
  target_classes: "car"
  target_classes: "two-wheeler"
  target_classes: "person"
  target_classes: "stop_sign"
  # Inference dimensions.
  image_width: 960
  image_height: 544
  # Must match what the model was trained for.
  image_channels: 3
  batch_size: 24
  gpu_index: 0
  # model handler config
  tlt_config{
    model: "/trafficcamnet_activeLearning/experiments_AL/model_nvactivelearning_1_100.tlt"
  }
}
bbox_handler_config{
  kitti_dump: true
  disable_overlay: false
  overlay_linewidth: 2
  classwise_bbox_handler_config{
    key:"car"
    value: {
      confidence_model: "mean_cov"
      output_map: "car"
      bbox_color{
        R: 0
        G: 255
        B: 0
      }
      clustering_config{
        clustering_algorithm: DBSCAN
        coverage_threshold: 0.005
        dbscan_eps: 0.3
        dbscan_min_samples: 0.05
        dbscan_confidence_threshold: 0.9
        minimum_bounding_box_height: 4
      }
    }
  }
  classwise_bbox_handler_config{
    key:"two-wheeler"
    value: {
      confidence_model: "mean_cov"
      output_map: "two-wheeler"
      bbox_color{
        R: 0
        G: 255
        B: 255
      }
      clustering_config{
        clustering_algorithm: DBSCAN
        coverage_threshold: 0.005
        dbscan_eps: 0.3
        dbscan_min_samples: 0.05
        dbscan_confidence_threshold: 0.9
        minimum_bounding_box_height: 4
      }
    }
  }
  classwise_bbox_handler_config{
    key:"person"
    value: {
      confidence_model: "mean_cov"
      output_map: "person"
      bbox_color{
        R: 255
        G: 0
        B: 0
      }
      clustering_config{
        clustering_algorithm: DBSCAN
        coverage_threshold: 0.005
        dbscan_eps: 0.3
        dbscan_min_samples: 0.05
        dbscan_confidence_threshold: 0.9
        minimum_bounding_box_height: 4
      }
    }
  }
  classwise_bbox_handler_config{
    key:"stop_sign"
    value: {
      confidence_model: "mean_cov"
      output_map: "stop_sign"
      bbox_color{
        R: 0
        G: 0
        B: 255
      }
      clustering_config{
        clustering_algorithm: DBSCAN
        coverage_threshold: 0.005
        dbscan_eps: 0.3
        dbscan_min_samples: 0.05
        dbscan_confidence_threshold: 0.9
        minimum_bounding_box_height: 4
      }
    }
  }
    classwise_bbox_handler_config{
    key:"default"
    value: {
      confidence_model: "mean_cov"
      bbox_color{
        R: 255
        G: 255
        B: 0
      }
      clustering_config{
        clustering_algorithm: DBSCAN
        dbscan_confidence_threshold: 0.9
        coverage_threshold: 0.005
        dbscan_eps: 0.3
        dbscan_min_samples: 0.05
        minimum_bounding_box_height: 4
      }
    }
  }
}
13

Please set a lower dbscan_confidence_threshold for all classes and retry.

14

Ok… it generates the bboxes in kitti format and confidence ranged 0 - 1.
but the warning is still there, I dont know if in the inference it will loss some bboxes because of the warning.

Thanks