Not clear documentation about TAO inference output

• Network Type Detectnet_v2

Hi, I did not find good documentation about tao inference. I know that it generates some .txt in kitti format (15 values) and 1 extra value which its supposed to be the confidence right? but there are values from 0.* to 131. 8 for example when I use “aggregate_cov” and I dont understand that part.
It supposed to be values between 0-1 or between 0-100?

In the spec file of inference there is “aggregate_cov” and “mean_cov” for confidence _model. When I tried “mean_cov” there was a warning like:

/usr/local/lib/python3.6/dist-packages/iva/detectnet_v2/postprocessor/utilities.py:352: RuntimeWarning: invalid value encountered in double_scalars

And it generates only empties .txt

So my last question, how can I get the confidence for each bbox?

It is aggregate_cov instead of aggregate_conv.

And the description is mentioned in DetectNet_v2 — TAO Toolkit 3.21.11 documentation

Algorithm to compute the final confidence of the clustered bboxes. In the aggregate_cov mode, the final confidence of a detection is the sum of the confidences of all the candidate bboxes in a cluster. In mean_cov mode, the final confidence is the mean confidence of all the bboxes in the cluster.

Thanks for replying.
Yeah, before writting this post I had read the documentation. I dont want the sum which is from 0.* to 131.* (in my case)
I will rephrase my question. Is there any way to get a confidence between 0-1 or 0-100 for each bbox?

I was thinking about normalizing the aggregate_cov maybe, i dont know if it is correct.

Please use mean_cov mode. It ranges from 0.0 to 1.0.

Yeah, but as I said at the beginning it returns this empty txts

It is not expected. Did you ever try running with official jupyter notebook? TAO Toolkit Quick Start Guide — TAO Toolkit 3.21.11 documentation
There is an example inference spec file.

Yep, the same thing, using aggregate_cov it works perfect, but for mean_cov occurs that error

my inference spec file:

inferencer_config{
  # defining target class names for the experiment.
  # Note: This must be mentioned in order of the networks classes.
  target_classes: "car"
  target_classes: "two-wheeler"
  target_classes: "person"
  target_classes: "stop_sign"
  # Inference dimensions.
  image_width: 960
  image_height: 544
  # Must match what the model was trained for.
  image_channels: 3
  batch_size: 24
  gpu_index: 0
  # model handler config
  tlt_config{
    model: "/trafficcamnet_activeLearning/experiments_AL/model_nvactivelearning_1_100.tlt"
  }
}
bbox_handler_config{
  kitti_dump: true
  disable_overlay: false
  overlay_linewidth: 2
  classwise_bbox_handler_config{
    key:"car"
    value: {
      confidence_model: "aggregate_cov"
      output_map: "car"
      bbox_color{
        R: 0
        G: 255
        B: 0
      }
      clustering_config{
        clustering_algorithm: DBSCAN
        coverage_threshold: 0.005
        dbscan_eps: 0.3
        dbscan_min_samples: 0.05
        dbscan_confidence_threshold: 0.9
        minimum_bounding_box_height: 4
      }
    }
  }
  classwise_bbox_handler_config{
    key:"two-wheeler"
    value: {
      confidence_model: "aggregate_cov"
      output_map: "two-wheeler"
      bbox_color{
        R: 0
        G: 255
        B: 255
      }
      clustering_config{
        clustering_algorithm: DBSCAN
        coverage_threshold: 0.005
        dbscan_eps: 0.3
        dbscan_min_samples: 0.05
        dbscan_confidence_threshold: 0.9
        minimum_bounding_box_height: 4
      }
    }
  }
  classwise_bbox_handler_config{
    key:"person"
    value: {
      confidence_model: "aggregate_cov"
      output_map: "person"
      bbox_color{
        R: 255
        G: 0
        B: 0
      }
      clustering_config{
        clustering_algorithm: DBSCAN
        coverage_threshold: 0.005
        dbscan_eps: 0.3
        dbscan_min_samples: 0.05
        dbscan_confidence_threshold: 0.9
        minimum_bounding_box_height: 4
      }
    }
  }
  classwise_bbox_handler_config{
    key:"stop_sign"
    value: {
      confidence_model: "aggregate_cov"
      output_map: "stop_sign"
      bbox_color{
        R: 0
        G: 0
        B: 255
      }
      clustering_config{
        clustering_algorithm: DBSCAN
        coverage_threshold: 0.005
        dbscan_eps: 0.3
        dbscan_min_samples: 0.05
        dbscan_confidence_threshold: 0.9
        minimum_bounding_box_height: 4
      }
    }
  }
    classwise_bbox_handler_config{
    key:"default"
    value: {
      confidence_model: "aggregate_cov"
      bbox_color{
        R: 255
        G: 255
        B: 0
      }
      clustering_config{
        clustering_algorithm: DBSCAN
        dbscan_confidence_threshold: 0.9
        coverage_threshold: 0.005
        dbscan_eps: 0.3
        dbscan_min_samples: 0.05
        minimum_bounding_box_height: 4
      }
    }
  }
}

It looks like a problem of division by zero that you didnt solve…

Can you share the spec when you set to “mean_cov” mode?

Yep, I just changed that word.

inferencer_config{
  # defining target class names for the experiment.
  # Note: This must be mentioned in order of the networks classes.
  target_classes: "car"
  target_classes: "two-wheeler"
  target_classes: "person"
  target_classes: "stop_sign"
  # Inference dimensions.
  image_width: 960
  image_height: 544
  # Must match what the model was trained for.
  image_channels: 3
  batch_size: 24
  gpu_index: 0
  # model handler config
  tlt_config{
    model: "/trafficcamnet_activeLearning/experiments_AL/model_nvactivelearning_1_100.tlt"
  }
}
bbox_handler_config{
  kitti_dump: true
  disable_overlay: false
  overlay_linewidth: 2
  classwise_bbox_handler_config{
    key:"car"
    value: {
      confidence_model: "mean_cov"
      output_map: "car"
      bbox_color{
        R: 0
        G: 255
        B: 0
      }
      clustering_config{
        clustering_algorithm: DBSCAN
        coverage_threshold: 0.005
        dbscan_eps: 0.3
        dbscan_min_samples: 0.05
        dbscan_confidence_threshold: 0.9
        minimum_bounding_box_height: 4
      }
    }
  }
  classwise_bbox_handler_config{
    key:"two-wheeler"
    value: {
      confidence_model: "mean_cov"
      output_map: "two-wheeler"
      bbox_color{
        R: 0
        G: 255
        B: 255
      }
      clustering_config{
        clustering_algorithm: DBSCAN
        coverage_threshold: 0.005
        dbscan_eps: 0.3
        dbscan_min_samples: 0.05
        dbscan_confidence_threshold: 0.9
        minimum_bounding_box_height: 4
      }
    }
  }
  classwise_bbox_handler_config{
    key:"person"
    value: {
      confidence_model: "mean_cov"
      output_map: "person"
      bbox_color{
        R: 255
        G: 0
        B: 0
      }
      clustering_config{
        clustering_algorithm: DBSCAN
        coverage_threshold: 0.005
        dbscan_eps: 0.3
        dbscan_min_samples: 0.05
        dbscan_confidence_threshold: 0.9
        minimum_bounding_box_height: 4
      }
    }
  }
  classwise_bbox_handler_config{
    key:"stop_sign"
    value: {
      confidence_model: "mean_cov"
      output_map: "stop_sign"
      bbox_color{
        R: 0
        G: 0
        B: 255
      }
      clustering_config{
        clustering_algorithm: DBSCAN
        coverage_threshold: 0.005
        dbscan_eps: 0.3
        dbscan_min_samples: 0.05
        dbscan_confidence_threshold: 0.9
        minimum_bounding_box_height: 4
      }
    }
  }
    classwise_bbox_handler_config{
    key:"default"
    value: {
      confidence_model: "mean_cov"
      bbox_color{
        R: 255
        G: 255
        B: 0
      }
      clustering_config{
        clustering_algorithm: DBSCAN
        dbscan_confidence_threshold: 0.9
        coverage_threshold: 0.005
        dbscan_eps: 0.3
        dbscan_min_samples: 0.05
        minimum_bounding_box_height: 4
      }
    }
  }
}

Please set a lower dbscan_confidence_threshold for all classes and retry.

Ok… it generates the bboxes in kitti format and confidence ranged 0 - 1.
but the warning is still there, I dont know if in the inference it will loss some bboxes because of the warning.

Thanks

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.