Not clear documentation about TAO inference output

leo2105 · March 8, 2022, 8:15pm

• Network Type Detectnet_v2

Hi, I did not find good documentation about tao inference. I know that it generates some .txt in kitti format (15 values) and 1 extra value which its supposed to be the confidence right? but there are values from 0.* to 131. 8 for example when I use “aggregate_cov” and I dont understand that part.
It supposed to be values between 0-1 or between 0-100?

In the spec file of inference there is “aggregate_cov” and “mean_cov” for confidence _model. When I tried “mean_cov” there was a warning like:

/usr/local/lib/python3.6/dist-packages/iva/detectnet_v2/postprocessor/utilities.py:352: RuntimeWarning: invalid value encountered in double_scalars

And it generates only empties .txt

So my last question, how can I get the confidence for each bbox?

Morganh · March 9, 2022, 2:06am

It is aggregate_cov instead of aggregate_conv.

And the description is mentioned in DetectNet_v2 - NVIDIA Docs

Algorithm to compute the final confidence of the clustered bboxes. In the aggregate_cov mode, the final confidence of a detection is the sum of the confidences of all the candidate bboxes in a cluster. In mean_cov mode, the final confidence is the mean confidence of all the bboxes in the cluster.

leo2105 · March 9, 2022, 2:12am

Thanks for replying.
Yeah, before writting this post I had read the documentation. I dont want the sum which is from 0.* to 131.* (in my case)
I will rephrase my question. Is there any way to get a confidence between 0-1 or 0-100 for each bbox?

leo2105 · March 9, 2022, 2:15am

I was thinking about normalizing the aggregate_cov maybe, i dont know if it is correct.

Morganh · March 9, 2022, 2:21am

Please use mean_cov mode. It ranges from 0.0 to 1.0.

leo2105 · March 9, 2022, 2:23am

Yeah, but as I said at the beginning it returns this empty txts

Morganh · March 9, 2022, 2:26am

It is not expected. Did you ever try running with official jupyter notebook? TAO Toolkit Quick Start Guide — TAO Toolkit 3.22.05 documentation
There is an example inference spec file.

leo2105 · March 9, 2022, 4:08am

Yep, the same thing, using aggregate_cov it works perfect, but for mean_cov occurs that error

my inference spec file:

inferencer_config{
  # defining target class names for the experiment.
  # Note: This must be mentioned in order of the networks classes.
  target_classes: "car"
  target_classes: "two-wheeler"
  target_classes: "person"
  target_classes: "stop_sign"
  # Inference dimensions.
  image_width: 960
  image_height: 544
  # Must match what the model was trained for.
  image_channels: 3
  batch_size: 24
  gpu_index: 0
  # model handler config
  tlt_config{
    model: "/trafficcamnet_activeLearning/experiments_AL/model_nvactivelearning_1_100.tlt"
  }
}
bbox_handler_config{
  kitti_dump: true
  disable_overlay: false
  overlay_linewidth: 2
  classwise_bbox_handler_config{
    key:"car"
    value: {
      confidence_model: "aggregate_cov"
      output_map: "car"
      bbox_color{
        R: 0
        G: 255
        B: 0
      }
      clustering_config{
        clustering_algorithm: DBSCAN
        coverage_threshold: 0.005
        dbscan_eps: 0.3
        dbscan_min_samples: 0.05
        dbscan_confidence_threshold: 0.9
        minimum_bounding_box_height: 4
      }
    }
  }
  classwise_bbox_handler_config{
    key:"two-wheeler"
    value: {
      confidence_model: "aggregate_cov"
      output_map: "two-wheeler"
      bbox_color{
        R: 0
        G: 255
        B: 255
      }
      clustering_config{
        clustering_algorithm: DBSCAN
        coverage_threshold: 0.005
        dbscan_eps: 0.3
        dbscan_min_samples: 0.05
        dbscan_confidence_threshold: 0.9
        minimum_bounding_box_height: 4
      }
    }
  }
  classwise_bbox_handler_config{
    key:"person"
    value: {
      confidence_model: "aggregate_cov"
      output_map: "person"
      bbox_color{
        R: 255
        G: 0
        B: 0
      }
      clustering_config{
        clustering_algorithm: DBSCAN
        coverage_threshold: 0.005
        dbscan_eps: 0.3
        dbscan_min_samples: 0.05
        dbscan_confidence_threshold: 0.9
        minimum_bounding_box_height: 4
      }
    }
  }
  classwise_bbox_handler_config{
    key:"stop_sign"
    value: {
      confidence_model: "aggregate_cov"
      output_map: "stop_sign"
      bbox_color{
        R: 0
        G: 0
        B: 255
      }
      clustering_config{
        clustering_algorithm: DBSCAN
        coverage_threshold: 0.005
        dbscan_eps: 0.3
        dbscan_min_samples: 0.05
        dbscan_confidence_threshold: 0.9
        minimum_bounding_box_height: 4
      }
    }
  }
    classwise_bbox_handler_config{
    key:"default"
    value: {
      confidence_model: "aggregate_cov"
      bbox_color{
        R: 255
        G: 255
        B: 0
      }
      clustering_config{
        clustering_algorithm: DBSCAN
        dbscan_confidence_threshold: 0.9
        coverage_threshold: 0.005
        dbscan_eps: 0.3
        dbscan_min_samples: 0.05
        minimum_bounding_box_height: 4
      }
    }
  }
}

leo2105 · March 9, 2022, 4:11am

It looks like a problem of division by zero that you didnt solve…

Morganh · March 9, 2022, 4:26am

Can you share the spec when you set to “mean_cov” mode?

leo2105 · March 9, 2022, 4:30am

Yep, I just changed that word.

inferencer_config{
  # defining target class names for the experiment.
  # Note: This must be mentioned in order of the networks classes.
  target_classes: "car"
  target_classes: "two-wheeler"
  target_classes: "person"
  target_classes: "stop_sign"
  # Inference dimensions.
  image_width: 960
  image_height: 544
  # Must match what the model was trained for.
  image_channels: 3
  batch_size: 24
  gpu_index: 0
  # model handler config
  tlt_config{
    model: "/trafficcamnet_activeLearning/experiments_AL/model_nvactivelearning_1_100.tlt"
  }
}
bbox_handler_config{
  kitti_dump: true
  disable_overlay: false
  overlay_linewidth: 2
  classwise_bbox_handler_config{
    key:"car"
    value: {
      confidence_model: "mean_cov"
      output_map: "car"
      bbox_color{
        R: 0
        G: 255
        B: 0
      }
      clustering_config{
        clustering_algorithm: DBSCAN
        coverage_threshold: 0.005
        dbscan_eps: 0.3
        dbscan_min_samples: 0.05
        dbscan_confidence_threshold: 0.9
        minimum_bounding_box_height: 4
      }
    }
  }
  classwise_bbox_handler_config{
    key:"two-wheeler"
    value: {
      confidence_model: "mean_cov"
      output_map: "two-wheeler"
      bbox_color{
        R: 0
        G: 255
        B: 255
      }
      clustering_config{
        clustering_algorithm: DBSCAN
        coverage_threshold: 0.005
        dbscan_eps: 0.3
        dbscan_min_samples: 0.05
        dbscan_confidence_threshold: 0.9
        minimum_bounding_box_height: 4
      }
    }
  }
  classwise_bbox_handler_config{
    key:"person"
    value: {
      confidence_model: "mean_cov"
      output_map: "person"
      bbox_color{
        R: 255
        G: 0
        B: 0
      }
      clustering_config{
        clustering_algorithm: DBSCAN
        coverage_threshold: 0.005
        dbscan_eps: 0.3
        dbscan_min_samples: 0.05
        dbscan_confidence_threshold: 0.9
        minimum_bounding_box_height: 4
      }
    }
  }
  classwise_bbox_handler_config{
    key:"stop_sign"
    value: {
      confidence_model: "mean_cov"
      output_map: "stop_sign"
      bbox_color{
        R: 0
        G: 0
        B: 255
      }
      clustering_config{
        clustering_algorithm: DBSCAN
        coverage_threshold: 0.005
        dbscan_eps: 0.3
        dbscan_min_samples: 0.05
        dbscan_confidence_threshold: 0.9
        minimum_bounding_box_height: 4
      }
    }
  }
    classwise_bbox_handler_config{
    key:"default"
    value: {
      confidence_model: "mean_cov"
      bbox_color{
        R: 255
        G: 255
        B: 0
      }
      clustering_config{
        clustering_algorithm: DBSCAN
        dbscan_confidence_threshold: 0.9
        coverage_threshold: 0.005
        dbscan_eps: 0.3
        dbscan_min_samples: 0.05
        minimum_bounding_box_height: 4
      }
    }
  }
}

Morganh · March 9, 2022, 4:45am

Please set a lower dbscan_confidence_threshold for all classes and retry.

leo2105 · March 9, 2022, 1:22pm

Ok… it generates the bboxes in kitti format and confidence ranged 0 - 1.
but the warning is still there, I dont know if in the inference it will loss some bboxes because of the warning.

Thanks

system · March 29, 2022, 2:27am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Tao inference detectnet v2 clustering - large differences between NMS and DBSCAN TAO Toolkit	2	534	November 9, 2023
Label files generated by tlt-infer TAO Toolkit	10	851	October 12, 2021
Error detectnet_V2 train with TAO : dbscan_min_samples: 0.05' TAO Toolkit tao	4	403	November 7, 2023
Error while using Tlt-infer TAO Toolkit	6	705	October 12, 2021
Problem of tao detectnet_v2 evaluate 0% TAO Toolkit python	21	424	July 7, 2023
Invalid argument: Invalid JPEG data or crop window, data size 786432 TAO Toolkit	9	1389	March 20, 2023
Setting a Confidence Algorithm DeepStream SDK	9	1565	October 12, 2021
Relationship between training dataset size and inference data size TAO Toolkit	12	718	February 22, 2022
Exec inside tao container TAO Toolkit	16	588	January 26, 2022
TAO deploy docker eval accuracy lower vs TAO tf1 docker TAO Toolkit	13	39	August 27, 2024

Not clear documentation about TAO inference output

Related topics