Hello,
I’m trying to export a model using DetectNet_v2 model in INT8 mode, but when comparing the results making inference in the same test dataset, I'm getting a lower mAP than previous to the export (20% decrease approx.) (the model being exported has been trained using quantization, and similar results were obtained from the first retraining of the model and the second retraining but with quantization).
The steps I’m following for exporting the model are the following:
- First, I generate calibration tensor file, using the following command
!tao detectnet_v2 calibration_tensorfile -e $SPECS_DIR/trafficcamnet_retrain_resnet18_kitti.txt \
-m 500 \
-o $USER_EXPERIMENT_DIR/experiment_dir_final/calibration.tensor
After this, I’m exporting the model
!tao detectnet_v2 export \
-m $USER_EXPERIMENT_DIR/experiment_dir_retrain/weights/resnet18_detector_pruned_qat.tlt \
-o $USER_EXPERIMENT_DIR/experiment_dir_final/resnet18_detector_qat.etlt \
-k $KEY \
--cal_data_file $USER_EXPERIMENT_DIR/experiment_dir_final/calibration.tensor \
-e $SPECS_DIR/trafficcamnet_retrain_resnet18_kitti.txt \
--data_type int8 \
--batch_size 64 \
--max_batch_size 64\
--engine_file $USER_EXPERIMENT_DIR/experiment_dir_final/resnet18_detector_qat.trt.int8 \
--cal_cache_file $USER_EXPERIMENT_DIR/experiment_dir_final/calibration_qat.bin \
--gen_ds_config \
--verbose
Finally, the inference is made with the generated .trt file resnet18_detector_qat.trt.int8. It is worth mentioning that different values have been tested for generating the calibration file. Is there any parameter I could change to recover the majority of the lost precision?
Here is the inference spec file that I’m using:
inferencer_config{
# Defining input node of the model
# defining target class names for the experiment.
# Note: This must be mentioned in order of the networks classes.
target_classes: "car"
target_classes: "bicycle"
target_classes: "motorcycle"
# Inference dimensions.
image_width: 1920
image_height: 1072
# Must match what the model was trained for.
image_channels: 3
batch_size: 16
gpu_index: 0
# model handler config
tensorrt_config{
parser: ETLT
etlt_model: "/workspace/tao-experiments/retraining/trafficcamnet/experiment_dir_final/resnet18_detector_qat.etlt"
backend_data_type: INT8
trt_engine: "/workspace/tao-experiments/retraining/trafficcamnet/experiment_dir_final/resnet18_detector_qat.trt.int8"
calibrator_config{
calibration_cache: "/workspace/tao-experiments/retraining/trafficcamnet/experiment_dir_final/calibration_qat.bin"
n_batches: 500
}
}
}
bbox_handler_config{
kitti_dump: true
disable_overlay: false
overlay_linewidth: 2
classwise_bbox_handler_config{
key:"car"
value: {
confidence_model: "aggregate_cov"
output_map: "car"
bbox_color{
R: 0
G: 255
B: 0
}
clustering_config{
clustering_algorithm: DBSCAN
coverage_threshold: 0.005
dbscan_eps: 0.3
dbscan_min_samples: 0.05
dbscan_confidence_threshold: 0.9
minimum_bounding_box_height: 4
}
}
}
classwise_bbox_handler_config{
key:"bicycle"
value: {
confidence_model: "aggregate_cov"
output_map: "bicycle"
bbox_color{
R: 0
G: 255
B: 255
}
clustering_config{
clustering_algorithm: DBSCAN
coverage_threshold: 0.005
dbscan_eps: 0.3
dbscan_min_samples: 0.05
dbscan_confidence_threshold: 0.9
minimum_bounding_box_height: 4
}
}
}
classwise_bbox_handler_config{
key:"motorcycle"
value: {
confidence_model: "aggregate_cov"
output_map: "motorcycle"
bbox_color{
R: 255
G: 0
B: 0
clustering_config{
clustering_algorithm: DBSCAN
coverage_threshold: 0.005
dbscan_eps: 0.3
dbscan_min_samples: 0.05
dbscan_confidence_threshold: 0.9
minimum_bounding_box_height: 4
}
}
}
classwise_bbox_handler_config{
key:"default"
value: {
confidence_model: "aggregate_cov"
bbox_color{
R: 255
G: 0
B: 0
}
clustering_config{
clustering_algorithm: DBSCAN
dbscan_confidence_threshold: 0.9
coverage_threshold: 0.005
dbscan_eps: 0.3
dbscan_min_samples: 0.05
minimum_bounding_box_height: 4
}
}
}
}