Detectnet_v2, tlt inference error

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

Hi,
I am facing this issue which has a mismatch shape.
I want to understand if it is due to the fp32 or int8 trt, and etlt file or is it due to image shape?

File “./common/magnet_infer.py”, line 56, in main
File “./detectnet_v2/scripts/inference.py”, line 194, in main
File “./detectnet_v2/scripts/inference.py”, line 150, in inference_wrapper_batch
File “./detectnet_v2/inferencer/trt_inferencer.py”, line 443, in infer_batch
ValueError: could not broadcast input array from shape (14745600) into shape (33841152)

2

Can you share the command, spec file and full log?

3

tlt-infer detectnet_v2 -e /workspace/train_config/inference_config.txt -i /workspace/train_config/data/demo_data2/testing/image_2 -o /workspace/train_config/data/demo_data2/testing/output -k tlt_encode

Using TensorFlow backend.
2021-09-01 08:13:07,120 [INFO] iva.detectnet_v2.scripts.inference: Overlain images will be saved in the output path.
2021-09-01 08:13:07,120 [INFO] iva.detectnet_v2.inferencer.build_inferencer: Constructing inferencer
2021-09-01 08:13:07,285 [INFO] iva.detectnet_v2.inferencer.trt_inferencer: Reading from engine file at: /results/model/unpruned/export/LPD_Training_detectnet_v2_resnet18_detector_gpu0_fp32.trt
[TensorRT] WARNING: Current optimization profile is: 0. Please ensure there are no enqueued operations pending in this context prior to switching profiles
2021-09-01 08:13:08,313 [INFO] iva.detectnet_v2.scripts.inference: Initialized model
2021-09-01 08:13:08,314 [INFO] iva.detectnet_v2.scripts.inference: Commencing inference
0%| | 0/14 [00:00<?, ?it/s]
Traceback (most recent call last):
File “/usr/local/bin/tlt-infer”, line 8, in
sys.exit(main())
File “./common/magnet_infer.py”, line 56, in main
File “./detectnet_v2/scripts/inference.py”, line 194, in main
File “./detectnet_v2/scripts/inference.py”, line 150, in inference_wrapper_batch
File “./detectnet_v2/inferencer/trt_inferencer.py”, line 443, in infer_batch
ValueError: could not broadcast input array from shape (14745600) into shape (33841152)

4

random_seed: 42
dataset_config {
data_sources {
tfrecords_path: “/workspace/train_config/data/demo_data/tfrecords/kitti_trainval/*”
image_directory_path: “/workspace/train_config/data/demo_data/training/”
}
image_extension: “jpg”
target_class_mapping {
key: “lpd”
value: “lpd”
}
validation_fold: 0
}
augmentation_config {
preprocessing {
output_image_width: 640
output_image_height: 480
min_bbox_width: 1.0
min_bbox_height: 1.0
output_image_channel: 3
}
spatial_augmentation {
hflip_probability: 0.5
zoom_min: 1.0
zoom_max: 1.0
translate_max_x: 8.0
translate_max_y: 8.0
}
color_augmentation {
hue_rotation_max: 25.0
saturation_shift_max: 0.20000000298
contrast_scale_max: 0.10000000149
contrast_center: 0.5
}
}
postprocessing_config {
target_class_config {
key: “lpd”
value {
clustering_config {
coverage_threshold: 0.00499999988824
dbscan_eps: 0.20000000298
dbscan_min_samples: 0.0500000007451
minimum_bounding_box_height: 4
}
}
}
}
model_config {
pretrained_model_file: “/workspace/train_config/pretrained_model/tlt_pretrained_detectnet_v2_vresnet18/resnet18.hdf5”
num_layers: 18
use_batch_norm: true
objective_set {
bbox {
scale: 35.0
offset: 0.5
}
cov {
}
}
training_precision {
backend_floatx: FLOAT32
}
arch: “resnet”
}
evaluation_config {
validation_period_during_training: 10
first_validation_epoch: 5
minimum_detection_ground_truth_overlap {
key: “lpd”
value: 0.699999988079
}
evaluation_box_config {
key: “lpd”
value {
minimum_height: 10
maximum_height: 9999
minimum_width: 10
maximum_width: 9999
}
}
average_precision_mode: INTEGRATE
}
cost_function_config {
target_classes {
name: “lpd”
class_weight: 1.0
coverage_foreground_weight: 0.0500000007451
objectives {
name: “cov”
initial_weight: 1.0
weight_target: 1.0
}
objectives {
name: “bbox”
initial_weight: 10.0
weight_target: 10.0
}
}
enable_autoweighting: true
max_objective_weight: 0.999899983406
min_objective_weight: 9.99999974738e-05
}
training_config {
batch_size_per_gpu: 16
num_epochs: 120
#enable_qat: False
learning_rate {
soft_start_annealing_schedule {
min_learning_rate: 5e-06
max_learning_rate: 5e-04
soft_start: 0.10000000149
annealing: 0.699999988079
}
}
regularizer {
type: L1
weight: 3.00000002618e-09
}
optimizer {
adam {
epsilon: 9.99999993923e-09
beta1: 0.899999976158
beta2: 0.999000012875
}
}
cost_scaling {
initial_exponent: 20.0
increment: 0.005
decrement: 1.0
}
checkpoint_interval: 10
}
bbox_rasterizer_config {
target_class_config {
key: “lpd”
value {
cov_center_x: 0.5
cov_center_y: 0.5
cov_radius_x: 0.40000000596
cov_radius_y: 0.40000000596
bbox_min_radius: 1.0
}
}
deadzone_radius: 0.400000154972
}

5

kitti_config {
root_directory_path: “/workspace/train_config/data/demo_data/training/”
image_dir_name: “image_2”
label_dir_name: “label_2”
image_extension: “.jpg”
partition_mode: “random”
num_partitions: 2
val_split: 14
num_shards: 4
}
image_directory_path: “/workspace/train_config/data/demo_data/training/”

6

Did you attach correct /workspace/train_config/inference_config.txt ? Seems that it is only a training spec file.

7

Here is the inference Specification file.

inferencer_config{
target_classes: “lpd”

Inference dimensions.

image_width: 640
image_height: 480

Must match what the model was trained for.

image_channels: 3
batch_size: 16
gpu_index: 0

model handler config

tensorrt_config{
parser: ETLT
etlt_model: “/results/model/unpruned/export/LPD_Training_detectnet_v2_resnet18.etlt”
backend_data_type: FP32
save_engine: true
trt_engine: “/results/model/unpruned/export/LPD_Training_detectnet_v2_resnet18_detector_gpu0_fp32.trt”
#calibrator_config{
# calibration_cache: “/results/model/unpruned/export/LPD_Training_detectnet_v2_resnet18_calibration.bin”
# n_batches: 1
# batch_size: 8
#}
}
}
bbox_handler_config{
kitti_dump: true
disable_overlay: false
overlay_linewidth: 2
classwise_bbox_handler_config{
key:“lpd”
value: {
confidence_model: “aggregate_cov”
output_map: “lpd”
confidence_threshold: 0.6
bbox_color{
R: 0
G: 255
B: 0
}
clustering_config{
coverage_threshold: 0.00
dbscan_eps: 0.9
dbscan_min_samples: 0.1
minimum_bounding_box_height: 10
}
}
}

}

8

@Morganh I fixed the issue.
Thank you!

9

Can you share what is happened?

10

Hi,
The Training and testing shape of images did not match.