Following detectnet_v2 jupyter notebook for tlt 2.0, after Int8 optimisation (in Step 10 Verify Deployed Model) when running tlt-infer detectnet_v2 the following error occurs. Prior to Int8 optimisation, tlt-infer worked.
Using TensorFlow backend.
2020-08-16 10:28:08.944632: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2020-08-16 10:28:11,462 [INFO] iva.detectnet_v2.scripts.inference: Overlain images will be saved in the output path.
2020-08-16 10:28:11,462 [INFO] iva.detectnet_v2.inferencer.build_inferencer: Constructing inferencer
2020-08-16 10:28:11,952 [INFO] iva.detectnet_v2.inferencer.trt_inferencer: Reading from engine file at: /workspace/tlt-experiments/detectnet_v2/experiment_dir_final/resnet18_detector.trt
[TensorRT] WARNING: Current optimization profile is: 0. Please ensure there are no enqueued operations pending in this context prior to switching profiles
2020-08-16 10:28:12,974 [INFO] iva.detectnet_v2.scripts.inference: Initialized model
2020-08-16 10:28:12,975 [INFO] iva.detectnet_v2.scripts.inference: Commencing inference
0%| | 0/4 [00:00<?, ?it/s]
Traceback (most recent call last):
File “/usr/local/bin/tlt-infer”, line 8, in
sys.exit(main())
File “/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/common/magnet_infer.py”, line 54, in main
File “/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/scripts/inference.py”, line 194, in main
File “/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/scripts/inference.py”, line 150, in inference_wrapper_batch
File “/home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/inferencer/trt_inferencer.py”, line 449, in infer_batch
File “<array_function internals>”, line 6, in copyto
ValueError: could not broadcast input array from shape (9953280) into shape (2875392)
detectnet_v2_inference_kitti_etlt.txt below.
inferencer_config{
Defining input node of the model
defining target class names for the experiment.
Note: This must be mentioned in order of the networks classes.
target_classes: “container”
target_classes: “pm”
target_classes: “lorry”
target_classes: “forklift”
Inference dimensions.
image_width: 1280
image_height: 1296
Must match what the model was trained for.
image_channels: 3
batch_size: 2 #16
gpu_index: 0
model handler config
tensorrt_config{
trt_engine: “/workspace/tlt-experiments/detectnet_v2/experiment_dir_final/resnet18_detector.trt”
}
}
bbox_handler_config{
kitti_dump: true
disable_overlay: false
overlay_linewidth: 2
classwise_bbox_handler_config{
key:“container”
value: {
confidence_model: “aggregate_cov”
output_map: “container”
confidence_threshold: 0.9
bbox_color{
R: 0
G: 255
B: 0
}
clustering_config{
coverage_threshold: 0.00
dbscan_eps: 0.3
dbscan_min_samples: 0.05
minimum_bounding_box_height: 50
}
}
}
classwise_bbox_handler_config{
key:“pm”
value: {
confidence_model: “aggregate_cov”
output_map: “pm”
confidence_threshold: 0.9
bbox_color{
R: 0
G: 0
B: 255
}
clustering_config{
coverage_threshold: 0.00
dbscan_eps: 0.3
dbscan_min_samples: 0.05
minimum_bounding_box_height: 50
}
}
}
classwise_bbox_handler_config{
key:“lorry”
value: {
confidence_model: “aggregate_cov”
output_map: “lorry”
confidence_threshold: 0.9
bbox_color{
R: 255
G: 0
B: 0
}
clustering_config{
coverage_threshold: 0.00
dbscan_eps: 0.3
dbscan_min_samples: 0.05
minimum_bounding_box_height: 50
}
}
}
classwise_bbox_handler_config{
key:“forklift”
value: {
confidence_model: “aggregate_cov”
output_map: “forklift”
confidence_threshold: 0.5
bbox_color{
R: 255
G: 255
B: 0
}
clustering_config{
coverage_threshold: 0.00
dbscan_eps: 0.3
dbscan_min_samples: 0.05
minimum_bounding_box_height: 25
}
}
}
}