Great input! this made the process run! but we are not done yet, because the process ended in error :
executing the tao-converter with the -t int8 argument ( as shown in the yolov4_tiny) gives the follow error output:
./tao-converter -k ... -p Input,1x3x480x640,8x3x480x640,16x3x480x640 -e export/trt.engine -b 8 **-t int8** export/yolov4_cspdarknet_tiny_epoch_080.etlt
[INFO] [MemUsageChange] Init CUDA: CPU +353, GPU +0, now: CPU 371, GPU 6476 (MiB)
[INFO] ----------------------------------------------------------------
[INFO] Input filename: /tmp/fileCnwAkI
[INFO] ONNX IR version: 0.0.7
[INFO] Opset version: 13
[INFO] Producer name:
[INFO] Producer version:
[INFO] Domain:
[INFO] Model version: 0
[INFO] Doc string:
[INFO] ----------------------------------------------------------------
[WARNING] onnx2trt_utils.cpp:364: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[WARNING] onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[INFO] No importer registered for op: BatchedNMSDynamic_TRT. Attempting to import as plugin.
[INFO] Searching for plugin: BatchedNMSDynamic_TRT, plugin_version: 1, plugin_namespace:
[INFO] Successfully created plugin: BatchedNMSDynamic_TRT
[INFO] Detected input dimensions from the model: (-1, 3, 480, 640)
[INFO] Model has dynamic shape. Setting up optimization profiles.
[INFO] Using optimization profile min shape: (1, 3, 480, 640) for input: Input
[INFO] Using optimization profile opt shape: (8, 3, 480, 640) for input: Input
[INFO] Using optimization profile max shape: (16, 3, 480, 640) for input: Input
**[WARNING] DLA requests all profiles have same min, max, and opt value. All dla layers are falling back to GPU**
[INFO] [MemUsageSnapshot] Builder begin: CPU 395 MiB, GPU 6546 MiB
[INFO] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +226, GPU +188, now: CPU 622, GPU 6734 (MiB)
[INFO] [MemUsageChange] Init cuDNN: CPU +307, GPU +313, now: CPU 929, GPU 7047 (MiB)
**[WARNING] Calibration Profile is not defined. Running calibration with Profile 0**
[INFO] Detected 1 inputs and 4 output network tensors.
[INFO] Total Host Persistent Memory: 6896
[INFO] Total Device Persistent Memory: 0
[INFO] Total Scratch Memory: 4134912
[INFO] [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 0 MiB, GPU 150 MiB
[INFO] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +1, now: CPU 1404, GPU 7365 (MiB)
[INFO] [MemUsageChange] Init cuDNN: CPU +0, GPU +0, now: CPU 1404, GPU 7365 (MiB)
[INFO] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 1404, GPU 7365 (MiB)
[INFO] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 1403, GPU 7365 (MiB)
[INFO] [MemUsageSnapshot] ExecutionContext creation begin: CPU 1403 MiB, GPU 7365 MiB
[INFO] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 1403, GPU 7365 (MiB)
[INFO] [MemUsageChange] Init cuDNN: CPU +1, GPU +0, now: CPU 1404, GPU 7365 (MiB)
[INFO] [MemUsageSnapshot] ExecutionContext creation end: CPU 1404 MiB, GPU 7478 MiB
[INFO] Starting Calibration.
[INFO] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 1403, GPU 7469 (MiB)
[INFO] Post Processing Calibration data in 3.776e-06 seconds.
**[ERROR] 1: Unexpected exception _Map_base::at**
**[ERROR] Unable to create engine**
Segmentation fault (core dumped)
Although the export step was executed using the --data-type int8 using to following code scope :
!tao yolo_v4_tiny export -m $USER_EXPERIMENT_DIR/experiment_dir_retrain/weights/yolov4_cspdarknet_tiny_epoch_$EPOCH.tlt \
-o $USER_EXPERIMENT_DIR/export/yolov4_cspdarknet_tiny_epoch_$EPOCH.etlt \
-e $SPECS_DIR/yolo_v4_tiny_retrain_chimera.txt \
-k $KEY \
--data_type int8 \
--cal_cache_file $USER_EXPERIMENT_DIR/export/cal.bin
Any thoughts what can cause the converter step not to finish?
Many thanks!