Running on a computer with a GeForce GTX 1080ti, Intel i9 CPU 64GB RAM, Ubuntu 18.04
Attaching the notebook with all messages here: unet_isbi.ipynb (633.4 KB)
After training, pruning and retraining,
In the notebook, at the instruction
# Convert to TensorRT engine(FP16).
!tao converter -k $KEY \
-e $USER_EXPERIMENT_DIR/export/trt.fp16.isbi.retrained.engine \
-t fp16 \
-p input_1,1x1x320x320,4x1x320x320,16x1x320x320 \
$USER_EXPERIMENT_DIR/isbi_experiment_retrain/weights/model_isbi_retrained.etlt
Produces the following messages:
2022-01-18 23:26:46,401 [INFO] root: Registry: [‘nvcr.io’]
2022-01-18 23:26:46,517 [INFO] tlt.components.instance_handler.local_instance: Running command in container: nvcr.io/nvidia/tao/tao-toolkit-tf:v3.21.11-tf1.15.5-py3
[INFO] [MemUsageChange] Init CUDA: CPU +160, GPU +0, now: CPU 166, GPU 1195 (MiB)
[INFO] ----------------------------------------------------------------
[INFO] Input filename: /tmp/fileWoPiuq
[INFO] ONNX IR version: 0.0.6
[INFO] Opset version: 11
[INFO] Producer name: keras2onnx
[INFO] Producer version: 1.8.1
[INFO] Domain: onnxmltools
[INFO] Model version: 0
[INFO] Doc string:
[INFO] ----------------------------------------------------------------
[WARNING] /trt_oss_src/TensorRT/parsers/onnx/onnx2trt_utils.cpp:364: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[INFO] Detected input dimensions from the model: (-1, 3, 512, 512)
[INFO] Model has dynamic shape. Setting up optimization profiles.
[INFO] Using optimization profile min shape: (1, 1, 320, 320) for input: input_1
[INFO] Using optimization profile opt shape: (4, 1, 320, 320) for input: input_1
[INFO] Using optimization profile max shape: (16, 1, 320, 320) for input: input_1
> [WARNING] Requesting FP16 data type but platform has no support, ignored.
> [ERROR] 4: [network.cpp::validate::2724] Error Code 4: Internal Error (input_1: for dimension number 1 in profile 0 does not match network definition (got min=1, opt=1, max=1), expected min=opt=max=3).)
> [ERROR] Unable to create engine
2022-01-18 23:26:48,094 [INFO] tlt.components.docker_handler.docker_handler: Stopping container.
The Convert to TensorRT engine(INT8)
!tao converter -k $KEY \
-c $USER_EXPERIMENT_DIR/export/isbi_cal.bin \
-e $USER_EXPERIMENT_DIR/export/trt.int8.tlt.isbi.engine \
-i nchw \
-t int8 \
-p input_1,1x1x320x320,4x1x320x320,16x1x320x320 \
$USER_EXPERIMENT_DIR/isbi_experiment_retrain/weights/model_isbi_retrained.etlt
Also fails with
2022-01-18 23:39:23,260 [INFO] root: Registry: [‘nvcr.io’]
2022-01-18 23:39:23,381 [INFO] tlt.components.instance_handler.local_instance: Running command in container: nvcr.io/nvidia/tao/tao-toolkit-tf:v3.21.11-tf1.15.5-py3
[INFO] [MemUsageChange] Init CUDA: CPU +160, GPU +0, now: CPU 166, GPU 1720 (MiB)
[INFO] ----------------------------------------------------------------
[INFO] Input filename: /tmp/fileHq9FvN
[INFO] ONNX IR version: 0.0.6
[INFO] Opset version: 11
[INFO] Producer name: keras2onnx
[INFO] Producer version: 1.8.1
[INFO] Domain: onnxmltools
[INFO] Model version: 0
[INFO] Doc string:
[INFO] ----------------------------------------------------------------
[WARNING] /trt_oss_src/TensorRT/parsers/onnx/onnx2trt_utils.cpp:364: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[INFO] Detected input dimensions from the model: (-1, 3, 512, 512)
[INFO] Model has dynamic shape. Setting up optimization profiles.
[INFO] Using optimization profile min shape: (1, 1, 320, 320) for input: input_1
[INFO] Using optimization profile opt shape: (4, 1, 320, 320) for input: input_1
[INFO] Using optimization profile max shape: (16, 1, 320, 320) for input: input_1
> [ERROR] 4: [network.cpp::validate::2724] Error Code 4: Internal Error (input_1: for dimension number 1 in profile 0 does not match network definition (got min=1, opt=1, max=1), expected min=opt=max=3).)
> [ERROR] Unable to create engine
2022-01-18 23:39:24,987 [INFO] tlt.components.docker_handler.docker_handler: Stopping container.