Unet trt.engine Inference error

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

Hi.
I generated Unet trt.engine file through export path. But I get following error while inference task:

100%|██████████| 13/13 [00:01<00:00, 10.92it/s]
0it [00:00, ?it/s]

Phase test: Total 25 files.
Traceback (most recent call last):
File "/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/unet/scripts/inference.py", line 412, in <module>
File "/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/unet/scripts/inference.py", line 408, in main
File "/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/unet/scripts/inference.py", line 318, in run_experiment
File "/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/unet/scripts/inference.py", line 280, in infer_unet
File "/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/unet/scripts/inference.py", line 228, in run_inference_trt
File "/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/unet/scripts/inference.py", line 142, in visualize_masks
File "/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/unet/scripts/inference.py", line 85, in overlay_seg_image
KeyError: ''
Using TensorFlow backend.
2

Can you share your command and full log?

3

Hi. Sorry for being late
This my command:

unet inference --gpu_index 0 \
                         -e /workspace/tlt/results/unet/export/final_spec.txt \
                         -m /workspace/tlt/results/unet/export/trt.engine \
                         -o /workspace/tlt/results/unet/export/inference/ \
                         -k $KEY

Full log:

WARNING:tensorflow:Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.
Using TensorFlow backend.
WARNING:tensorflow:From /root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/unet/scripts/inference.py:45: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.

WARNING:tensorflow:From /root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/unet/scripts/inference.py:45: The name tf.logging.INFO is deprecated. Please use tf.compat.v1.logging.INFO instead.

2021-10-16 15:04:44,860 [INFO] __main__: Loading experiment spec at /workspace/tlt/results/tlt_unet_corrosion1000_resnet101/unpruned_model/export//final_spec.txt.
2021-10-16 15:04:44,861 [INFO] iva.unet.spec_handler.spec_loader: Merging specification from /workspace/tlt/results/tlt_unet_corrosion1000_resnet101/unpruned_model/export//final_spec.txt
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:153: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.

2021-10-16 15:04:44,862 [WARNING] tensorflow: From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:153: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.

2021-10-16 15:04:44,873 [INFO] iva.unet.model.utilities: Label Id 0: Train Id 0
2021-10-16 15:04:44,873 [INFO] iva.unet.model.utilities: Label Id 255: Train Id 1
0%|          | 0/13 [00:00<?, ?it/s]WARNING:tensorflow:From /root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/unet/utils/data_loader.py:414: The name tf.image.resize_images is deprecated. Please use tf.image.resize instead.

2021-10-16 15:04:47,239 [WARNING] tensorflow: From /root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/unet/utils/data_loader.py:414: The name tf.image.resize_images is deprecated. Please use tf.image.resize instead.

100%|██████████| 13/13 [00:01<00:00,  9.92it/s]
0it [00:00, ?it/s]

Phase test: Total 25 files.
Traceback (most recent call last):
File "/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/unet/scripts/inference.py", line 412, in <module>
File "/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/unet/scripts/inference.py", line 408, in main
File "/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/unet/scripts/inference.py", line 318, in run_experiment
File "/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/unet/scripts/inference.py", line 280, in infer_unet
File "/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/unet/scripts/inference.py", line 228, in run_inference_trt
File "/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/unet/scripts/inference.py", line 142, in visualize_masks
File "/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/unet/scripts/inference.py", line 85, in overlay_seg_image
KeyError: ''
Using TensorFlow backend.
4

Can you modify above and try
/workspace/tlt/results/unet/export/inference

5

I tried but got the same result.

6

@Morganh please check this problem

7

Did you run inference against pruned tlt model or unpruned tlt model?

8

I can run inference against unpruned model.
But according to this topic: Unet pruned model Inference error I can not inference against pruned model

10

Refer to Unet pruned model Inference error - #10
Currently, UNet does not support Inference/ Evaluation for pruned model. It only supports the evaluation/ inference for re-trained pruned model. Please do the re-training and then do evaluation.

1 Like
12

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.