Error while pruning .tlt model created during efficientdet-d0 model

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

Please provide the following information when requesting support.

• Hardware (NVIDIA RTX 3080Ti)
• Network Type (Efficientdet)
• TLT Version (tao-tf1)
• Training Spec file
training_config {
train_batch_size: 8
iterations_per_loop: 1
use_xla: False
disable_logging: False
stop_at_epoch: 2
resume: False
checkpoint_period: 10
lr_warmup_init: 0.00005
learning_rate: 0.1
keep_checkpoint_max: 10
num_examples_per_epoch: 50
num_epochs: 2
profile_skip_steps: 100
tf_random_seed: 42
}
dataset_config {
num_classes: 91
image_size: “512, 512”
training_file_pattern: “/workspace/tao-tf1/nvidia_tao_tf1/cv/dataset/coco/tfrecords/"
validation_file_pattern: "/workspace/tao-tf1/nvidia_tao_tf1/cv/dataset/coco/tfrecords/”
validation_json_file: “/workspace/tao-tf1/nvidia_tao_tf1/cv/dataset/coco/annotations/instances_val2017.json”
}
eval_config {
eval_batch_size: 8
eval_epoch_cycle: 1
eval_after_training: False
eval_samples: 5
}
augmentation_config {
random_crop_min_scale: 0.1
random_crop_min_scale: 2.0
rand_hflip: True
}
model_config {
model_name: ‘efficientdet-d0’
}
• How to reproduce the issue ?
python efficientdet/scripts/prune.py -m /workspace/tao-tf1/nvidia_tao_tf1/cv/efficientdet/results/model.epoch-0.tlt -o /workspace/tao-tf1/nvidia_tao_tf1/cv/efficientdet/results/prune -pth 0.5

INFO:tensorflow:Restoring parameters from /tmp/tmpuufymj1m/model.ckpt-1837
2024-07-05 04:59:04,389 [TAO Toolkit] [INFO] tensorflow 1284: Restoring parameters from /tmp/tmpuufymj1m/model.ckpt-1837
2024-07-05 04:59:04,886 [TAO Toolkit] [INFO] main 227: Pruning process will take some time. Please wait…
2024-07-05 04:59:04,918 [TAO Toolkit] [INFO] nvidia_tao_tf1.core.pruning.pruning 981: Exploring graph for retainable indices
2024-07-05 04:59:04,918 [TAO Toolkit] [INFO] root 2102: Unknown layer type: <class ‘tensorflow.python.keras.engine.input_layer.InputLayer’>
Traceback (most recent call last):
File “efficientdet/scripts/prune.py”, line 310, in
main()
File “efficientdet/scripts/prune.py”, line 306, in main
raise e
File “efficientdet/scripts/prune.py”, line 294, in main
run_pruning(args)
File “efficientdet/scripts/prune.py”, line 229, in run_pruning
pruned_model = prune(
File “/workspace/tao-tf1/nvidia_tao_tf1/core/pruning/pruning.py”, line 1606, in prune
return pruner.prune(model, layer_config_overrides, output_layers_with_outbound_nodes)
File “/workspace/tao-tf1/nvidia_tao_tf1/core/pruning/pruning.py”, line 1211, in prune
model = self._explore(model)
File “/workspace/tao-tf1/nvidia_tao_tf1/core/pruning/pruning.py”, line 1142, in _explore
raise NotImplementedError(“Unknown layer type: %s” % type(layer))
NotImplementedError: Unknown layer type: <class ‘tensorflow.python.keras.engine.input_layer.InputLayer’>

3

May I know that if you can follow the command in the doc or notebook(tao_tutorials/notebooks/tao_launcher_starter_kit/efficientdet_tf1/efficientdet.ipynb at main · NVIDIA/tao_tutorials · GitHub) to check if it works?
You can open a terminal, then

$ tao model efficientdet_tf1 /bin/bash

then,

# efficientdet_tf1 prune -m $USER_EXPERIMENT_DIR/experiment_dir_unpruned/model.epoch-$NUM_EPOCH.tlt \
                        -o $USER_EXPERIMENT_DIR/experiment_dir_pruned \
                        -pth 0.7
4

I am using tensorflow tf1 backend.

5

(joel) quest@INTVMLT3947:~/nvidia_tao/tensorflow_backend/tao_tensorflow1_backend$ tao model efficientdet_tf1 prune -m /home/quest/nvidia_tao/tensorflow_backend/tao_tensorflow1_backend/nvidia_tao_tf1/cv/efficientdet/results/efficiendet-d0/model.epoch-0.tlt /home/quest/nvidia_tao/tensorflow_backend/tao_tensorflow1_backend/nvidia_tao_tf1/cv/efficientdet/results/efficiendet-d0/prune
2024-07-17 21:57:50,935 [TAO Toolkit] [INFO] root 160: Registry: [‘nvcr.io’]
2024-07-17 21:57:50,978 [TAO Toolkit] [INFO] nvidia_tao_cli.components.instance_handler.local_instance 360: Running command in container: nvcr.io/nvidia/tao/tao-toolkit:5.0.0-tf1.15.5
2024-07-17 21:57:50,986 [TAO Toolkit] [WARNING] nvidia_tao_cli.components.docker_handler.docker_handler 288:
Docker will run the commands as root. If you would like to retain your
local host permissions, please add the “user”:“UID:GID” in the
DockerOptions portion of the “/home/quest/.tao_mounts.json” file. You can obtain your
users UID and GID by using the “id -u” and “id -g” commands on the
terminal.
2024-07-17 21:57:50,986 [TAO Toolkit] [INFO] nvidia_tao_cli.components.docker_handler.docker_handler 301: Printing tty value True
Error response from daemon: No such container: 8c93c8d67ed591d12a46e4108f55207aed72babc5b361e3b325d6b6e7ada48b2
2024-07-17 21:57:51,606 [TAO Toolkit] [INFO] nvidia_tao_cli.components.docker_handler.docker_handler 363: Stopping container.
(joel) quest@INTVMLT3947:~/nvidia_tao/tensorflow_backend/tao_tensorflow1_backend$

6

Can you double check the command? Is it missing -o ?

7

(base) quest@INTVMLT3947:~/nvidia_tao/tensorflow_backend/tao_tensorflow1_backend/nvidia_tao_tf1/cv/efficientdet$ tao model efficientdet_tf1 prune -m /home/quest/nvidia_tao/tensorflow_backend/tao_tensorflow1_backend/nvidia_tao_tf1/cv/efficientdet/results/efficiendet-d0/model.epoch-0.tlt -o /home/quest/nvidia_tao/tensorflow_backend/tao_tensorflow1_backend/nvidia_tao_tf1/cv/efficientdet/results/efficiendet-d0/prune
2024-07-18 09:33:42,003 [TAO Toolkit] [INFO] root 160: Registry: [‘nvcr.io’]
2024-07-18 09:33:42,041 [TAO Toolkit] [INFO] nvidia_tao_cli.components.instance_handler.local_instance 360: Running command in container: nvcr.io/nvidia/tao/tao-toolkit:5.0.0-tf1.15.5
2024-07-18 09:33:42,053 [TAO Toolkit] [WARNING] nvidia_tao_cli.components.docker_handler.docker_handler 288:
Docker will run the commands as root. If you would like to retain your
local host permissions, please add the “user”:“UID:GID” in the
DockerOptions portion of the “/home/quest/.tao_mounts.json” file. You can obtain your
users UID and GID by using the “id -u” and “id -g” commands on the
terminal.
2024-07-18 09:33:42,053 [TAO Toolkit] [INFO] nvidia_tao_cli.components.docker_handler.docker_handler 301: Printing tty value True
Error response from daemon: No such container: 582498b8c0b962d1c9941b5657d283aba825d9656ecfdb98dd7901a0442096c0
2024-07-18 09:33:44,531 [TAO Toolkit] [INFO] nvidia_tao_cli.components.docker_handler.docker_handler 363: Stopping container.
(base) quest@INTVMLT3947:~/nvidia_tao/tensorflow_backend/tao_tensorflow1_backend/nvidia_tao_tf1/cv/efficientdet$

8

Please run below to check if the pruned model is generated successfully.
$ tao model efficientdet_tf1 run ls /home/quest/nvidia_tao/tensorflow_backend/tao_tensorflow1_backend/nvidia_tao_tf1/cv/efficientdet/results/efficiendet-d0/prune

9

(base) quest@INTVMLT3947:~/nvidia_tao/tensorflow_backend/tao_tensorflow1_backend/nvidia_tao_tf1/cv/efficientdet$ tao model efficientdet_tf1 run ls /home/quest/nvidia_tao/tensorflow_backend/tao_tensorflow1_backend/nvidia_tao_tf1/cv/efficientdet/results/efficiendet-d0/prune
2024-07-19 23:09:57,361 [TAO Toolkit] [INFO] root 160: Registry: [‘nvcr.io’]
2024-07-19 23:09:57,397 [TAO Toolkit] [INFO] nvidia_tao_cli.components.instance_handler.local_instance 360: Running command in container: nvcr.io/nvidia/tao/tao-toolkit:5.0.0-tf1.15.5
2024-07-19 23:09:57,403 [TAO Toolkit] [WARNING] nvidia_tao_cli.components.docker_handler.docker_handler 288:
Docker will run the commands as root. If you would like to retain your
local host permissions, please add the “user”:“UID:GID” in the
DockerOptions portion of the “/home/quest/.tao_mounts.json” file. You can obtain your
users UID and GID by using the “id -u” and “id -g” commands on the
terminal.
2024-07-19 23:09:57,403 [TAO Toolkit] [INFO] nvidia_tao_cli.components.docker_handler.docker_handler 301: Printing tty value True
Error response from daemon: No such container: 81b802d1e1b1786d3b2caebe82e40087d04fc9994b534c0c4458338949c5594b
2024-07-19 23:09:59,762 [TAO Toolkit] [INFO] nvidia_tao_cli.components.docker_handler.docker_handler 363: Stopping container

10

I directly ran the pruning from Tensorflow TF_1 backend. This issue

root@INTVMLT3947:/workspace/tao-tf1/nvidia_tao_tf1/cv# python efficientdet/scripts/prune.py -m /workspace/tao-tf1/nvidia_tao_tf1/cv/efficientdet/results/exp2/model.epoch-1.tlt -o /workspace/tao-tf1/nvidia_tao_tf1/cv/efficientdet/results/exp2/prune
2024-07-19 17:37:38.602357: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcudart.so.12
WARNING:tensorflow:Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.
Using TensorFlow backend.
WARNING:tensorflow:TensorFlow will not use sklearn by default. This improves performance in some cases. To enable sklearn export the environment variable TF_ALLOW_IOLIBS=1.
2024-07-19 17:37:39,409 [TAO Toolkit] [WARNING] tensorflow 43: TensorFlow will not use sklearn by default. This improves performance in some cases. To enable sklearn export the environment variable TF_ALLOW_IOLIBS=1.
WARNING:tensorflow:TensorFlow will not use Dask by default. This improves performance in some cases. To enable Dask export the environment variable TF_ALLOW_IOLIBS=1.
2024-07-19 17:37:39,434 [TAO Toolkit] [WARNING] tensorflow 42: TensorFlow will not use Dask by default. This improves performance in some cases. To enable Dask export the environment variable TF_ALLOW_IOLIBS=1.
WARNING:tensorflow:TensorFlow will not use Pandas by default. This improves performance in some cases. To enable Pandas export the environment variable TF_ALLOW_IOLIBS=1.
2024-07-19 17:37:39,436 [TAO Toolkit] [WARNING] tensorflow 43: TensorFlow will not use Pandas by default. This improves performance in some cases. To enable Pandas export the environment variable TF_ALLOW_IOLIBS=1.
2024-07-19 17:37:39,753 [TAO Toolkit] [INFO] root 2102: Starting EfficientDet pruning.
2024-07-19 17:37:42,226 [TAO Toolkit] [INFO] root 2082: Loading weights from /workspace/tao-tf1/nvidia_tao_tf1/cv/efficientdet/results/exp2/model.epoch-1.tlt
WARNING:tensorflow:From efficientdet/scripts/prune.py:214: The name tf.keras.backend.get_session is deprecated. Please use tf.compat.v1.keras.backend.get_session instead.

2024-07-19 17:37:42,271 [TAO Toolkit] [WARNING] tensorflow 137: From efficientdet/scripts/prune.py:214: The name tf.keras.backend.get_session is deprecated. Please use tf.compat.v1.keras.backend.get_session instead.

2024-07-19 17:37:42.281337: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcuda.so.1
2024-07-19 17:37:44.063305: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1082] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-07-19 17:37:44.063397: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1674] Found device 0 with properties:
name: NVIDIA GeForce RTX 3080 Ti Laptop GPU major: 8 minor: 6 memoryClockRate(GHz): 1.395
pciBusID: 0000:01:00.0
2024-07-19 17:37:44.063411: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcudart.so.12
2024-07-19 17:37:44.078957: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcublas.so.12
2024-07-19 17:37:44.080002: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcufft.so.11
2024-07-19 17:37:44.080211: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcurand.so.10
2024-07-19 17:37:44.081825: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcusolver.so.11
2024-07-19 17:37:44.082335: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcusparse.so.12
2024-07-19 17:37:44.082444: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcudnn.so.8
2024-07-19 17:37:44.082509: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1082] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-07-19 17:37:44.082620: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1082] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-07-19 17:37:44.082676: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1802] Adding visible gpu devices: 0
2024-07-19 17:37:44.113546: I tensorflow/core/platform/profile_utils/cpu_utils.cc:109] CPU Frequency: 2918400000 Hz
2024-07-19 17:37:44.114615: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x173fbc0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2024-07-19 17:37:44.114640: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
2024-07-19 17:37:44.155043: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1082] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-07-19 17:37:44.155235: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x4683ac0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2024-07-19 17:37:44.155250: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 3080 Ti Laptop GPU, Compute Capability 8.6
2024-07-19 17:37:44.155362: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1082] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-07-19 17:37:44.155429: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1674] Found device 0 with properties:
name: NVIDIA GeForce RTX 3080 Ti Laptop GPU major: 8 minor: 6 memoryClockRate(GHz): 1.395
pciBusID: 0000:01:00.0
2024-07-19 17:37:44.155443: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcudart.so.12
2024-07-19 17:37:44.155457: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcublas.so.12
2024-07-19 17:37:44.155462: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcufft.so.11
2024-07-19 17:37:44.155467: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcurand.so.10
2024-07-19 17:37:44.155472: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcusolver.so.11
2024-07-19 17:37:44.155476: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcusparse.so.12
2024-07-19 17:37:44.155481: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcudnn.so.8
2024-07-19 17:37:44.155510: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1082] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-07-19 17:37:44.155572: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1082] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-07-19 17:37:44.155616: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1802] Adding visible gpu devices: 0
2024-07-19 17:37:44.155626: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcudart.so.12
2024-07-19 17:37:44.159099: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1214] Device interconnect StreamExecutor with strength 1 edge matrix:
2024-07-19 17:37:44.159109: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1220] 0
2024-07-19 17:37:44.159113: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1233] 0: N
2024-07-19 17:37:44.159216: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1082] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-07-19 17:37:44.159295: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1082] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-07-19 17:37:44.159361: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1359] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14109 MB memory) → physical GPU (device: 0, name: NVIDIA GeForce RTX 3080 Ti Laptop GPU, pci bus id: 0000:01:00.0, compute capability: 8.6)
WARNING:tensorflow:From efficientdet/scripts/prune.py:215: The name tf.global_variables_initializer is deprecated. Please use tf.compat.v1.global_variables_initializer instead.

2024-07-19 17:37:45,403 [TAO Toolkit] [WARNING] tensorflow 137: From efficientdet/scripts/prune.py:215: The name tf.global_variables_initializer is deprecated. Please use tf.compat.v1.global_variables_initializer instead.

INFO:tensorflow:Restoring parameters from /tmp/tmp0k397qe4/model.ckpt-3674
2024-07-19 17:37:45,651 [TAO Toolkit] [INFO] tensorflow 1284: Restoring parameters from /tmp/tmp0k397qe4/model.ckpt-3674
2024-07-19 17:37:46,175 [TAO Toolkit] [INFO] main 227: Pruning process will take some time. Please wait…
2024-07-19 17:37:46,208 [TAO Toolkit] [INFO] nvidia_tao_tf1.core.pruning.pruning 981: Exploring graph for retainable indices
2024-07-19 17:37:46,208 [TAO Toolkit] [INFO] root 2102: Unknown layer type: <class ‘tensorflow.python.keras.engine.input_layer.InputLayer’>
Traceback (most recent call last):
File “efficientdet/scripts/prune.py”, line 310, in
main()
File “efficientdet/scripts/prune.py”, line 306, in main
raise e
File “efficientdet/scripts/prune.py”, line 294, in main
run_pruning(args)
File “efficientdet/scripts/prune.py”, line 229, in run_pruning
pruned_model = prune(
File “/workspace/tao-tf1/nvidia_tao_tf1/core/pruning/pruning.py”, line 1606, in prune
return pruner.prune(model, layer_config_overrides, output_layers_with_outbound_nodes)
File “/workspace/tao-tf1/nvidia_tao_tf1/core/pruning/pruning.py”, line 1211, in prune
model = self._explore(model)
File “/workspace/tao-tf1/nvidia_tao_tf1/core/pruning/pruning.py”, line 1142, in _explore
raise NotImplementedError(“Unknown layer type: %s” % type(layer))
NotImplementedError: Unknown layer type: <class ‘tensorflow.python.keras.engine.input_layer.InputLayer’>

11

I have trained using this tensorflow backend container…

The issue is this:

NotImplementedError: Unknown layer type: <class ‘tensorflow.python.keras.engine.input_layer.InputLayer’>

12

I run training for 1 epoch and then prune the model successfully.
Please refer to my steps.
$ docker run --runtime=nvidia -it --rm -v /home/morganh:/workspace nvcr.io/nvidia/tao/tao-toolkit:5.0.0-tf1.15.5 /bin/bash
$ efficientdet_tf1 prune -m /workspace/demo_3.0/forum_repro/efficientdet/298730/result/model.epoch-0.tlt -o /workspace/demo_3.0/forum_repro/efficientdet/298730/experiment_dir_pruned -pth 0.7

Attach log as well.
202400720_forum_298730_not_repro_efficientdet_prune.txt (420.9 KB)

13

root@26393b88cca4:/workspace# efficientdet_tf1 prune -m /workspace/nvidia_tao_tf1/cv/efficientdet/results/exp2/model.epoch-1.tlt -o /workspace/nvidia_tao_tf1/cv/efficientdet/results/exp2/prune -pth 0.7
2024-07-20 06:32:54.572755: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcudart.so.12
2024-07-20 06:32:54,599 [TAO Toolkit] [WARNING] tensorflow 40: Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.
2024-07-20 06:32:55.048630: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcudart.so.12
Using TensorFlow backend.
2024-07-20 06:32:55,218 [TAO Toolkit] [WARNING] tensorflow 43: TensorFlow will not use sklearn by default. This improves performance in some cases. To enable sklearn export the environment variable TF_ALLOW_IOLIBS=1.
2024-07-20 06:32:55,238 [TAO Toolkit] [WARNING] tensorflow 42: TensorFlow will not use Dask by default. This improves performance in some cases. To enable Dask export the environment variable TF_ALLOW_IOLIBS=1.
2024-07-20 06:32:55,241 [TAO Toolkit] [WARNING] tensorflow 43: TensorFlow will not use Pandas by default. This improves performance in some cases. To enable Pandas export the environment variable TF_ALLOW_IOLIBS=1.
2024-07-20 06:32:56,239 [TAO Toolkit] [INFO] matplotlib.font_manager 1633: generated new fontManager
2024-07-20 06:32:56.421796: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libnvinfer.so.8
2024-07-20 06:32:56.431582: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcuda.so.1
WARNING:tensorflow:Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.
Using TensorFlow backend.
WARNING:tensorflow:TensorFlow will not use sklearn by default. This improves performance in some cases. To enable sklearn export the environment variable TF_ALLOW_IOLIBS=1.
2024-07-20 06:32:57,481 [TAO Toolkit] [WARNING] tensorflow 43: TensorFlow will not use sklearn by default. This improves performance in some cases. To enable sklearn export the environment variable TF_ALLOW_IOLIBS=1.
WARNING:tensorflow:TensorFlow will not use Dask by default. This improves performance in some cases. To enable Dask export the environment variable TF_ALLOW_IOLIBS=1.
2024-07-20 06:32:57,501 [TAO Toolkit] [WARNING] tensorflow 42: TensorFlow will not use Dask by default. This improves performance in some cases. To enable Dask export the environment variable TF_ALLOW_IOLIBS=1.
WARNING:tensorflow:TensorFlow will not use Pandas by default. This improves performance in some cases. To enable Pandas export the environment variable TF_ALLOW_IOLIBS=1.
2024-07-20 06:32:57,503 [TAO Toolkit] [WARNING] tensorflow 43: TensorFlow will not use Pandas by default. This improves performance in some cases. To enable Pandas export the environment variable TF_ALLOW_IOLIBS=1.
2024-07-20 06:32:57,705 [TAO Toolkit] [INFO] root 2102: Starting EfficientDet pruning.
2024-07-20 06:33:00,236 [TAO Toolkit] [INFO] root 2082: Loading weights from /workspace/nvidia_tao_tf1/cv/efficientdet/results/exp2/model.epoch-1.tlt
WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/efficientdet/scripts/prune.py:214: The name tf.keras.backend.get_session is deprecated. Please use tf.compat.v1.keras.backend.get_session instead.

2024-07-20 06:33:00,279 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/efficientdet/scripts/prune.py:214: The name tf.keras.backend.get_session is deprecated. Please use tf.compat.v1.keras.backend.get_session instead.

WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/efficientdet/scripts/prune.py:215: The name tf.global_variables_initializer is deprecated. Please use tf.compat.v1.global_variables_initializer instead.

2024-07-20 06:33:01,466 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/efficientdet/scripts/prune.py:215: The name tf.global_variables_initializer is deprecated. Please use tf.compat.v1.global_variables_initializer instead.

INFO:tensorflow:Restoring parameters from /tmp/tmp3_9waxvz/model.ckpt-3674
2024-07-20 06:33:01,715 [TAO Toolkit] [INFO] tensorflow 1284: Restoring parameters from /tmp/tmp3_9waxvz/model.ckpt-3674
2024-07-20 06:33:02,213 [TAO Toolkit] [INFO] main 227: Pruning process will take some time. Please wait…
2024-07-20 06:33:02,251 [TAO Toolkit] [INFO] nvidia_tao_tf1.core.pruning.pruning 981: Exploring graph for retainable indices
2024-07-20 06:33:09,022 [TAO Toolkit] [INFO] nvidia_tao_tf1.core.pruning.pruning 1225: Pruning model and appending pruned nodes to new graph

2024-07-20 06:36:37,153 [TAO Toolkit] [INFO] main 245: Pruning ratio (pruned model / original model): 0.9939813452253533
2024-07-20 06:37:33,140 [TAO Toolkit] [INFO] root 2102: Pruning finished successfully.
Execution status: PASS

14

Thanks a lot for the support.

15

but why i am not able to do the same using tao tensorflow docker container available in this GitHub - NVIDIA/tao_tensorflow1_backend: TAO Toolkit deep learning networks with TensorFlow 1.x backend repository. All other tasks are working fine…

16

Glad to know the pruning is working on your side now.

Not sure what happens since your previous log does not show you are working on this github repo.

17

(base) quest@INTVMLT3947:~/nvidia_tao/tensorflow_backend/tao_tensorflow1_backend$ source scripts/envsetup.sh

TAO Toolkit TensorFlow2 build environment set up.

The following environment variables have been set:

NV_TAO_TF_TOP /home/quest/nvidia_tao/tensorflow_backend/tao_tensorflow1_backend

The following functions have been added to your environment:

tao_tf Run command inside the container.
(base) quest@INTVMLT3947:~/nvidia_tao/tensorflow_backend/tao_tensorflow1_backend$ tao_tf
Current root directory /home/quest/nvidia_tao/tensorflow_backend/tao_tensorflow1_backend
/home/quest/nvidia_tao/tensorflow_backend/tao_tensorflow1_backend/runner/tao_tf.py:93: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
if LooseVersion(docker_version) >= LooseVersion(“1.40”):
docker run -it --rm --gpus all -v /home/quest/nvidia_tao/tensorflow_backend/tao_tensorflow1_backend:/workspace/tao-tf1 -v /home/quest/getting_started_v5.3.0/notebooks/tao_data_services:/workspace/tao-experiments -v /home/quest/getting_started_v5.3.0/notebooks/tao_data_services/data:/data -v /home/quest/getting_started_v5.3.0/notebooks/tao_data_services/specs:/specs -v /home/quest/getting_started_v5.3.0/notebooks/tao_data_services/data_services:/results -e PYTHONPATH=/workspace/tao-tf1:$PYTHONPATH --shm-size 16G -w /workspace/tao-tf1 --net=host nvcr.io/nvidia/tao/tao-toolkit:5.0.0-tf1.15.5-base

================
== TensorFlow ==

NVIDIA Release 23.02-tf1 (build 52693369)
TensorFlow Version 1.15.5

Container image Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
Copyright 2017-2023 The TensorFlow Authors. All rights reserved.

Various files include modifications (c) NVIDIA CORPORATION & AFFILIATES. All rights reserved.

This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:

root@INTVMLT3947:/workspace/tao-tf1# python nvidia_tao_tf1/cv/efficientdet/scripts/prune.py -m /workspace/tao-tf1/nvidia_tao_tf1/cv/efficientdet/results/exp2/model.epoch-1.tlt -o /workspace/tao-tf1/nvidia_tao_tf1/cv/efficientdet/results/exp2/prune
prune/ prune1/
root@INTVMLT3947:/workspace/tao-tf1# python nvidia_tao_tf1/cv/efficientdet/scripts/prune.py -m /workspace/tao-tf1/nvidia_tao_tf1/cv/efficientdet/results/exp2/model.epoch-1.tlt -o /workspace/tao-tf1/nvidia_tao_tf1/cv/efficientdet/results/exp2/prune1/ -pth 0.7
2024-07-21 07:07:43.745220: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcudart.so.12
WARNING:tensorflow:Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.
Using TensorFlow backend.
WARNING:tensorflow:TensorFlow will not use sklearn by default. This improves performance in some cases. To enable sklearn export the environment variable TF_ALLOW_IOLIBS=1.
2024-07-21 07:07:44,729 [TAO Toolkit] [WARNING] tensorflow 43: TensorFlow will not use sklearn by default. This improves performance in some cases. To enable sklearn export the environment variable TF_ALLOW_IOLIBS=1.
WARNING:tensorflow:TensorFlow will not use Dask by default. This improves performance in some cases. To enable Dask export the environment variable TF_ALLOW_IOLIBS=1.
2024-07-21 07:07:44,754 [TAO Toolkit] [WARNING] tensorflow 42: TensorFlow will not use Dask by default. This improves performance in some cases. To enable Dask export the environment variable TF_ALLOW_IOLIBS=1.
WARNING:tensorflow:TensorFlow will not use Pandas by default. This improves performance in some cases. To enable Pandas export the environment variable TF_ALLOW_IOLIBS=1.
2024-07-21 07:07:44,757 [TAO Toolkit] [WARNING] tensorflow 43: TensorFlow will not use Pandas by default. This improves performance in some cases. To enable Pandas export the environment variable TF_ALLOW_IOLIBS=1.
2024-07-21 07:07:45,053 [TAO Toolkit] [INFO] root 2102: Starting EfficientDet pruning.
2024-07-21 07:07:47,909 [TAO Toolkit] [INFO] root 2082: Loading weights from /workspace/tao-tf1/nvidia_tao_tf1/cv/efficientdet/results/exp2/model.epoch-1.tlt
WARNING:tensorflow:From nvidia_tao_tf1/cv/efficientdet/scripts/prune.py:214: The name tf.keras.backend.get_session is deprecated. Please use tf.compat.v1.keras.backend.get_session instead.

2024-07-21 07:07:47,974 [TAO Toolkit] [WARNING] tensorflow 137: From nvidia_tao_tf1/cv/efficientdet/scripts/prune.py:214: The name tf.keras.backend.get_session is deprecated. Please use tf.compat.v1.keras.backend.get_session instead.

2024-07-21 07:07:47.975935: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcuda.so.1
2024-07-21 07:07:49.768124: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1082] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-07-21 07:07:49.768221: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1674] Found device 0 with properties:
name: NVIDIA GeForce RTX 3080 Ti Laptop GPU major: 8 minor: 6 memoryClockRate(GHz): 1.395
pciBusID: 0000:01:00.0
2024-07-21 07:07:49.768239: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcudart.so.12
2024-07-21 07:07:49.814583: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcublas.so.12
2024-07-21 07:07:49.818500: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcufft.so.11
2024-07-21 07:07:49.819475: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcurand.so.10
2024-07-21 07:07:49.835410: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcusolver.so.11
2024-07-21 07:07:49.837742: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcusparse.so.12
2024-07-21 07:07:49.837900: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcudnn.so.8
2024-07-21 07:07:49.837972: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1082] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-07-21 07:07:49.838098: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1082] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-07-21 07:07:49.838158: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1802] Adding visible gpu devices: 0
2024-07-21 07:07:49.863250: I tensorflow/core/platform/profile_utils/cpu_utils.cc:109] CPU Frequency: 2918400000 Hz
2024-07-21 07:07:49.863874: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1154340 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2024-07-21 07:07:49.863893: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
2024-07-21 07:07:49.900444: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1082] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-07-21 07:07:49.900592: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x4099f60 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2024-07-21 07:07:49.900608: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 3080 Ti Laptop GPU, Compute Capability 8.6
2024-07-21 07:07:49.900943: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1082] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-07-21 07:07:49.901034: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1674] Found device 0 with properties:
name: NVIDIA GeForce RTX 3080 Ti Laptop GPU major: 8 minor: 6 memoryClockRate(GHz): 1.395
pciBusID: 0000:01:00.0
2024-07-21 07:07:49.901051: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcudart.so.12
2024-07-21 07:07:49.901068: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcublas.so.12
2024-07-21 07:07:49.901073: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcufft.so.11
2024-07-21 07:07:49.901079: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcurand.so.10
2024-07-21 07:07:49.901084: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcusolver.so.11
2024-07-21 07:07:49.901089: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcusparse.so.12
2024-07-21 07:07:49.901095: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcudnn.so.8
2024-07-21 07:07:49.901129: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1082] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-07-21 07:07:49.901202: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1082] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-07-21 07:07:49.901256: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1802] Adding visible gpu devices: 0
2024-07-21 07:07:49.901269: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcudart.so.12
2024-07-21 07:07:49.910379: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1214] Device interconnect StreamExecutor with strength 1 edge matrix:
2024-07-21 07:07:49.910416: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1220] 0
2024-07-21 07:07:49.910428: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1233] 0: N
2024-07-21 07:07:49.910795: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1082] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-07-21 07:07:49.911052: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1082] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-07-21 07:07:49.911206: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1359] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14109 MB memory) → physical GPU (device: 0, name: NVIDIA GeForce RTX 3080 Ti Laptop GPU, pci bus id: 0000:01:00.0, compute capability: 8.6)
WARNING:tensorflow:From nvidia_tao_tf1/cv/efficientdet/scripts/prune.py:215: The name tf.global_variables_initializer is deprecated. Please use tf.compat.v1.global_variables_initializer instead.

2024-07-21 07:07:51,186 [TAO Toolkit] [WARNING] tensorflow 137: From nvidia_tao_tf1/cv/efficientdet/scripts/prune.py:215: The name tf.global_variables_initializer is deprecated. Please use tf.compat.v1.global_variables_initializer instead.

INFO:tensorflow:Restoring parameters from /tmp/tmp_3_um2bm/model.ckpt-3674
2024-07-21 07:07:51,506 [TAO Toolkit] [INFO] tensorflow 1284: Restoring parameters from /tmp/tmp_3_um2bm/model.ckpt-3674
2024-07-21 07:07:52,111 [TAO Toolkit] [INFO] main 227: Pruning process will take some time. Please wait…
2024-07-21 07:07:52,160 [TAO Toolkit] [INFO] nvidia_tao_tf1.core.pruning.pruning 981: Exploring graph for retainable indices
2024-07-21 07:07:52,160 [TAO Toolkit] [INFO] root 2102: Unknown layer type: <class ‘tensorflow.python.keras.engine.input_layer.InputLayer’>
Traceback (most recent call last):
File “nvidia_tao_tf1/cv/efficientdet/scripts/prune.py”, line 310, in
main()
File “nvidia_tao_tf1/cv/efficientdet/scripts/prune.py”, line 306, in main
raise e
File “nvidia_tao_tf1/cv/efficientdet/scripts/prune.py”, line 294, in main
run_pruning(args)
File “nvidia_tao_tf1/cv/efficientdet/scripts/prune.py”, line 229, in run_pruning
pruned_model = prune(
File “/workspace/tao-tf1/nvidia_tao_tf1/core/pruning/pruning.py”, line 1606, in prune
return pruner.prune(model, layer_config_overrides, output_layers_with_outbound_nodes)
File “/workspace/tao-tf1/nvidia_tao_tf1/core/pruning/pruning.py”, line 1211, in prune
model = self._explore(model)
File “/workspace/tao-tf1/nvidia_tao_tf1/core/pruning/pruning.py”, line 1142, in _explore
raise NotImplementedError(“Unknown layer type: %s” % type(layer))
NotImplementedError: Unknown layer type: <class ‘tensorflow.python.keras.engine.input_layer.InputLayer’>
root@INTVMLT3947:/workspace/tao-tf1#

18

while i am running with this method its showing the error. But by using the CLI command you have provided its working fine.

19

any update on this?

20

Could you please double check software requirement in GitHub - NVIDIA/tao_tensorflow1_backend: TAO Toolkit deep learning networks with TensorFlow 1.x backend ?
Another topic mentioned that

" The prune part, gave some trouble as it did not recognize the first layer throwing a NotImplementedError: Unknown layer type ... InputLayer . Long story short, I had to reinstall everything. Then I was able to execute the prune command.".

Although it is not exactly the same as your case, you can take a look.

21

i have done pruning for other models, it was working fine. anyway will take a look into it and get back to you.