TAO classification export issue

CSPdarknet19 classifier was trained with the latest TAO version, there is an error during export. Please find the command and its output below.

tao classification export -m /workspace/tlt-experiments/trainings/results_cspdarknet19/cspdarknet_150.tlt -o /workspace/tlt-experiments/trainings/results_cspdarknet19/cspdarknet_150_fp16_new.etlt --data_type fp16 --gen_ds_config --engine_file /workspace/tlt-experiments/trainings/results_cspdarknet19/cspdarknet_150_fp16_new.engine --verbose

2022-02-14 04:46:59,459 [INFO] root: Registry: ['nvcr.io']
2022-02-14 04:46:59,639 [INFO] tlt.components.instance_handler.local_instance: Running command in container: nvcr.io/nvidia/tao/tao-toolkit-tf:v3.21.11-tf1.15.5-py3
2022-02-14 04:46:59,673 [WARNING] tlt.components.docker_handler.docker_handler:
Docker will run the commands as root. If you would like to retain your
local host permissions, please add the "user":"UID:GID" in the
DockerOptions portion of the "/home/ubuntu/.tao_mounts.json" file. You can obtain your
users UID and GID by using the "id -u" and "id -g" commands on the
terminal.
Using TensorFlow backend.
Using TensorFlow backend.
WARNING:tensorflow:Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.
2022-02-14 04:47:27,016 [INFO] root: Building exporter object.
2022-02-14 04:47:32,291 [INFO] root: Exporting the model.
2022-02-14 04:47:32,291 [INFO] root: Using input nodes: ['input_1']
2022-02-14 04:47:32,292 [INFO] root: Using output nodes: ['predictions/Softmax']
2022-02-14 04:47:32,292 [INFO] iva.common.export.keras_exporter: Using input nodes: ['input_1']
2022-02-14 04:47:32,292 [INFO] iva.common.export.keras_exporter: Using output nodes: ['predictions/Softmax']
NOTE: UFF has been tested with TensorFlow 1.14.0.
WARNING: The version of TensorFlow installed on this system is not guaranteed to work with UFF.
Warning: keepdims is ignored by the UFF Parser and defaults to True
DEBUG [/usr/local/lib/python3.6/dist-packages/uff/converters/tensorflow/converter.py:96] Marking ['predictions/Softmax'] as outputs
Traceback (most recent call last):
File "/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/makenet/scripts/export.py", line 36, in <module>
File "/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/makenet/scripts/export.py", line 32, in main
File "/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/common/export/app.py", line 247, in run_export
File "/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/common/export/keras_exporter.py", line 394, in export
File "/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/makenet/export/classification_exporter.py", line 71, in get_class_labels
File "/usr/lib/python3.6/genericpath.py", line 19, in exists
os.stat(path)
TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType
2022-02-14 04:47:55,824 [INFO] tlt.components.docker_handler.docker_handler: Stopping container.

• Classification CSPdarknet19
• TAO Version nvidia/tao/tao-toolkit-tf: v3.21.11-tf1.15.5-py3

I can reproduce with your command. But please add --classmap_json your_training_result/classmap.json and run again.

The classmap.json is generated in your training result folder.

usage: classification export [-h] [--num_processes NUM_PROCESSES]
                             [--gpus GPUS]
                             [--gpu_index GPU_INDEX [GPU_INDEX ...]]
                             [--use_amp] [--log_file LOG_FILE] -m MODEL -k KEY
                             [-o OUTPUT_FILE] [--force_ptq]
                             [--cal_data_file CAL_DATA_FILE]
                             [--cal_image_dir CAL_IMAGE_DIR]
                             [--data_type {fp32,fp16,int8}] [-s]
                             [--gen_ds_config]
                             [--cal_cache_file CAL_CACHE_FILE]
                             [--batches BATCHES]
                             [--max_workspace_size MAX_WORKSPACE_SIZE]
                             [--max_batch_size MAX_BATCH_SIZE]
                             [--batch_size BATCH_SIZE]
                             [--min_batch_size MIN_BATCH_SIZE]
                             [--opt_batch_size OPT_BATCH_SIZE]
                             [-e EXPERIMENT_SPEC] [--engine_file ENGINE_FILE]
                             [--static_batch_size STATIC_BATCH_SIZE]
                             [--results_dir RESULTS_DIR] [-v]
                             [--classmap_json CLASSMAP_JSON]
                             {calibration_tensorfile,evaluate,export,inference,prune,train}

After adding --classmap_json your_training_result/classmap.json , the label.txt is generated.

Thanks for the quick response, it worked. But both documentation and help command are confusing, --classmap_json appears as an optional parameter there while in fact it’s required. It would be nice to fix it. Thank you in advance.

Thanks for the suggestion. We will improve the document.

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.