# Training Failure for License Plate Detection Toturial **URL:** \ **Category:** TAO Toolkit\ **Tags:** training, tao\ **Created:** [September 16, 2021, 11:19pm UTC](https://forums.developer.nvidia.com/t/training-failure-for-license-plate-detection-toturial/189472 "2021-09-16T23:19:05Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![sina.radmard](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@sina.radmard](https://forums.developer.nvidia.com/u/sina.radmard)\ **Post date:** [September 16, 2021, 11:19pm UTC](https://forums.developer.nvidia.com/t/training-failure-for-license-plate-detection-toturial/189472/1 "2021-09-16T23:19:05Z")
Please provide the following information when requesting support. • Hardware (T4/V100/Xavier/Nano/etc) GeForce RTX 2070 ![Screenshot from 2021-09-16 16-12-23](https://global.discourse-cdn.com/nvidia/original/3X/9/d/9d4f2bea2b04138ffc7064f52f7b1842a7a0ac42.png) • Network Type (Detectnet\_v2/Faster\_rcnn/Yolo\_v4/LPRnet/Mask\_rcnn/Classification/etc) Detectnet\_v2 • TLT Version (Please run “tlt info --verbose” and share “docker\_tag” here) v3.21.08-py3 (I’m using the latest tao toolbox) • Training spec file(If have, please share here) The default provided in the tutorial. • How to reproduce the issue ? (This is for errors. Please share the command line and the detailed log here.) I have followed all of the steps provided in the [tutorial](https://developer.nvidia.com/blog/creating-a-real-time-license-plate-detection-and-recognition-app), and I can download and split the data-set of 222 images that is provided in the tutorial, but when I get to the training scrip as below, I get the **Training was interrupted** error without even going through a single epoch. `$ tao detectnet_v2 train -e /workspace/openalpr/SPECS_train.txt -r /workspace/openalpr/exp_unpruned -k nvidia_tao` Here is error I am getting: ```auto 2021-09-16 16:13:22,765 [INFO] root: Registry: ['nvcr.io'] 2021-09-16 16:13:22,825 [WARNING] tlt.components.docker_handler.docker_handler: Docker will run the commands as root. If you would like to retain your local host permissions, please add the "user":"UID:GID" in the DockerOptions portion of the "/home/sina/.tao_mounts.json" file. You can obtain your users UID and GID by using the "id -u" and "id -g" commands on the terminal. Using TensorFlow backend. WARNING:tensorflow:Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them. Using TensorFlow backend. WARNING:tensorflow:From /root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/cost_function/cost_auto_weight_hook.py:43: The name tf.train.SessionRunHook is deprecated. Please use tf.estimator.SessionRunHook instead. WARNING:tensorflow:From /root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/tfhooks/checkpoint_saver_hook.py:25: The name tf.train.CheckpointSaverHook is deprecated. Please use tf.estimator.CheckpointSaverHook instead. WARNING:tensorflow:From /root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/scripts/train.py:68: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead. WARNING:tensorflow:From /root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/scripts/train.py:68: The name tf.logging.INFO is deprecated. Please use tf.compat.v1.logging.INFO instead. WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/horovod/tensorflow/ __init__.py:117: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead. 2021-09-16 23:13:28,452 [WARNING] tensorflow: From /usr/local/lib/python3.6/dist-packages/horovod/tensorflow/ __init__.py:117: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead. WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/horovod/tensorflow/ __init__.py:143: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead. 2021-09-16 23:13:28,452 [WARNING] tensorflow: From /usr/local/lib/python3.6/dist-packages/horovod/tensorflow/ __init__.py:143: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead. 2021-09-16 23:13:28,878 [INFO] iva.common.logging.logging: Log file already exists at /workspace/openalpr/exp_unpruned/status.json 2021-09-16 23:13:28,878 [INFO] __main__ : Loading experiment spec at /workspace/openalpr/SPECS_train.txt. 2021-09-16 23:13:28,879 [INFO] iva.detectnet_v2.spec_handler.spec_loader: Merging specification from /workspace/openalpr/SPECS_train.txt WARNING:tensorflow:From /root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/cost_function/cost_auto_weight_hook.py:107: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead. 2021-09-16 23:13:28,937 [WARNING] tensorflow: From /root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/cost_function/cost_auto_weight_hook.py:107: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead. WARNING:tensorflow:From /root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/cost_function/cost_auto_weight_hook.py:110: The name tf.get_variable is deprecated. Please use tf.compat.v1.get_variable instead. 2021-09-16 23:13:28,938 [WARNING] tensorflow: From /root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/cost_function/cost_auto_weight_hook.py:110: The name tf.get_variable is deprecated. Please use tf.compat.v1.get_variable instead. WARNING:tensorflow:From /root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/cost_function/cost_auto_weight_hook.py:113: The name tf.assign is deprecated. Please use tf.compat.v1.assign instead. 2021-09-16 23:13:28,939 [WARNING] tensorflow: From /root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/cost_function/cost_auto_weight_hook.py:113: The name tf.assign is deprecated. Please use tf.compat.v1.assign instead. WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:517: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead. 2021-09-16 23:13:28,959 [WARNING] tensorflow: From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:517: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead. WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:4138: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead. 2021-09-16 23:13:28,960 [WARNING] tensorflow: From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:4138: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead. WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:1834: The name tf.nn.fused_batch_norm is deprecated. Please use tf.compat.v1.nn.fused_batch_norm instead. 2021-09-16 23:13:28,977 [WARNING] tensorflow: From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:1834: The name tf.nn.fused_batch_norm is deprecated. Please use tf.compat.v1.nn.fused_batch_norm instead. 2021-09-16 23:13:29,628 [INFO] __main__ : Training was interrupted. Time taken to run __main__ :main: 0:00:01.182444. 2021-09-16 16:13:30,507 [INFO] tlt.components.docker_handler.docker_handler: Stopping container. ``` Here is my cpu info in case it helps: ![Screenshot from 2021-09-16 16-16-28](https://global.discourse-cdn.com/nvidia/original/3X/a/2/a2d2106335139b96f8b816ec7090f577f66ce17b.png) I would appreciate any help as the error log is not helpful at all. ---
**Author:** ![Morganh](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/morganh/32/9748_2.png) [@Morganh](https://forums.developer.nvidia.com/u/Morganh)\ **Post date:** [September 17, 2021, 3:03am UTC](https://forums.developer.nvidia.com/t/training-failure-for-license-plate-detection-toturial/189472/3 "2021-09-17T03:03:23Z")
Can you change `-k nvidia_tao` to `-k nvidia_tlt` and retry? There is error in the blog when we update the name from tlt to tao. The model’s key is still nvidia\_tlt instead of nvidia\_tao. ---
**Author:** ![sina.radmard](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@sina.radmard](https://forums.developer.nvidia.com/u/sina.radmard)\ **Post date:** [September 17, 2021, 11:03pm UTC](https://forums.developer.nvidia.com/t/training-failure-for-license-plate-detection-toturial/189472/4 "2021-09-17T23:03:55Z")
Thank you so much, it finally resolved my frustration after 2 days! Just as a feedback, NVIDIA is providing an amazing resource for 3rd party developers and I really appreciate that, but it is very confusing navigating through all of the different libraries, tutorials, and posts. ---
**Author:** ![Morganh](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/morganh/32/9748_2.png) [@Morganh](https://forums.developer.nvidia.com/u/Morganh)\ **Post date:** [September 18, 2021, 12:53am UTC](https://forums.developer.nvidia.com/t/training-failure-for-license-plate-detection-toturial/189472/5 "2021-09-18T00:53:57Z")
Sorry for that, we’ll fix it in the blog. ---
**Author:** ![TomNVIDIA](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/tomnvidia/32/14181_2.png) [@TomNVIDIA](https://forums.developer.nvidia.com/u/TomNVIDIA)\ **Post date:** [October 12, 2021, 4:23pm UTC](https://forums.developer.nvidia.com/t/training-failure-for-license-plate-detection-toturial/189472/7 "2021-10-12T16:23:27Z")
Closed on October 12, 2021, 4:23pm UTC ---
**Author:** ![Morganh](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/morganh/32/9748_2.png) [@Morganh](https://forums.developer.nvidia.com/u/Morganh)\ **Post date:** [October 13, 2021, 1:50am UTC](https://forums.developer.nvidia.com/t/training-failure-for-license-plate-detection-toturial/189472/9 "2021-10-13T01:50:37Z")
The key is modified in the blog.