Training Interrupted during loading pretrained weights

Please provide the following information when requesting support.

• RTX3050
• Network Type LPRnet
• TAO Version: toolkit_version:
• Training spec file(If have, please share here)
tutorial_spec.txt (1.2 KB)

• How to reproduce the issue ? (This is for errors. Please share the command line and the detailed log here.)

I am trying to train the LPRnet with my own dataset, which contains 1 training set and 1 label set only to make sure it can train before i prepare my dataset. However, I have the following issue.

INFO: Log file already exists at /home/jason/Desktop/jason/runlpr/results/status.json
INFO: Merging specification from /home/jason/Desktop/jason/runlpr/tutorial_spec.txt
INFO: Loading pretrained weights. This may take a while...
INFO: Training was interrupted
INFO: Training was interrupted.
Execution status: PASS
2024-03-05 11:25:20,804 [TAO Toolkit] [INFO] nvidia_tao_cli.components.docker_handler.docker_handler 363: Stopping container.

Command line used:

tao model lprnet train -e /home/jason/Desktop/jason/runlpr/tutorial_spec.txt -r /home/jason/Desktop/jason/runlpr/results/ -k nvidia-tao -m /home/jason/Desktop/jason/runlpr/us_lprnet_baseline18_trainable.tlt 

Any idea on why the training is interrupted?

Please check if the key is correct. If you are using ngc pretrained model, need to find and check the key under the ngc model page.

I have checked and changed the model key to nvidia_tlt, but is still having the same error

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

Did you ever run default notebook successfully? tao_tutorials/notebooks/tao_launcher_starter_kit/lprnet/lprnet.ipynb at main · NVIDIA/tao_tutorials · GitHub.

More, to debug, you can also open a terminal, then run as below.
$ tao model lprnet run /bin/bash

Then inside the docker,
# lprnet train -e /home/jason/Desktop/jason/runlpr/tutorial_spec.txt -r /home/jason/Desktop/jason/runlpr/results/ -k nvidia_tlt -m /home/jason/Desktop/jason/runlpr/us_lprnet_baseline18_trainable.tlt

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.