LPRNet accuracy doesn't translate when testing on new data

[emilstefanvoicu]

I’m currently training LPRNet as part of my bachelor’s. I have a dataset of about 500 license plate photos which I have expanded in pre-processing by adding rotation, blurring, shearing etc. As per my bachelor’s I have to use Romanian license plates. Here are a few examples:

Imgur: The magic of the Internet (can’t create more than one media attachment as a new user, sorry)
I have tried training the model using exclusively the base augmented dataset without adding any synthetic license plates. After augmentation and adding synth plates I have 15k images in an 80/20 train/validation split. At epoch 535 I got my first 80% accuracy. At this point I’m thinking the model is already overfitted but I was aiming for 90% accuracy.

{“epoch”: 530, “max_epoch”: 575, “time_per_epoch”: “0:01:10.170579”, “eta”: “0:52:37.676053”, “date”: “7/3/2025”, “time”: “17:11:31”, “status”: “RUNNING”, “verbosity”: “INFO”, “message”: “Training loop in progress”, “graphical”: {“loss”: “0.56262165”, “learning_rate”: “0.0010627531”, “accuracy”: “0.8195”}, “kpi”: {“accuracy”: “0.8195”}}

This is my current config after tweaking it around and trying multiple datasets (with or without synth, 10k instead of 15k etc.)

random_seed: 42

lpr_config {
hidden_units: 512
max_label_length: 7
arch: “baseline”
nlayers: 10
}

training_config {
batch_size_per_gpu: 128
num_epochs: 350
learning_rate {
soft_start_annealing_schedule {
min_learning_rate: 8e-5
max_learning_rate: 2e-3
soft_start: 0.1
annealing: 0.7
}
}
regularizer {
type: L2
weight: 1e-5
}
}

eval_config {
validation_period_during_training: 5
batch_size: 1
}

augmentation_config {
output_width: 96
output_height: 48
output_channel: 3
max_rotate_degree: 3
rotate_prob: 0.4
gaussian_kernel_size: 3
gaussian_kernel_size: 5
blur_prob: 0.3
reverse_color_prob: 0.1
keep_original_prob: 0.5
}

dataset_config {
data_sources: {
label_directory_path: “/workspace/tao-experiments/Dataset_02/train/labels”
image_directory_path: “/workspace/tao-experiments/Dataset_02/train/images”
}
characters_list_file: “/workspace/tao-experiments/characters_list.txt”
validation_data_sources: {
label_directory_path: “/workspace/tao-experiments/Dataset_02/val/labels”
image_directory_path: “/workspace/tao-experiments/Dataset_02/val/images”
}
}
My specs are as follows: RTX 3070 Laptop Edition 8GB VRAM, 64GB system RAM I’m running TAO on WSL2. I installed TAO using the old guide. Making a conda virtual env installing all the prereqs etc.

My main problem is that when running inference on my 82% accuracy model it fails to correctly predict even one license plate out of a sample of 40. It gets close on many e.g AR10MSE predicts it as AR0MSE, or IS16LWZ is predicted as either IS15WZ or IS16WZ. Now my main question is, can I continue trying to train with maybe a different spec file with some changes. Or should I keep my dataset and start from scratch using the notebook version of LPRNet? If you think I can continue like this I’d like some pointers for the spec file since I’m pretty stumped.

Also really important: do output_width and output_height have to be exactly the resolution of the input images?

[Morganh]

You can continue training with your dataset and spec file but try these steps:

1. Please use the pretrained LPRNet models mentioned in the notebook.
2. Increase dataset with more real images if possible.
3. For hard-to-distinguish characters, consider increasing the diversity of this kind of real plates.
4. Ensure synthetic plates are realistic and varied.
5. setup experiments with smaller batch-size

[emilstefanvoicu]

Thank you for the information! It worked, I got to 90%+ accuracy on both training values and inference results.

These are the improvements I did if anyone else has similar problems:

1. I pulled the nvidia docker container for tao 5.0.0. The one I was using was for a much earlier version of TAO.
2. I used the pretrained LPRNet model for US license plates since I am training the model on Romanian license plates.
3. I lowered the amount of synthetic data plates I was using to about 3000. The total dataset came out to be 10k with 7000 real life images.
4. I tuned learning parameters to be closer to the training spec file in the TAO notebook guides and lowered batch size to 64.

The biggest impact out of these changes by far was using the pretrained model.