The LPD and LPR models in the TAO tool do not work well

The config file is not correct. This is an issue in it. Please modify

uff-input-dims=3;480;640;0

to

uff-input-dims=3;1168;720;0

And also modify

input-object-min-height=30
input-object-min-width=40

to

input-object-min-height=73
input-object-min-width=45

Also, for your tao infer spec file,
please modify

image_width: 1248
image_height: 384

to

image_width: 720
image_height: 1168

Thank you very much. The CCPD model can correctly inference the pictures downloaded through Google, but the LPDnet cannot correctly recognize the pictures and videos taken with the mobile phone. Any ideas?

Any difference between Google’s pictures and mobile phone’s pictures?

I think there is no difference between them. When I put the video into the deepstream-lpr-app test, LPDnet cannot locate the license plate position correctly, but the result is correct when I use LPDnet to infer the screenshot of the vehicle. I think there is still an issue in lpd_ccpd_config. My configuration file is as follows. Any ideas?

4

################################################################################
# Copyright (c) 2020, NVIDIA CORPORATION. All rights reserved.
#
# Permission is hereby granted, free of charge, to any person obtaining a
# copy of this software and associated documentation files (the "Software"),
# to deal in the Software without restriction, including without limitation
# the rights to use, copy, modify, merge, publish, distribute, sublicense,
# and/or sell copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
# DEALINGS IN THE SOFTWARE.
################################################################################

# Following properties are mandatory when engine files are not specified:
#   int8-calib-file(Only in INT8)
#   Caffemodel mandatory properties: model-file, proto-file, output-blob-names
#   UFF: uff-file, input-dims, uff-input-blob-name, output-blob-names
#   ONNX: onnx-file
#
# Mandatory properties for detectors:
#   num-detected-classes
#
# Mandatory properties for classifiers:
#   classifier-threshold, is-classifier
#
# Optional properties for classifiers:
#   classifier-async-mode(Secondary mode only, Default=false)
#
# Following properties are always recommended:
#   batch-size(Default=1)
#
# Other optional properties:
#   net-scale-factor(Default=1), network-mode(Default=0 i.e FP32),
#   mean-file, gie-unique-id(Default=0), offsets, gie-mode (Default=1 i.e. primary),
#   custom-lib-path, network-mode(Default=0 i.e FP32)
#
# The values in the config file are overridden by values set through GObject
# properties.

[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
model-color-format=1
labelfile-path=../models/LP/LPD/ccpd_label.txt
tlt-encoded-model=../models/LP/LPD/ccpd_pruned.etlt
tlt-model-key=nvidia_tlt
model-engine-file=../models/LP/LPD/ccpd_pruned.etlt_b16_gpu0_int8.engine
int8-calib-file=../models/LP/LPD/ccpd_cal.bin
uff-input-dims=3;1168;720;0
uff-input-blob-name=input_1
batch-size=16
## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=1
num-detected-classes=1
##1 Primary 2 Secondary
process-mode=2
interval=0
gie-unique-id=2
#0 detector 1 classifier 2 segmentatio 3 instance segmentation
network-type=0
operate-on-gie-id=1
operate-on-class-ids=0
#no cluster
cluster-mode=3
output-blob-names=output_cov/Sigmoid;output_bbox/BiasAdd
input-object-min-height=73
input-object-min-width=45
#GPU:1  VIC:2(Jetson only)
#scaling-compute-hw=2
#enable-dla=1

[class-attrs-all]
pre-cluster-threshold=0.3
roi-top-offset=0
roi-bottom-offset=0
detected-min-w=0
detected-min-h=0
detected-max-w=0
detected-max-h=0


Please try below lower pre-cluster-threshold, for example,

     pre-cluster-threshold=0.05

Then, try fp16 or fp32 mode.

I followed your suggestion lower pre-cluster-threshold and tried fp32 mode, the result is still not good, as shown below:

Could you add below and retry?

maintain-aspect-ratio=1

LPDnet works sometimes, but it is still bad.

So, the tlt/tao inference is good but deepstream inference is not good. I will check further.
Is it possible to share a small video clip?

More, please try to use usa lpd model as well.
Here is the video we shared previously in the blog. License plate detection and recognition demo using NVIDIA pre-trained models - YouTube

I will continue to try to use the lpd model. Could you please provide an email address? I will send video clip to it soon.

I already send a forum message to you. You can attach video clip or a link for it. Thanks.

Please set model-color-format=0 . It works on my side.

Thank you, it also works on my side.So what is model-color-format means.

model-color-format: Color format required by the model
0: RGB
1: BGR
2: GRAY

1 Like

I want to use rtsp url as input, how should I modify deepstream-lpr-app

Reference: deepstream_lpr_app.c (30.3 KB)
In the commandline, change to rtsp://xxxxx .
Please note that in only supports playing.

1 Like

Hi, Morganh:
Now I am training lprnet with a custom data set, and I use the following spec:

################################################################################
# The MIT License (MIT)
#
# Copyright (c) 2019-2021 NVIDIA CORPORATION
#
# Permission is hereby granted, free of charge, to any person obtaining a
# copy of this software and associated documentation files (the "Software"),
# to deal in the Software without restriction, including without limitation
# the rights to use, copy, modify, merge, publish, distribute, sublicense,
# and/or sell copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
# DEALINGS IN THE SOFTWARE.
################################################################################

random_seed: 42
lpr_config {
  hidden_units: 512
  max_label_length: 8
  arch: "baseline"
  nlayers: 18 #setting nlayers to be 10 to use baseline10 model
}
training_config {
  batch_size_per_gpu: 32
  num_epochs: 24
  learning_rate {
  soft_start_annealing_schedule {
    min_learning_rate: 1e-6
    max_learning_rate: 1e-5
    soft_start: 0.001
    annealing: 0.5
  }
  }
  regularizer {
    type: L2
    weight: 5e-4
  }
}
eval_config {
  validation_period_during_training: 5
  batch_size: 1
}
augmentation_config {
    output_width: 96
    output_height: 48
    output_channel: 3
    keep_original_prob: 0.3
    transform_prob: 0.5
    rotate_degree: 5
}
dataset_config {
  data_sources: {
    label_directory_path: "/workspace/tlt-experiments/data/train/label"
    image_directory_path: "/workspace/tlt-experiments/data/train/image"
  }
  characters_list_file: "/workspace/tlt-experiments/model/ch_lp_characters.txt"
  validation_data_sources: {
    label_directory_path: "/workspace/tlt-experiments/data/val/label"
    image_directory_path: "/workspace/tlt-experiments/data/val/image"
  }
}

The following error occurred:

2021-11-01 09:20:28,676 [INFO] iva.lprnet.utils.spec_loader: Merging specification from /workspace/openalpr/tutorial_spec.txt
Traceback (most recent call last):
  File "/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/lprnet/scripts/train.py", line 277, in <module>
  File "/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/lprnet/scripts/train.py", line 273, in main
  File "/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/lprnet/scripts/train.py", line 64, in run_experiment
  File "/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/lprnet/utils/spec_loader.py", line 126, in load_experiment_spec
  File "/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/lprnet/utils/spec_loader.py", line 106, in load_proto
  File "/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/lprnet/utils/spec_loader.py", line 92, in _load_from_file
  File "/usr/local/lib/python3.6/dist-packages/google/protobuf/text_format.py", line 735, in Merge
    allow_unknown_field=allow_unknown_field)
  File "/usr/local/lib/python3.6/dist-packages/google/protobuf/text_format.py", line 803, in MergeLines
    return parser.MergeLines(lines, message)
  File "/usr/local/lib/python3.6/dist-packages/google/protobuf/text_format.py", line 828, in MergeLines
    self._ParseOrMerge(lines, message)
  File "/usr/local/lib/python3.6/dist-packages/google/protobuf/text_format.py", line 850, in _ParseOrMerge
    self._MergeField(tokenizer, message)
  File "/usr/local/lib/python3.6/dist-packages/google/protobuf/text_format.py", line 980, in _MergeField
    merger(tokenizer, message, field)
  File "/usr/local/lib/python3.6/dist-packages/google/protobuf/text_format.py", line 1055, in _MergeMessageField
    self._MergeField(tokenizer, sub_message)
  File "/usr/local/lib/python3.6/dist-packages/google/protobuf/text_format.py", line 947, in _MergeField
    (message_descriptor.full_name, name))
google.protobuf.text_format.ParseError: 57:5 : Message type "AugmentationConfig" has no field named "transform_prob".
2021-11-01 17:20:31,245 [INFO] tlt.components.docker_handler.docker_handler: Stopping container.

What will happen if I delete the “transform_prob” parameter.

BTW,i am training “ch_lprnet_baseline18_trainable.tlt”. Is there anything to modify in the spec?

Can you create a new topic? Thanks.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.