Custom LPRnet model

Hi,

I have developed a custom LPRnet for Arabic license plates since the example has only US and Chinese plates.

I used this repo for model training GitHub - mesakarghm/LPRNET: CNN based Automatic License Plate Recognition Network

I made the training process in a collab notebook.

There’s a problem with the deployment using deepstream 6.0 and jetson nano board in that the output layer in deepstream is different from the output layer in the jupyter notebook for the same input.

There is a similar shape, but different values in both, which causes a wrong output in deepstream.

I have edited the parser for CTC greedy decoding and tested it; I believe it works fine and the problem is in the config files, I have tried changing many config parameters but I couldn’t solve the problem.

here are the configs I have used:
https://drive.google.com/drive/folders/1Z9lnrWQZiX3baWtUMETfMyrOGliJP4Qx?usp=sharing

Sorry for the long post, and thanks in advanced

Config_file:

[property]

#net-scale-factor=0.0039215697906911373
model-color-format=0
network-type=1
operate-on-gie-id=1
operate-on-class-ids=0

gpu-id=0

#0=RGB, 1=BGR
model-color-format=0

batch-size=1

onnx-file= model8_dynamic_en.onnx
model-engine-file=model8_dynamic_en.onnx_b1_gpu0_fp16.engine
labelfile-path=dict.txt

0=FP32, 1=INT8, 2=FP16 mode

network-mode=1
process-mode=2
operate-on-gie-id=1
gie-unique-id=2

is-classifier=1

output-blob-names=sequential_20/re_lu_15/Relu:0
parse-classifier-func-name = NvDsInferParseCustomNVPlate
custom-lib-path=/home/amr/Workspace/yoloV4/ocr_test4/nvinfer_custom_lpr_parser/libnvdsinfer_custom_impl_lpr.so

scaling-compute-hw=1

input-tensor-from-meta=1

deepsteam_LPRnet:

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5

[tiled-display]
enable=0
rows=1
columns=1
width=1280
height=720
gpu-id=0
#(0): nvbuf-mem-default - Default memory allocated, specific to particular platform

(1): nvbuf-mem-cuda-pinned - Allocate Pinned/Host cuda memory, applicable for Tesla

(2): nvbuf-mem-cuda-device - Allocate Device cuda memory, applicable for Tesla

(3): nvbuf-mem-cuda-unified - Allocate Unified cuda memory, applicable for Tesla

(4): nvbuf-mem-surface-array - Allocate Surface Array memory, applicable for Jetson

nvbuf-memory-type=0

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=2

uri=file:///home/amr/Workspace/yoloV4/ocr_test4/5.mp4

num-sources=1
gpu-id=0

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

cudadec-memtype=0

[osd]
enable=1
gpu-id=0
border-width=1
text-size=10
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=100
clock-y-offset=100
clock-text-size=10
clock-color=1;0;0;0
nvbuf-memory-type=0

[streammux]
gpu-id=0
#Boolean property to inform muxer that sources are live
live-source=0
batch-size=1
#time out in usec, to wait after the first buffer is available
#to push the batch even if the complete batch is not formed
batched-push-timeout=40000

Set muxer output width and height

width=1080
height=720
#Enable to maintain aspect ratio wrt source, and allow black borders, works
#along with width, height properties
enable-padding=0
nvbuf-memory-type=0

config-file property is mandatory for any gie section.

Other properties are optional and if set will override the properties set in

the infer config file.

[primary-gie]
enable=1
gpu-id=0
#model-engine-file=model_b1_int8.engine
labelfile-path=dict_us.txt
batch-size=1
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
#interval=0
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_file.txt

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=3
sync=0
source-id=0
gpu-id=0
container=1
codec=1
output-file=re2.mp4
nvbuf-memory-type=0

[tests]
file-loop=0

Hi,

There is an LPRNet in our TAO toolkit as well.
Do you mind using the TAO version instead?

The model can be deployed with Deepstream without issue.
Thanks.


I want to use the model for a different country and this is one of the limitations of the TAO LPRNet

and they haven’t clarify how to retrain the detection and recognition model with different dataset

plus, I want to know why the output layer values changed, this is very important to me because I faced this problem in many models.
understanding this problem and solving it is very important to me, can you please help me to figure it out?

Hi,

Please check if you use the same input (preprocessing) between deepstream and your training frameworks first.
You can find some information below:

https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_plugin_gst-nvinfer.html

Thanks.

I don’t have any input preprocessing, I just resize the image, without scaling and without shifting

so, I added these two lines:

offsets = 0;0;0
net-scale-factor=1.0

according to this equation y = net scale factor*(x-mean)
but, that doesn’t change anything

are there any other parameters that affect the input value?

Hi,

For GitHub, the script read the image with OpenCV which should be in BGR format.
Please set the color format in Deepstream into the BGR as well.
https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_plugin_gst-nvinfer.html#gst-nvinfer-file-configuration-specifications

[property]
...
model-color-format=1
...

Thanks.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.