Tao toolkit observations

The .etlt is actually encrypted onnx file. Since TAO5.0, the source code is open and the training result will be .onnx instead. Under deepstream environment, you can use trtexec to generate tensorrt engine.
TRTEXEC with LPRNet - NVIDIA Docs. Then only config model-engine-file. Refer to deepstream_lpr_app/deepstream-lpr-app/lpr_config_sgie_us.yml at master · NVIDIA-AI-IOT/deepstream_lpr_app · GitHub.

model-engine-file=models/LP/LPR/us_lprnet_baseline18_deployable.etlt_b16_gpu0_fp16.engine
labelfile-path=models/LP/LPR/labels_us.txt
#tlt-encoded-model=models/LP/LPR/us_lprnet_baseline18_deployable.etlt
#tlt-model-key=nvidia_tlt

The .etlt is actually encrypted onnx file.

Well, yes… Sorry, if you know, you know…

model-engine-file=models/LP/LPR/us_lprnet_baseline18_deployable.etlt_b16_gpu0_fp16.engine

You mean that should now be the model file created during the training?

model-engine-file=models/LP/LPR/lprnet_epoch-024.fp16.engine

And I should comment these two entries in my config?

#tlt-encoded-model=models/LP/LPR/us_lprnet_baseline18_deployable.etlt
#tlt-model-key=nvidia_tlt

I don’t have a trtexec available, but from the documentation it looks like I could run that via the docker. Is that true?

I did find this here https://docs.nvidia.com/tao/tao-toolkit/text/character_recognition/lprnet.html?_gl=1*7mn606*_gcl_au*NzI5Nzk2NTUzLjE3MTEyNTg3Mjk.

tao model lprnet export -m <model>
                  -k <key>
                  -e <experiment_spec>
                  [--gpu_index <gpu_index>]
                  [--log_file <log_file>]
                  [-o <output_file>]
                  [--data_type {fp32,fp16}]
                  [--max_workspace_size <max_workspace_size>]
                  [--max_batch_size <max_batch_size>]
                  [--engine_file <engine_file>]
                  [-v]

but it’s always just producing a *.onnx

The onnx file is also supported in deepstream_lpr_app. Please refer to below.
You can config the onnx file directly.
deepstream_lpr_app/deepstream-lpr-app/lpr_config_pgie.txt at master · NVIDIA-AI-IOT/deepstream_lpr_app · GitHub.

Wow, indeed. It works (Note: It is still the original training configuration, not able to deal with spaces)

Now, this is a good start to add my own training data set and retry… Will be back for more questions, for sure…

Thanks so far, this makes my day

Hmm. Haven’t seen this until now. My reference was this so far

This clearly uses the *.elt. What is the difference to the other config?

And what describes the “num-detected-classes=45” here?

It is not been used since the postprocessing is customized. It can be removed.
I will sync with deepstream team about it.

Thanks. Learned a lot lately

But again: I see the configuration you where quoting was for pgie, mine for sgie. Is that somehow an important difference? And if - what is the difference at all? I never got the magic behind pgie and sgie. I always thought, if an inference is depending on another it is a secondary. So the LPD depends on LPR, I would say, secondary is OK. Confusing…

I will sync with deepstream team to check it further. Thanks for the catching.

Thanks. And I can confirm: It is not possible to train lprnet with [space] or something, which is a pity.

But with the nice experience of lprnet I will retry ocrnet now. What notebook would you suggest? There are two.

For OCRNet, please use vit version. tao_tutorials/notebooks/tao_launcher_starter_kit/ocrnet/ocrnet-vit.ipynb at main · NVIDIA/tao_tutorials · GitHub because it is needed to use attention decode way to retrain the OCRNet model.

Yes, this was the one I already started with in my first attempt.

One thing: While taking over your ONNX configuration I got an error at runtime:

ERROR: [TRT]: 3: Cannot find binding of given name: output_bbox/BiasAdd
0:00:09.626027892 13488      0x1e2b760 WARN                 nvinfer gstnvinfer.cpp:679:gst_nvinfer_logger:<sgie2-lpr> NvDsInferContext[UID 3]: Warning from NvDsInferContextImpl::checkBackendParams() <nvdsinfer_context_impl.cpp:2062> [UID = 3]: Could not find output layer 'output_bbox/BiasAdd' in engine
ERROR: [TRT]: 3: Cannot find binding of given name: output_cov/Sigmoid
0:00:09.626062854 13488      0x1e2b760 WARN                 nvinfer gstnvinfer.cpp:679:gst_nvinfer_logger:<sgie2-lpr> NvDsInferContext[UID 3]: Warning from NvDsInferContextImpl::checkBackendParams() <nvdsinfer_context_impl.cpp:2062> [UID = 3]: Could not find output layer 'output_cov/Sigmoid' in engine
0:00:09.627751012 13488      0x1e2b760 INFO                 nvinfer gstnvinfer.cpp:682:gst_nvinfer_logger:<sgie2-lpr> NvDsInferContext[UID 3]: Info from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2198> [UID = 3]: Use deserialized engine model: /home/ubuntu/vx-ai-golang/models/LP/LPR/lprnet_epoch-024.onnx_b16_gpu0_fp16.engine

What to do with this?

Yes, I think that is a problem for the config. These nodes are not from LPR but from LPD.
Could you please create a topic in deepstream forum for the issues you found in https://github.com/NVIDIA-AI-IOT/deepstream_lpr_app/blob/master/deepstream-lpr-app/lpr_config_pgie.txt? Thanks.

Yepp. Will do

One thing I forgot to mention: If you are running TAO on Ubuntu 22.04 with Python 3.10 (and maybe >) there is a problem with the requests package, which has nothing to do with TAO, but hits TAO when trying to run any docker.

Downgrading requests to 2.31.0 helps. Maybe you can check that with your requirements.

1 Like