# LPRNet gives different results when running on Jetson

**URL:** <https://forums.developer.nvidia.com/t/lprnet-gives-different-results-when-running-on-jetson/285398>\
**Category:** TAO Toolkit\
**Tags:** jetson\
**Created:** [March 8, 2024, 11:21am UTC](https://forums.developer.nvidia.com/t/lprnet-gives-different-results-when-running-on-jetson/285398 "2024-03-08T11:21:25Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![nikola2](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/nikola2/32/306481_2.png) [@nikola2](https://forums.developer.nvidia.com/u/nikola2)\
**Post date:** [March 8, 2024, 11:21am UTC](https://forums.developer.nvidia.com/t/lprnet-gives-different-results-when-running-on-jetson/285398/1 "2024-03-08T11:21:25Z")

</div>

**• Hardware Platform** Jetson  
**• DeepStream Version** 6.3  
**• JetPack Version** 5.1  
**• TensorRT Version** 5.1  
**• Issue Type:** questions

LPRNet I have trained on custom dataset produces different results when running on Jetson.

In my experiment I would process video on Jetson and save crops from object detector, and then run inference in my training environment. In some cases results are the same, while in others model outputs a wrong character. Sometimes it would drop one character or give wrong output for one or two characters.  
For example:  
 ![1200-0](https://global.discourse-cdn.com/nvidia/original/4X/7/e/b/7ebdba283d3fe7fb00c6ac43d47bf1c894d12b5e.jpeg)  
On Jetson I get MLY0005, while on Docker I get correct value of MLY9905

I train LPRNet on [nvcr.io/nvidia/tao/tao-toolkit:5.0.0-tf1.15.5](http://nvcr.io/nvidia/tao/tao-toolkit:5.0.0-tf1.15.5) image, use it to export to onnx and then transfer it to Jetson where it is automatically converted to tlt once I start the app.

What could be the reason for this behavior and how to get my model to give better results?

---

<div class="post-metadata">

**Author:** ![Morganh](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/morganh/32/9748_2.png) [@Morganh](https://forums.developer.nvidia.com/u/Morganh)\
**Post date:** [March 11, 2024, 2:26am UTC](https://forums.developer.nvidia.com/t/lprnet-gives-different-results-when-running-on-jetson/285398/5 "2024-03-11T02:26:44Z")

</div>

> [@nikola2](#):
>
> In some cases results are the same, while in others model outputs a wrong character. Sometimes it would drop one character or give wrong output for one or two characters.

Hi @nikola2  
Do you mean the outputs are fine when run inference in docker [nvcr.io/nvidia/tao/tao-toolkit:5.0.0-tf1.15.5](http://nvcr.io/nvidia/tao/tao-toolkit:5.0.0-tf1.15.5) but sometimes wrong when run inference in Jetson device?

---

<div class="post-metadata">

**Author:** ![nikola2](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/nikola2/32/306481_2.png) [@nikola2](https://forums.developer.nvidia.com/u/nikola2)\
**Post date:** [March 11, 2024, 7:55am UTC](https://forums.developer.nvidia.com/t/lprnet-gives-different-results-when-running-on-jetson/285398/6 "2024-03-11T07:55:28Z")

</div>

> [@Morganh](#):
>
> Do you mean the outputs are fine when run inference in docker [nvcr.io/nvidia/tao/tao-toolkit:5.0.0-tf1.15.5](http://nvcr.io/nvidia/tao/tao-toolkit:5.0.0-tf1.15.5) but sometimes wrong when run inference in Jetson device?

Yes, exactly

---

<div class="post-metadata">

**Author:** ![Morganh](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/morganh/32/9748_2.png) [@Morganh](https://forums.developer.nvidia.com/u/Morganh)\
**Post date:** [March 11, 2024, 7:58am UTC](https://forums.developer.nvidia.com/t/lprnet-gives-different-results-when-running-on-jetson/285398/7 "2024-03-11T07:58:35Z")

</div>

For Jetson, may I know what is the tensorrt version?  
And how did you convert onnx file to tensorrt engine, with trtexec, or tao-converter, or deepstream-tao-apps or your own script?

---

<div class="post-metadata">

**Author:** ![nikola2](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/nikola2/32/306481_2.png) [@nikola2](https://forums.developer.nvidia.com/u/nikola2)\
**Post date:** [March 11, 2024, 8:30am UTC](https://forums.developer.nvidia.com/t/lprnet-gives-different-results-when-running-on-jetson/285398/8 "2024-03-11T08:30:36Z")

</div>

> [@nikola2](#):
>
> **• TensorRT Version** 5.1

Like I said, TensorRT version is 5.1.

I think I used trtexec, I would just write location of onnx and engine file and application would automatically convert it:

```auto
  onnx-file: ../weights/LP/LPR/lprnet_03.05.2024/lprnet_epoch-070.onnx
  model-engine-file: ../weights/LP/LPR/lprnet_03.05.2024/lprnet_epoch-070.onnx_b16_gpu0_fp16.engine

```

For conversion to onnx I used thsi command withing docker:

```auto
lprnet export -m results/weights/lprnet_epoch-070.hdf5 -e experiment.txt -k nvidia_tlt

```

---

<div class="post-metadata">

**Author:** ![Morganh](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/morganh/32/9748_2.png) [@Morganh](https://forums.developer.nvidia.com/u/Morganh)\
**Post date:** [March 11, 2024, 9:32am UTC](https://forums.developer.nvidia.com/t/lprnet-gives-different-results-when-running-on-jetson/285398/9 "2024-03-11T09:32:09Z")

</div>

> [@nikola2](#):
>
> application would automatically convert it:

May I know which application did you use?

---

<div class="post-metadata">

**Author:** ![nikola2](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/nikola2/32/306481_2.png) [@nikola2](https://forums.developer.nvidia.com/u/nikola2)\
**Post date:** [March 11, 2024, 9:40am UTC](https://forums.developer.nvidia.com/t/lprnet-gives-different-results-when-running-on-jetson/285398/10 "2024-03-11T09:40:31Z")

</div>

> [@Morganh](#):
>
> May I know which application did you use?

My own python application I built from deepstream-test3 example.

---

<div class="post-metadata">

**Author:** ![Morganh](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/morganh/32/9748_2.png) [@Morganh](https://forums.developer.nvidia.com/u/Morganh)\
**Post date:** [March 11, 2024, 9:52am UTC](https://forums.developer.nvidia.com/t/lprnet-gives-different-results-when-running-on-jetson/285398/11 "2024-03-11T09:52:16Z")

</div>

To narrow down, suggest you to run trtexec to convert as well.  
Refer to [https://docs.nvidia.com/tao/tao-toolkit/text/trtexec\_integration/trtexec\_lprnet.html](https://docs.nvidia.com/tao/tao-toolkit/text/trtexec_integration/trtexec_lprnet.html).

---

<div class="post-metadata">

**Author:** ![nikola2](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/nikola2/32/306481_2.png) [@nikola2](https://forums.developer.nvidia.com/u/nikola2)\
**Post date:** [March 11, 2024, 11:06am UTC](https://forums.developer.nvidia.com/t/lprnet-gives-different-results-when-running-on-jetson/285398/12 "2024-03-11T11:06:46Z")

</div>

I converted with this command and issue still persists

```auto
/usr/src/tensorrt/bin/trtexec --onnx=weights/LP/LPR/lprnet_03.05.2024/lprnet_epoch-070.onnx --maxShapes=image_input:32x3x48x96 --minShapes=image_input:4x3x48x96 --optShapes=image_input:16x3x48x96 --fp16 --saveEngine=weights/LP/LPR/lprnet_03.05.2024/lprnet_test.engine

```

---

<div class="post-metadata">

**Author:** ![Morganh](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/morganh/32/9748_2.png) [@Morganh](https://forums.developer.nvidia.com/u/Morganh)\
**Post date:** [March 11, 2024, 3:28pm UTC](https://forums.developer.nvidia.com/t/lprnet-gives-different-results-when-running-on-jetson/285398/13 "2024-03-11T15:28:59Z")

</div>

> [@nikola2](#):
>
> TensorRT version is 5.1

The TensorRT version is a little old.  
Could you confirm it? You can run `$dpkg -l |grep cuda` in your Jetson device?

More, could you update to latest Jetpack?

---

<div class="post-metadata">

**Author:** ![nikola2](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/nikola2/32/306481_2.png) [@nikola2](https://forums.developer.nvidia.com/u/nikola2)\
**Post date:** [March 12, 2024, 7:44am UTC](https://forums.developer.nvidia.com/t/lprnet-gives-different-results-when-running-on-jetson/285398/14 "2024-03-12T07:44:49Z")

</div>

Jtop says this is my version, but here’s the full dpkg output:

```auto
ii cuda-11-4 11.4.19-1 arm64 CUDA 11.4 meta-package
ii cuda-cccl-11-4 11.4.298-1 arm64 CUDA CCCL
ii cuda-command-line-tools-11-4 11.4.19-1 arm64 CUDA command-line tools
ii cuda-compiler-11-4 11.4.19-1 arm64 CUDA compiler
ii cuda-cudart-11-4 11.4.298-1 arm64 CUDA Runtime native Libraries
ii cuda-cudart-dev-11-4 11.4.298-1 arm64 CUDA Runtime native dev links, headers
ii cuda-cuobjdump-11-4 11.4.315-1 arm64 CUDA cuobjdump
ii cuda-cupti-11-4 11.4.298-1 arm64 CUDA profiling tools runtime libs.
ii cuda-cupti-dev-11-4 11.4.298-1 arm64 CUDA profiling tools interface.
ii cuda-cuxxfilt-11-4 11.4.300-1 arm64 CUDA cuxxfilt
ii cuda-documentation-11-4 11.4.298-1 arm64 CUDA documentation
ii cuda-driver-dev-11-4 11.4.298-1 arm64 CUDA Driver native dev stub library
ii cuda-gdb-11-4 11.4.298-1 arm64 CUDA-GDB
ii cuda-libraries-11-4 11.4.19-1 arm64 CUDA Libraries 11.4 meta-package
ii cuda-libraries-dev-11-4 11.4.19-1 arm64 CUDA Libraries 11.4 development meta-package
ii cuda-nvcc-11-4 11.4.315-1 arm64 CUDA nvcc
ii cuda-nvdisasm-11-4 11.4.298-1 arm64 CUDA disassembler
ii cuda-nvml-dev-11-4 11.4.298-1 arm64 NVML native dev links, headers
ii cuda-nvprune-11-4 11.4.298-1 arm64 CUDA nvprune
ii cuda-nvrtc-11-4 11.4.300-1 arm64 NVRTC native runtime libraries
ii cuda-nvrtc-dev-11-4 11.4.300-1 arm64 NVRTC native dev links, headers
ii cuda-nvtx-11-4 11.4.300-1 arm64 NVIDIA Tools Extension
ii cuda-profiler-api-11-4 11.4.298-1 arm64 CUDA Profiler API
ii cuda-runtime-11-4 11.4.19-1 arm64 CUDA Runtime 11.4 meta-package
ii cuda-samples-11-4 11.4.300-1 arm64 CUDA example applications
ii cuda-sanitizer-11-4 11.4.298-1 arm64 CUDA Sanitizer
ii cuda-toolkit-11-4 11.4.19-1 arm64 CUDA Toolkit 11.4 meta-package
ii cuda-toolkit-11-4-config-common 11.4.298-1 all Common config package for CUDA Toolkit 11.4.
ii cuda-toolkit-11-config-common 11.4.298-1 all Common config package for CUDA Toolkit 11.
ii cuda-toolkit-config-common 11.4.298-1 all Common config package for CUDA Toolkit.
ii cuda-tools-11-4 11.4.19-1 arm64 CUDA Tools meta-package
ii cuda-visual-tools-11-4 11.4.19-1 arm64 CUDA visual tools
ii graphsurgeon-tf 8.5.2-1+cuda11.4 arm64 GraphSurgeon for TensorRT package
ii libcudnn8 8.6.0.166-1+cuda11.4 arm64 cuDNN runtime libraries
ii libcudnn8-dev 8.6.0.166-1+cuda11.4 arm64 cuDNN development libraries and headers
ii libcudnn8-samples 8.6.0.166-1+cuda11.4 arm64 cuDNN samples
ii libnvinfer-bin 8.5.2-1+cuda11.4 arm64 TensorRT binaries
ii libnvinfer-dev 8.5.2-1+cuda11.4 arm64 TensorRT development libraries and headers
ii libnvinfer-plugin-dev 8.5.2-1+cuda11.4 arm64 TensorRT plugin libraries
ii libnvinfer-plugin8 8.5.2-1+cuda11.4 arm64 TensorRT plugin libraries
ii libnvinfer-samples 8.5.2-1+cuda11.4 all TensorRT samples
ii libnvinfer8 8.5.2-1+cuda11.4 arm64 TensorRT runtime libraries
ii libnvonnxparsers-dev 8.5.2-1+cuda11.4 arm64 TensorRT ONNX libraries
ii libnvonnxparsers8 8.5.2-1+cuda11.4 arm64 TensorRT ONNX libraries
ii libnvparsers-dev 8.5.2-1+cuda11.4 arm64 TensorRT parsers libraries
ii libnvparsers8 8.5.2-1+cuda11.4 arm64 TensorRT parsers libraries
ii nvidia-cuda 5.1-b147 arm64 NVIDIA CUDA Meta Package
ii nvidia-cuda-dev 5.1-b147 arm64 NVIDIA CUDA dev Meta Package
ii nvidia-l4t-cuda 35.2.1-20230124153320 arm64 NVIDIA CUDA Package
ii python3-libnvinfer 8.5.2-1+cuda11.4 arm64 Python 3 bindings for TensorRT
ii python3-libnvinfer-dev 8.5.2-1+cuda11.4 arm64 Python 3 development package for TensorRT
ii tensorrt 8.5.2.2-1+cuda11.4 arm64 Meta package for TensorRT
ii uff-converter-tf 8.5.2-1+cuda11.4 arm64 UFF converter for TensorRT package

```

I’m not sure, how do I update Jetpack?

---

<div class="post-metadata">

**Author:** ![nikola2](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/nikola2/32/306481_2.png) [@nikola2](https://forums.developer.nvidia.com/u/nikola2)\
**Post date:** [March 12, 2024, 12:31pm UTC](https://forums.developer.nvidia.com/t/lprnet-gives-different-results-when-running-on-jetson/285398/15 "2024-03-12T12:31:34Z")

</div>

I think I have newest Jetpack installed.

> sudo apt-get upgrade nvidia-jetpack
> 
> Reading package lists… Done  
> Building dependency tree  
> Reading state information… Done  
> nvidia-jetpack is already the newest version (5.1-b147).

---

<div class="post-metadata">

**Author:** ![Morganh](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/morganh/32/9748_2.png) [@Morganh](https://forums.developer.nvidia.com/u/Morganh)\
**Post date:** [March 13, 2024, 5:03am UTC](https://forums.developer.nvidia.com/t/lprnet-gives-different-results-when-running-on-jetson/285398/16 "2024-03-13T05:03:19Z")

</div>

> [@nikola2](#):
>
> I train LPRNet on [nvcr.io/nvidia/tao/tao-toolkit:5.0.0-tf1.15.5](http://nvcr.io/nvidia/tao/tao-toolkit:5.0.0-tf1.15.5) image, use it to export to onnx

To narrow down, can you generate a tensorrt engine in tao-deploy docker instead.  
[nvcr.io/nvidia/tao/tao-toolkit:5.2.0-deploy](http://nvcr.io/nvidia/tao/tao-toolkit:5.2.0-deploy)  
Please run the docker in a dgpu machine instead of Jetson device.

You can use `tao deploy lprnet gen_trt_engine` command mentioned in [tao\_tutorials/notebooks/tao\_launcher\_starter\_kit/lprnet/lprnet.ipynb at main · NVIDIA/tao\_tutorials · GitHub](https://github.com/NVIDIA/tao_tutorials/blob/main/notebooks/tao_launcher_starter_kit/lprnet/lprnet.ipynb).  
Or `docker run --runtime=nvidia -it --rm nvcr.io/nvidia/tao/tao-toolkit:5.2.0-deploy /bin/bash`, and then generate engine in it. And run inference inside the docker.

---

<div class="post-metadata">

**Author:** ![nikola2](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/nikola2/32/306481_2.png) [@nikola2](https://forums.developer.nvidia.com/u/nikola2)\
**Post date:** [March 13, 2024, 9:05am UTC](https://forums.developer.nvidia.com/t/lprnet-gives-different-results-when-running-on-jetson/285398/17 "2024-03-13T09:05:41Z")

</div>

> [@Morganh](#):
>
> To narrow down, can you generate a tensorrt engine in tao-deploy docker instead.  
> [nvcr.io/nvidia/tao/tao-toolkit:5.2.0-deploy](http://nvcr.io/nvidia/tao/tao-toolkit:5.2.0-deploy)

Since I’m using training on a shared server I will make a request for their team to prepare tao-deploy, but this might take some time.  
Is there something else I can try while waiting?

Also, during conversion to tensorrt engine I get this warning:

> Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.

---

<div class="post-metadata">

**Author:** ![Morganh](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/morganh/32/9748_2.png) [@Morganh](https://forums.developer.nvidia.com/u/Morganh)\
**Post date:** [March 13, 2024, 9:11am UTC](https://forums.developer.nvidia.com/t/lprnet-gives-different-results-when-running-on-jetson/285398/18 "2024-03-13T09:11:28Z")

</div>

> [@nikola2](#):
>
> `/usr/src/tensorrt/bin/trtexec --onnx=weights/LP/LPR/lprnet_03.05.2024/lprnet_epoch-070.onnx --maxShapes=image_input:32x3x48x96 --minShapes=image_input:4x3x48x96 --optShapes=image_input:16x3x48x96 --fp16 --saveEngine=weights/LP/LPR/lprnet_03.05.2024/lprnet_test.engine`

Could you try more in Jetson devices?  
experiment1: generate fp32 engine  
experiment2: generate engine with `--maxShapes=image_input:1x3x48x96 --minShapes=image_input:1x3x48x96 --optShapes=image_input:1x3x48x96`

---

<div class="post-metadata">

**Author:** ![nikola2](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/nikola2/32/306481_2.png) [@nikola2](https://forums.developer.nvidia.com/u/nikola2)\
**Post date:** [March 13, 2024, 10:22am UTC](https://forums.developer.nvidia.com/t/lprnet-gives-different-results-when-running-on-jetson/285398/19 "2024-03-13T10:22:37Z")

</div>

Just to confirm I generate fp32 engine just by dropping --fp16 flag?  
If that’s the case, I did the following experiments:

1. I generated fp32 engine - no change, model makes the same mistakes
2. I generated fp32 engine with max/min/opt shapes as suggested - no change, model makes the same mistakes
3. I generated fp16 engine with max/min/opt shapes as suggested - no change, model makes the same mistakes

---

<div class="post-metadata">

**Author:** ![Morganh](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/morganh/32/9748_2.png) [@Morganh](https://forums.developer.nvidia.com/u/Morganh)\
**Post date:** [March 13, 2024, 2:18pm UTC](https://forums.developer.nvidia.com/t/lprnet-gives-different-results-when-running-on-jetson/285398/20 "2024-03-13T14:18:20Z")

</div>

May I know how did you run inference in Jetson devices?

Please try to use [GitHub - NVIDIA-AI-IOT/deepstream\_lpr\_app: Sample app code for LPR deployment on DeepStream](https://github.com/NVIDIA-AI-IOT/deepstream_lpr_app).

---

<div class="post-metadata">

**Author:** ![nikola2](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/nikola2/32/306481_2.png) [@nikola2](https://forums.developer.nvidia.com/u/nikola2)\
**Post date:** [March 14, 2024, 9:38am UTC](https://forums.developer.nvidia.com/t/lprnet-gives-different-results-when-running-on-jetson/285398/21 "2024-03-14T09:38:31Z")

</div>

I have video of still images I feed to my deepstream application, with YoloV5 as my primary detector and LPRNet as secondary classifier. After inference, crops are saved in a folder. These crops I later upload to server where I can do inference with LPRNet.

---

<div class="post-metadata">

**Author:** ![Morganh](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/morganh/32/9748_2.png) [@Morganh](https://forums.developer.nvidia.com/u/Morganh)\
**Post date:** [March 15, 2024, 2:33am UTC](https://forums.developer.nvidia.com/t/lprnet-gives-different-results-when-running-on-jetson/285398/22 "2024-03-15T02:33:53Z")

</div>

> [@nikola2](#):
>
> These crops I later upload to server where I can do inference with LPRNet.

I am afraid there is something mismatching in preprocessing.

To narrow down, suggest you to install tao-deploy in Jetson device.

```auto
Please run with below steps. The tao-deploy can work on Jetson(For example, Orin).

docker run --runtime=nvidia -it --rm -v nvcr.io/nvidia/l4t-tensorrt:r8.5.2.2-devel /bin/bash                                                      

Then, inside the docker,

$apt-get update
$apt install libopenmpi-dev
$pip install nvidia_tao_deploy==5.0.0.423.dev0
$pip install https://urm.nvidia.com/artifactory/sw-eff-pypi/nvidia-eff-tao-encryption/0.1.7/nvidia_eff_tao_encryption-0.1.7-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
$pip install https://urm.nvidia.com/artifactory/sw-eff-pypi/nvidia-eff/0.6.2/nvidia_eff-0.6.2-py38-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl

$lprnet --help 

```

BTW, the source code can be found in [tao\_deploy/nvidia\_tao\_deploy/cv/lprnet/scripts/inference.py at main · NVIDIA/tao\_deploy · GitHub](https://github.com/NVIDIA/tao_deploy/blob/main/nvidia_tao_deploy/cv/lprnet/scripts/inference.py).

---

<div class="post-metadata">

**Author:** ![nikola2](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/nikola2/32/306481_2.png) [@nikola2](https://forums.developer.nvidia.com/u/nikola2)\
**Post date:** [March 15, 2024, 9:30am UTC](https://forums.developer.nvidia.com/t/lprnet-gives-different-results-when-running-on-jetson/285398/23 "2024-03-15T09:30:41Z")

</div>

I have tried running docker but I keep getting

> docker: invalid reference format.  
> See ‘docker run --help’.

I don’t understand what I’m missing.

[Next page](https://forums.developer.nvidia.com/t/lprnet-gives-different-results-when-running-on-jetson/285398.md?page=2)