Where are sample codes to use exported *.tlt in c++

moche.chan · August 27, 2021, 7:58am

Please provide the following information when requesting support.

• Hardware:Nano
• Network Type:LPRnet
• TLT Version : tao 3.0
• Training spec file(If have, please share here)
• How to reproduce the issue ? (This is for errors. Please share the command line and the detailed log here.)

After studying tao examples, I can obtain some weight files *.tlt. But I still have no idea to use these weights files to infer. How to use the *.tlt files. Are there some cpp examples to use the weight files?

For example, the command exports a model file for lprnet.

tao lprnet export -m /workspace/lprnet/weights/lprnet_epoch-24.tlt -k nvidia_tlt -e /workspace/lprnet_spec.txt

Where can I find sample codes in c++ to use the exported weights to infer?

github.com

NVIDIA-AI-IOT/deepstream_lpr_app/blob/master/deepstream-lpr-app/deepstream_lpr_app.c

/*
 * Copyright (c) 2020-2022, NVIDIA CORPORATION. All rights reserved.
 *
 * Permission is hereby granted, free of charge, to any person obtaining a
 * copy of this software and associated documentation files (the "Software"),
 * to deal in the Software without restriction, including without limitation
 * the rights to use, copy, modify, merge, publish, distribute, sublicense,
 * and/or sell copies of the Software, and to permit persons to whom the
 * Software is furnished to do so, subject to the following conditions:
 *
 * The above copyright notice and this permission notice shall be included in
 * all copies or substantial portions of the Software.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
 * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
 * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
 * DEALINGS IN THE SOFTWARE.

This file has been truncated. show original

The above sample code includes deepstream with video input/output. It’s too complex. Are there any c++ sample code with just a image and a corresponding output.

Morganh · August 27, 2021, 3:28pm

For .tlt model , you run run inference with tao inference inside the docker.
You can also export .tlt model to .etlt model, then deploy the .etlt model with deepstream.

More, you can also generate trt engine for inference. For LPRnet, just like the link you mention above, run the application to do inference.

For standalone inference with LPRnet trt engine, refer to Python run LPRNet with TensorRT show pycuda._driver.MemoryError: cuMemHostAlloc failed: out of memory - #8 by Morganh or Not Getting Correct output while running inference using TensorRT on LPRnet fp16 Model

moche.chan · August 30, 2021, 2:14am

@Morganh

Is the python code the only method for standalone inference with LPRnet trt engine?
Because we want to integrate LPRnet with our existing c++ codes in Jetson nano, the sample codes in c++ are prefered.

Morganh · August 30, 2021, 6:32am

No, c++ can be the method for sure. You can leverage GitHub - NVIDIA-AI-IOT/deepstream_lpr_app: Sample app code for LPR deployment on DeepStream .

moche.chan · August 30, 2021, 7:17am

Thank you. I’ll study the deepstream code to find out how to infer one single picture rather than a video stream.

Topic		Replies	Views
TensorRT C++ API to deploy tlt model TAO Toolkit tensorrt	1	814	April 12, 2021
How to do inference in own code? TAO Toolkit	7	1091	May 26, 2020
LPRNet Inference TAO Toolkit jetson	2	93	October 11, 2024
Using TLT trained models in TF (or Keras) to run inference TAO Toolkit	8	1565	June 22, 2021
Tlt yolov3 tensorrt c++ inference TAO Toolkit	1	603	August 13, 2021
Falure to do inference TAO Toolkit tensorrt	8	1244	December 28, 2021
Use etlt or trt weights without deep stream TAO Toolkit	5	1002	March 6, 2020
TLT trafficnet3.0 ，TRT C++ inference TAO Toolkit	3	818	May 24, 2021
How preform inference retinanet using a TLT export .engine file by python TAO Toolkit tensorrt	3	993	February 23, 2021
Doing tlt inference only with tensorrt TAO Toolkit	2	1040	July 30, 2021

Where are sample codes to use exported *.tlt in c++

Related topics