# TensorRT Inference form a .etlt model on Python **URL:** \ **Category:** TAO Toolkit\ **Tags:** tensorrt\ **Created:** [October 26, 2021, 12:46am UTC](https://forums.developer.nvidia.com/t/tensorrt-inference-form-a-etlt-model-on-python/193011 "2021-10-26T00:46:54Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Andres\_Godoy\_Vanguard](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/andres_godoy_vanguard/32/124263_2.png) [@Andres\_Godoy\_Vanguard](https://forums.developer.nvidia.com/u/Andres_Godoy_Vanguard)\ **Post date:** [October 26, 2021, 12:46am UTC](https://forums.developer.nvidia.com/t/tensorrt-inference-form-a-etlt-model-on-python/193011/1 "2021-10-26T00:46:55Z")
## Description Hi, I followed the example on [GitHub - NVIDIA/object-detection-tensorrt-example: Running object detection on a webcam feed using TensorRT on NVIDIA GPUs in Python.](https://github.com/NVIDIA/object-detection-tensorrt-example), using a .engine finetuned from the `[Nvidia's Catalog lpdNet](https://ngc.nvidia.com/catalog/models/nvidia:tao:lpdnet)`. When inference is done there are two arrays as output one of length 4800 and the other 1200 which make no sense on the desired output, namely the bounding box coordinates. Using the mapping dictionary (Model Layout) the variables are still nonsense. I’ve tried to use every SDK from Nvidia → TAO, DeepStream, TensorRT and the documentation is impossible! One spends more time solving issues on relatively easy tasks than developing applications, all for trying to use your hardware! If I train a model on TAO, why is it so difficult to infer from it on python! ## Environment **TensorRT Version** : 7.1.3 **CUDA Version** : 10.2 **CUDNN Version** : 8.0 **Operating System + Version** : Jtpack 4.5.1 **Python Version (if applicable)**: 3.8 ---
**Author:** ![NVES](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/nves/32/14043_2.png) [@NVES](https://forums.developer.nvidia.com/u/NVES)\ **Post date:** [October 26, 2021, 6:39am UTC](https://forums.developer.nvidia.com/t/tensorrt-inference-form-a-etlt-model-on-python/193011/2 "2021-10-26T06:39:30Z")
Hi, Can you try running your model with trtexec command, and share the “”–verbose"" log in case if the issue persist [https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/trtexec](https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/trtexec) You can refer below link for all the supported operators list, in case any operator is not supported you need to create a custom plugin to support that operation > Also, request you to share your model and script if not shared already so that we can help you better. Meanwhile, for some common errors and queries please refer to below link: > **[Developer Guide :: NVIDIA Deep Learning TensorRT Documentation](https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/#error-messaging)** > > This is the revision history of the NVIDIA TensorRT 8.4 Developer Guide. > **[Developer Guide :: NVIDIA Deep Learning TensorRT Documentation](https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/#faq)** > > This is the revision history of the NVIDIA TensorRT 8.4 Developer Guide. Thanks! ---
**Author:** ![Andres\_Godoy\_Vanguard](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/andres_godoy_vanguard/32/124263_2.png) [@Andres\_Godoy\_Vanguard](https://forums.developer.nvidia.com/u/Andres_Godoy_Vanguard)\ **Post date:** [October 26, 2021, 2:05pm UTC](https://forums.developer.nvidia.com/t/tensorrt-inference-form-a-etlt-model-on-python/193011/4 "2021-10-26T14:05:08Z")
Hi, Thanks for the response… The link you provided for trtexec is broken (404) Operators in the model are okay because the model may be used with deepstream (I can’t use deepstream this time for this project, not a solution right now) Attached is the project, just run the jupyter notebook and it should work… the outputs of length 4800 and 1200 are highlighted under the function `len()` afterward in the repo I linked in the initial post, they map those results with a model layout which I don’t know. Recap that the model is in the Nvidia’s Catalog as lpdNet. > **[tensorRT.zip](https://drive.google.com/file/d/1rxuFCj2sUb2_iFK3nBTApxszTa-7sodt/view?usp=sharing)** > > Google Drive file. ---
**Author:** ![spolisetty](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/spolisetty/32/14047_2.png) [@spolisetty](https://forums.developer.nvidia.com/u/spolisetty)\ **Post date:** [October 27, 2021, 5:04pm UTC](https://forums.developer.nvidia.com/t/tensorrt-inference-form-a-etlt-model-on-python/193011/5 "2021-10-27T17:04:55Z")
Hi, Could you please confirm if you’re facing this error on using TAO(.etlt) or using only TensorRT. Thank you. ---
**Author:** ![Andres\_Godoy\_Vanguard](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/andres_godoy_vanguard/32/124263_2.png) [@Andres\_Godoy\_Vanguard](https://forums.developer.nvidia.com/u/Andres_Godoy_Vanguard)\ **Post date:** [October 28, 2021, 12:23am UTC](https://forums.developer.nvidia.com/t/tensorrt-inference-form-a-etlt-model-on-python/193011/6 "2021-10-28T00:23:56Z")
I re-trained the lpdNet on TAO and got the .etlt model, then made the conversion (tao-converter) into a .engine file. That .engine is the one I want to run using python. If there is a way of running inference from the .etlt on python solves the issue as well. I hope am clear Here. What I need is to reproduce the notebook on the .zip file. and get bounding box coordinates. ---
**Author:** ![spolisetty](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/spolisetty/32/14047_2.png) [@spolisetty](https://forums.developer.nvidia.com/u/spolisetty)\ **Post date:** [October 28, 2021, 4:29am UTC](https://forums.developer.nvidia.com/t/tensorrt-inference-form-a-etlt-model-on-python/193011/7 "2021-10-28T04:29:58Z")
Hi, This looks like TAO related. Moving to TAO forum to get better help. ---
**Author:** ![Morganh](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/morganh/32/9748_2.png) [@Morganh](https://forums.developer.nvidia.com/u/Morganh)\ **Post date:** [October 28, 2021, 6:06am UTC](https://forums.developer.nvidia.com/t/tensorrt-inference-form-a-etlt-model-on-python/193011/9 "2021-10-28T06:06:36Z")
It is possible to run inference with tensorrt engine generated by LPD model. [https://forums.developer.nvidia.com/t/issue-in-lpd-while-running-with-custom-python-script/191131/2](https://forums.developer.nvidia.com/t/issue-in-lpd-while-running-with-custom-python-script/191131/2) Officially, please see [Integrating TAO CV Models with Triton Inference Server — TAO Toolkit 3.0 documentation](https://docs.nvidia.com/tao/tao-toolkit/text/tao_cv_triton_inf_server/index.html) and then leverage [tao-toolkit-triton-apps/detectnet\_processor.py at main · NVIDIA-AI-IOT/tao-toolkit-triton-apps · GitHub](https://github.com/NVIDIA-AI-IOT/tao-toolkit-triton-apps/blob/main/tao_triton/python/postprocessing/detectnet_processor.py) and [tao-toolkit-triton-apps/utils.py at fc7e222c036354498e53a8ed11b5cf7c0a3e5239 · NVIDIA-AI-IOT/tao-toolkit-triton-apps · GitHub](https://github.com/NVIDIA-AI-IOT/tao-toolkit-triton-apps/blob/fc7e222c036354498e53a8ed11b5cf7c0a3e5239/tao_triton/python/postprocessing/utils.py#L49) Also, there are similar topics shared by other customers in TAO fourm. > [@](#): > > Hey everyone. After looking into a lot of places, blogs and repos I could manage to run a .engine or .trt file for Detectnet\_v2 with preprocessing and proper postprocessing. Here is my working code, hope it helps future persons: import os import time import cv2 import matplotlib.pyplot as plt import numpy as np import pycuda.autoinit import pycuda.driver as cuda import tensorrt as trt from PIL import Image class HostDeviceMem(object): def **init** (self, host\_mem, device\_mem): se… > [@](#): > > Hello, I am using Jetson Nano with Jetpack 4.4(recommended in forum) and I want to use Nvidia’s purpose built model Peoplenet in my own application (not deepstream). For clarity, I downloaded the peoplenet’s pruned model from ngc container [peoplenet pruned model](https://ngc.nvidia.com/catalog/models/nvidia:tlt_peoplenet) and used TLT converter downloaded for Jetson and converted the etlt file to TRT engine successfully. I am able to use this generated engine file in deepstream-app, but I don’t find any references how to use it in my custom application. … ---
**Author:** ![system](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/system/32/68080_2.png) [@system](https://forums.developer.nvidia.com/u/system)\ **Post date:** [November 16, 2021, 2:17am UTC](https://forums.developer.nvidia.com/t/tensorrt-inference-form-a-etlt-model-on-python/193011/11 "2021-11-16T02:17:37Z")
Closed on November 16, 2021, 2:17am UTC This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.