TensorRT Inference form a .etlt model on Python

Description

Hi, I followed the example on GitHub - NVIDIA/object-detection-tensorrt-example: Running object detection on a webcam feed using TensorRT on NVIDIA GPUs in Python., using a .engine finetuned from the [Nvidia's Catalog lpdNet](https://ngc.nvidia.com/catalog/models/nvidia:tao:lpdnet). When inference is done there are two arrays as output one of length 4800 and the other 1200 which make no sense on the desired output, namely the bounding box coordinates. Using the mapping dictionary (Model Layout) the variables are still nonsense.

I’ve tried to use every SDK from Nvidia → TAO, DeepStream, TensorRT and the documentation is impossible! One spends more time solving issues on relatively easy tasks than developing applications, all for trying to use your hardware!

If I train a model on TAO, why is it so difficult to infer from it on python!

Environment

TensorRT Version: 7.1.3
CUDA Version: 10.2
CUDNN Version: 8.0
Operating System + Version: Jtpack 4.5.1
Python Version (if applicable): 3.8

Hi,
Can you try running your model with trtexec command, and share the “”–verbose"" log in case if the issue persist
https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/trtexec

You can refer below link for all the supported operators list, in case any operator is not supported you need to create a custom plugin to support that operation

Also, request you to share your model and script if not shared already so that we can help you better.

Meanwhile, for some common errors and queries please refer to below link:

Thanks!

Hi, Thanks for the response…

The link you provided for trtexec is broken (404)

Operators in the model are okay because the model may be used with deepstream (I can’t use deepstream this time for this project, not a solution right now)

Attached is the project, just run the jupyter notebook and it should work… the outputs of length 4800 and 1200 are highlighted under the function len() afterward in the repo I linked in the initial post, they map those results with a model layout which I don’t know. Recap that the model is in the Nvidia’s Catalog as lpdNet.

Hi,

Could you please confirm if you’re facing this error on using TAO(.etlt) or using only TensorRT.

Thank you.

I re-trained the lpdNet on TAO and got the .etlt model, then made the conversion (tao-converter) into a .engine file.

That .engine is the one I want to run using python. If there is a way of running inference from the .etlt on python solves the issue as well.

I hope am clear Here.

What I need is to reproduce the notebook on the .zip file. and get bounding box coordinates.

Hi,

This looks like TAO related. Moving to TAO forum to get better help.

It is possible to run inference with tensorrt engine generated by LPD model.

https://forums.developer.nvidia.com/t/issue-in-lpd-while-running-with-custom-python-script/191131/2

Officially, please see Integrating TAO CV Models with Triton Inference Server — TAO Toolkit 3.0 documentation and then leverage tao-toolkit-triton-apps/detectnet_processor.py at main · NVIDIA-AI-IOT/tao-toolkit-triton-apps · GitHub and tao-toolkit-triton-apps/utils.py at fc7e222c036354498e53a8ed11b5cf7c0a3e5239 · NVIDIA-AI-IOT/tao-toolkit-triton-apps · GitHub

Also, there are similar topics shared by other customers in TAO fourm.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.