How preform inference retinanet using a TLT export .engine file by python

[zhobin8]

I use TLT V2.0 training retinanet resnet 50 model, and export .engine file.
I don’t need deepstream .
How preform inference by python?
I only search SSD model inference sample and very old sample.

[Morganh]

Officially, TLT provides tlt-infer for inference. They can run inference against tlt model.
Some detection networks also provide the command how to run inference against trt engine. Such as , detectnet_v2, faster_rcnn, etc. See tlt user guide or jupyter notebooks for more details.

If end user wants to run inference against trt engine without tlt-infer or deepstream, they need to write their own codes.

Reference topics:
For classification network,

Inferring resnet18 classification etlt model with python

I have an .etlt model and a corresponding .trt classification model which i trained with nvidia tlt, from resnet18 pretrained model. How can i infer it in python? Any suggestion would be appreciated… Thank you.

For detectnet_v2 network,

Run PeopleNet with tensorrt

Hey everyone. After looking into a lot of places, blogs and repos I could manage to run a .engine or .trt file for Detectnet_v2 with preprocessing and proper postprocessing. Here is my working code, hope it helps future persons: import os import time import cv2 import matplotlib.pyplot as plt import numpy as np import pycuda.autoinit import pycuda.driver as cuda import tensorrt as trt from PIL import Image class HostDeviceMem(object): def __init__(self, host_mem, device_mem): se…

Reference post-processing:

[zhobin8]

How run tlt-infer in the jetson nx ?

[Morganh]

Reference: https://docs.nvidia.com/metropolis/TLT/archive/tlt-20/tlt-user-guide/text/deploying_to_deepstream.html#id11

Please copy etlt model into Jetson NX, then config the file with deepstream and then run inference with deepstream.

Or
Copy etlt model into Jetson NX, then use tlt-converter to generate trt engine, config the file with deepstream, and then run inference with deepstream.