How can I perform inference using a TLT output detectnet_v2 .trt model with in custom tensorflow and python

sk.ahmed401 · November 30, 2020, 12:59pm

I have a detectnet_v2 model which is developed using TLT. I would like to know about the steps involved in performing inference on python in DGPU, and tensorflow.

Morganh · November 30, 2020, 3:56pm

Reference:

github.com

NVIDIA/object-detection-tensorrt-example/blob/master/SSD_Model/utils/inference.py

# Utility functions for performing image inference
#
# Copyright 1993-2019 NVIDIA Corporation.  All rights reserved.
#
# NOTICE TO LICENSEE:
#
# This source code and/or documentation ("Licensed Deliverables") are
# subject to NVIDIA intellectual property rights under U.S. and
# international Copyright laws.
#
# These Licensed Deliverables contained herein is PROPRIETARY and
# CONFIDENTIAL to NVIDIA and is being provided under the terms and
# conditions of a form of NVIDIA software license agreement by and
# between NVIDIA and Licensee ("License Agreement") or electronically
# accepted by Licensee.  Notwithstanding any terms or conditions to
# the contrary in the License Agreement, reproduction or disclosure
# of the Licensed Deliverables to any third party without the express
# written consent of NVIDIA is prohibited.
#
# NOTWITHSTANDING ANY TERMS OR CONDITIONS TO THE CONTRARY IN THE

This file has been truncated. show original

github.com

dmikushin/detectrt/blob/master/src/utils/inference.py

# Utility functions for performing image inference
#
# Copyright 1993-2019 NVIDIA Corporation.  All rights reserved.
#
# NOTICE TO LICENSEE:
#
# This source code and/or documentation ("Licensed Deliverables") are
# subject to NVIDIA intellectual property rights under U.S. and
# international Copyright laws.
#
# These Licensed Deliverables contained herein is PROPRIETARY and
# CONFIDENTIAL to NVIDIA and is being provided under the terms and
# conditions of a form of NVIDIA software license agreement by and
# between NVIDIA and Licensee ("License Agreement") or electronically
# accepted by Licensee.  Notwithstanding any terms or conditions to
# the contrary in the License Agreement, reproduction or disclosure
# of the Licensed Deliverables to any third party without the express
# written consent of NVIDIA is prohibited.
#
# NOTWITHSTANDING ANY TERMS OR CONDITIONS TO THE CONTRARY IN THE

This file has been truncated. show original

https://forums.developer.nvidia.com/t/peoplenet-coverage-output-is-always-zero/160222/14

sk.ahmed401 · December 1, 2020, 2:32pm

I have cloned the repo and pulled the tensorrt container.
When I try to run the sample SSD model, I am getting below error.

root@1c120ecdecac:/mnt# cd SSD_Model
root@1c120ecdecac:/mnt/SSD_Model# python detect_objects_webcam.py
2020-12-01 14:28:46.867465: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
Traceback (most recent call last):
File “detect_objects_webcam.py”, line 12, in
import utils.inference as inference_utils # TRT/TF inference wrappers
File “/mnt/SSD_Model/utils/inference.py”, line 60, in
import utils.engine as engine_utils # TRT Engine creation/save/load utils
File “/mnt/SSD_Model/utils/engine.py”, line 11, in
from utils.model import ModelData
File “/mnt/SSD_Model/utils/model.py”, line 9, in
import graphsurgeon as gs
File “/usr/lib/python3.5/dist-packages/graphsurgeon/init.py”, line 9, in
from graphsurgeon.StaticGraph import *
File “/usr/lib/python3.5/dist-packages/graphsurgeon/StaticGraph.py”, line 7, in
from graphsurgeon._utils import _regex_list_contains_string, _generate_iterable_for_search, _clean_input_name
File “/usr/lib/python3.5/dist-packages/graphsurgeon/_utils.py”, line 2, in
from tensorflow import NodeDef
ImportError: cannot import name ‘NodeDef’

And it would be helpful if you can give different source, where I can get information about loading a trt model using tf-trt or tensorrt.

Morganh · December 7, 2020, 2:48am

Refer to Apart from Deepstream where else I can deploy tlt-converted models or .trt engine files

Topic		Replies	Views
How to use tlt trained model on Jetson Nano TAO Toolkit tensorrt , jetson-inference	7	2089	October 12, 2021
TensorRT Inference form a .etlt model on Python TAO Toolkit tensorrt	7	1226	November 16, 2021
Tensorrt engine file generated by TLT is not acceptable to inference server TensorRT	3	628	August 16, 2020
How to use TensorRT engine obtained using tlt-convertor TAO Toolkit	4	656	October 12, 2021
Inferencing of LPDNET Model of Nvidia TLT Triton Inference Server - archived	0	596	March 6, 2021
TensorRT Inference Server rejecting valid trt.engine file generated by TLT Triton Inference Server - archived	0	690	August 16, 2020
TensorRT deployment with engine generated from TLT example TensorRT tensorrt	8	776	December 5, 2020
Running nvidia pretrained models in Tensorrt inference TAO Toolkit	14	916	October 6, 2022
Object Detection with TF-TRT TensorRT tensorrt	2	908	September 21, 2021
Examples for porting from Tensorflow to TensorRT4 object detection inference TensorRT	4	2457	April 26, 2018

How can I perform inference using a TLT output detectnet_v2 .trt model with in custom tensorflow and python

Related topics