Gain insight into TensorRT Engine

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

A clear and concise description of the bug or issue.

I want to gain more insight into a tensorrt engine, and it seems that creating an EngineInspector would do the trick. I load the model using runtime, and do engine.create_execution_context() but I would also like to do engine.create_engine_inspector() which should work, since the inspector works on ICudaEngine, according to ICudaEngine — NVIDIA TensorRT Standard Python API Documentation 8.4.3 documentation. But this doesn’t work, throws this error, inspector = engine.create_engine_inspector()
AttributeError: ‘tensorrt.tensorrt.ICudaEngine’ object has no attribute ‘create_engine_inspector’

Environment

8.0.1.6:
**Jetson Nano 4gb **:
Nvidia Driver Version:
10.4:
CUDNN Version:
Jetpack 4.6:
Python Version (if applicable):
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):

2

Hi,
Please refer to below links related custom plugin implementation and sample:

While IPluginV2 and IPluginV2Ext interfaces are still supported for backward compatibility with TensorRT 5.1 and 6.0.x respectively, however, we recommend that you write new plugins or refactor existing ones to target the IPluginV2DynamicExt or IPluginV2IOExt interfaces instead.

Thanks!

5

But why doesn’t create_engine_inspector work? It’s not deprecated either according to here TensorRT: Deprecated List (nvidia.com)

6

Nvm, it just realised it is new from 8.2 release