Using latest tensorRT 8.2.1, but can't find EngineInspector API

AI & Data Science Deep Learning (Training & Inference) TensorRT
#1

Description

I start my work from Nvidia recommend docker image ‘nvcr.io/nvidia/tensorrt:21.11-py3’ , then upgrade to tensorrt 8.2.1 by following the instruction from Installation Guide :: NVIDIA Deep Learning TensorRT Documentation

then reinstall the latest pycuda, but I still don’t have the EngineInspector API in my tensorrt python API.

could anyone can help?

I need the EngineInspector API to insight the quantized engine of tensorrt.

>>> import tensorrt as trt
>>> dir(trt.EngineInspector)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'tensorrt' has no attribute 'EngineInspector'

Environment

TensorRT Version: 8.2.1
GPU Type: GTX2070
Nvidia Driver Version: 470.74
CUDA Version: 11.4
CUDNN Version: UNK
Operating System + Version: ubuntu18.04
Python Version (if applicable): 3.8
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):

Relevant Files

Please attach or include links to any models, data, files, or scripts necessary to reproduce your issue. (Github repo, Google Drive, Dropbox, etc.)

Steps To Reproduce

Please include:

  • Exact steps/commands to build your repro
  • Exact steps/commands to run your repro
  • Full traceback of errors encountered
#2

Hi,
Please refer to below links related custom plugin implementation and sample:
https://docs.nvidia.com/deeplearning/tensorrt/sample-support-guide/index.html#onnx_packnet

While IPluginV2 and IPluginV2Ext interfaces are still supported for backward compatibility with TensorRT 5.1 and 6.0.x respectively, however, we recommend that you write new plugins or refactor existing ones to target the IPluginV2DynamicExt or IPluginV2IOExt interfaces instead.

Thanks!

#5

thanks for the suggestion, but it’s not what I want.

What I want is just to get the latest python API by which I can use the tensorrt.EngineInspector API to look inside my trt engine.

I can’t find any information to guide me to do this right now.

I downloaded nv-tensorrt-repo-ubuntu1804-cuda11.4-trt8.2.1.8-ga-20211117_1-1_amd64.deb and installed it, and then installed pycuda 2021.1, but the python API were still missing tensorrt.EngineInspector.

#6

It’s resolved after I install nvidia-pyindex and then upgrade nvidia-tensorrt through pip. I can use the latest 8.2.* Python APIs now.

pip install nvidia-pyindex

pip install --upgrade nvidia-tensorrt