TensorRT Profiler Override

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

I am trying to override the report_layer_time method in tensorrt.Profiler class in the Python API to report layers timing but it doesn’t work as intended.

According to the API documentation here all I have to do is override the method in the child class, so I tried the following:

class MyProfiler(trt.Profiler):
    def __init__(self):
        super().__init__()

    def report_layer_time(self, layer_name: str, ms: float):
        print('Layer {} took {} ms'.format(layer_name, ms))

profiler = MyProfiler()

and then:

with engine.create_execution_context() as context:
context.profiler = profiler

However, my print method doesn’t have any effect. The default per-layer timing info is still printed as if my method didn’t override the default one.

Environment

TensorRT Version: 7.1
GPU Type: Volta (Xavier)

2

Hi @HazemAbdelhafez,
Please take reference from the link below

Thanks!

3

Hello, thanks for the reply. This is a C++ example, is there anything working for the Python API?

4

Hi @HazemAbdelhafez,
Hope this link helps you

Thanks!

5

Trying this I get the following error when importing the module:

ImportError: generic_type: type “PyProfiler” referenced unknown base type “nvinfer1::IProfiler”

I am using TensorRT 7.1 as opposed to 5 which is used in the example you attached.

6

Hi @HazemAbdelhafez,
Apologies for delayed response.
This is an issue and the team is looking into this.
Please stay tuned.

Thanks!

7

Is there any update on this? I’m encountering the same error.