Suppress TensorRT warning and logging messages in Terminal

AI & Data Science Deep Learning (Training & Inference) TensorRT
, , ,
1

Description

I’m running TensorRT models to inference images captured from a camera. However, I’m seeing a ton of terminal output which I suspect is slowing down the inference framerate. Sample terminal output is shown below - can I disable all TensorRT terminal logging and how?

Environment

TensorRT Version: 8.4.1
GPU Type: Jetson Xavier NX 16GB production
CUDA Version: 11.4.14
CUDNN Version: 8.4.1
Operating System + Version: Jetson Linux 35.1
Python Version (if applicable): 3.8

Sample Output Log to Suppress:

[03/07/2023-15:30:23] [TRT] [I] The logger passed into createInferRuntime differs from one already provided for an existing builder, runtime, or refitter. Uses of the global logger, returned by nvinfer1::getLogger(), will return the existing value.
[03/07/2023-15:30:23] [TRT] [I] [MemUsageChange] Init CUDA: CPU +0, GPU +0, now: CPU 1198, GPU 6153 (MiB)
[03/07/2023-15:30:23] [TRT] [I] Loaded engine size: 21 MiB
[03/07/2023-15:30:23] [TRT] [W] Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors.
[03/07/2023-15:30:23] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +1, GPU +1, now: CPU 1221, GPU 6154 (MiB)
[03/07/2023-15:30:23] [TRT] [I] [MemUsageChange] TensorRT-managed allocation in engine deserialization: CPU +0, GPU +22, now: CPU 0, GPU 227 (MiB)
[03/07/2023-15:30:23] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +0, GPU +0, now: CPU 1199, GPU 6154 (MiB)
[03/07/2023-15:30:23] [TRT] [I] [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +3, now: CPU 0, GPU 230 (MiB)

2

Hi @zhifeis ,
Can you please share the detailed logs with us?
For the TRT part code, youcan change the log level and can try setting the log level to kWARNING

Thanks

5

The code responsible for the terminal output above is shown here:

def load_trt_model(model_path, device):
    Binding = namedtuple('Binding', ('name', 'dtype', 'shape', 'data', 'ptr'))
    logger = trt.Logger(trt.Logger.INFO)
    trt.init_libnvinfer_plugins(logger, namespace="")
    with open(model_path, 'rb') as f, trt.Runtime(logger) as runtime:
        model = runtime.deserialize_cuda_engine(f.read())
    bindings = OrderedDict()
    for index in range(model.num_bindings):
        name = model.get_binding_name(index)
        dtype = trt.nptype(model.get_binding_dtype(index))
        shape = tuple(model.get_binding_shape(index))
        data = torch.from_numpy(np.empty(shape, dtype=np.dtype(dtype))).to(device)
        bindings[name] = Binding(name, dtype, shape, data, int(data.data_ptr()))
    binding_addrs = OrderedDict((n, d.ptr) for n, d in bindings.items())
    context = model.create_execution_context()

    # warmup for 10 times
    for _ in range(10):
        tmp = torch.ones(1,3,288,640).half().to(device)
        binding_addrs['input.1'] = int(tmp.data_ptr())
        context.execute_v2(list(binding_addrs.values()))
    return [binding_addrs, context, bindings]

How would I set the log level? Where in the code would I do that?

6

Hi @zhifeis ,

Please refer to the below link for details on enabling the logger.

Thanks

7

U r really lazy for the greenhands :) I have seen this answer many times from Google posted by u, still not knowing where to start @AakankshaS

  1. Do we need to modify a python file in the TensortRT python library? If so , could you tell us the name of the file or its typical system path?
  2. Or we can change the Log level of TRT in our python code? If so, is there an example?