How to Disable IExecutionContext execute() logging on terminal

AI & Data Science Deep Learning (Training & Inference) TensorRT
,
1

Description

I am using TensorRT engine for inference using Jetson TX2

with engine.create_execution_context() as context:
   context.debug_sync = False
   # Transfer input data to the GPU.
   cuda.memcpy_htod_async(d_input_1, h_input_1, stream)
   # Run inference.
   print('load profiler')
   context.profiler = trt.Profiler()
   print('execute')
   context.execute(batch_size=1, bindings=[int(d_input_1), int(d_output)])
   print('Transfer predictions back from the GPU.')
   # Transfer predictions back from the GPU.
   cuda.memcpy_dtoh_async(h_output, d_output, stream)
   # Synchronize the stream
   stream.synchronize()
   # Return the host output.
   print(h_output.shape)
   out = h_output.reshape((1,-1))
   return out

Each time context.execute(batch_size=1, bindings=[int(d_input_1), int(d_output)]) it logs the model layers all of them which makes the console not suitable for debugging. I tried to disable it but I couldn’t.
I think the problem is in this method engine.create_execution_context() as documented in the api this return IExecutionContext with debug_sync set to True . I don’t know if this is a bug or not.

I used context.debug_sync = False but it didn’t stop logging the execution layers.
how I could disable logging the execution layers on the terminal?

Environment

TensorRT Version: 7.1.3.0
GPU Type: Jetson TX2
CUDA Version: 10.2.89
CUDNN Version: 8.0.0.180
Operating System + Version: jetpack 4.4.1
TensorFlow Version (if applicable): 2.3.1+nv20.10

Relevant Files inference.py (3.0 KB)

2

Hi,
Please refer to the installation steps from the below link if in case you are missing on anything

However suggested approach is to use TRT NGC containers to avoid any system dependency related issues.

In order to run python sample, make sure TRT python packages are installed while using NGC container.
/opt/tensorrt/python/python_setup.sh
Thanks!

5

Hi @ammaz99,

IExecutionContext does not log layers. Looks like you’re using a profiler context.profiler = trt.Profiler()
which is why you’re seeing this behavior.

Thank you.

1 Like
6

@spolisetty .Thanks man. It worked!

1 Like