TensorRT Warning

[igal.kroyter]

Hi,
Could you please explain how each one of the warnings below can be fixed?

The line that invokes these warnings is re-invocation of the following of the following command:

x = tf.convert_to_tensor(np.random.normal(size=(batch_size,3,120)).astype(np.float32))
Warnings:

1. TF-TRT Warning: DefaultLogger Skipping tactic 7 due to oom error on requested size of 2835 detected for tactic 60.
2. DefaultLogger Tactic Device request: 2835MB Available: 585MB. Device memory is insufficient to use tactic.
3. TF-TRT Warning: DefaultLogger Detected invalid timing cache, setup a local cache instead
4. TF-TRT Warning: DefaultLogger The logger passed into createInferBuilder differs from one already provided for an existing builder, runtime, or refitter. TensorRT maintains only a single logger pointer at any given time, so the existing value, which can be retrieved with getLogger(), will be used instead. In order to use a new logger, first destroy all existing builder, runner or refitter objects.

Moreover I’m trying to silence TensroFlow logger without success:

tf.compat.v1.logging.set_verbosity(tf.compat.v1.logging.INFO)

What is the correct way to do it
Regards.

[AastaLLL]

Hi,

Most of the warning is related to out-of-memory.

May I know which model do you try to deploy?
Have you run it on a desktop environment before? If yes, could you check the memory size it takes first?

Thanks.

[igal.kroyter]

hi @AastaLLL,

I’ve tried to print the memory utilisation of the GPU with the following command:

mem = tf.config.experimental.get_memory_info(‘GPU:0’)
print(“current:”,mem[‘current’], " peak:",mem[‘peak’]);

current: 182272 peak: 234704384
2021-11-24 10:37:13.466708: W tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:38] TF-TRT Warning: DefaultLogger It is suggested to disable layer timing cache while using AlgorithmSelector. Please refer to the developer guide in Developer Guide :: NVIDIA Deep Learning TensorRT Documentation.
…
2021-11-24 10:37:48.742317: W tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:38] TF-TRT Warning: DefaultLogger It is suggested to disable layer timing cache while using AlgorithmSelector. Please refer to the developer guide in Developer Guide :: NVIDIA Deep Learning TensorRT Documentation.
2021-11-24 10:37:48.748542: W tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:38] TF-TRT Warning: DefaultLogger Detected invalid timing cache, setup a local cache instead
2021-11-24 10:37:49.287113: W tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:38] TF-TRT Warning: DefaultLogger The logger passed into createInferBuilder differs from one already provided for an existing builder, runtime, or refitter. TensorRT maintains only a single logger pointer at any given time, so the existing value, which can be retrieved with getLogger(), will be used instead. In order to use a new logger, first destroy all existing builder, runner or refitter objects.
2021-11-24 10:37:49.334867: W tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:38] TF-TRT Warning: DefaultLogger It is suggested to disable layer timing cache while using AlgorithmSelector. Please refer to the developer guide in Developer Guide :: NVIDIA Deep Learning TensorRT Documentation.
2021-11-24 10:37:49.363949: W tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:38] TF-TRT Warning: DefaultLogger Detected invalid timing cache, setup a local cache instead
2021-11-24 10:37:55.476560: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger Tactic Device request: 1430MB Available: 525MB. Device memory is insufficient to use tactic.
2021-11-24 10:37:56.185034: W tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:38] TF-TRT Warning: DefaultLogger Skipping tactic 3 due to oom error on requested size of 1430 detected for tactic 4.
2021-11-24 10:37:56.276412: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger Tactic Device request: 1430MB Available: 535MB. Device memory is insufficient to use tactic.
2021-11-24 10:37:56.276657: W tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:38] TF-TRT Warning: DefaultLogger Skipping tactic 8 due to oom error on requested size of 1430 detected for tactic 60.
current: 71713792 peak: 547478016

where did the 8637 MB are gone?

Could you please elaborate how to modify logger’s verbosity? I guess that I need to get access to the logger of the frozen graph, don’t I?

Regards.

[AastaLLL]

Hi,

Could you run sudo tegrastats on the other console at the same time?
Since Jetson has integrated memory, the CPU memory from the system also reduces the available memory amount.

Thanks.