TensorRT 8.0 Refitting Verbosity

Gudbach · November 20, 2021, 1:00am

Description

I am working with the TensorRT C++ API.

In TensorRT 8.0, it is noticed that it has become more verbose than previous versions, when calling the refitting API. For example, each call of refitting-weights logs something like below on the console.

**
Using cublasLt a tactic source
[MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +8, now: CPU 12589, GPU 2718 (MiB)
Using cuDNN as a tactic source
[MemUsageChange] Init cuDNN: CPU +0, GPU +8, now: CPU 12589, GPU 2726 (MiB)
[MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 12597, GPU 2710 (MiB)
[MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 12553, GPU 2508 (MiB)
[MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 12264, GPU 2154 (MiB)
[MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 12081, GPU 1870 (MiB)
**

Is there anyway to suppress these logs?

Environment

TensorRT Version: TensorRT-8.0.3.4
GPU Type: GeForce 2080
Nvidia Driver Version: 465.89
CUDA Version: 11.0
CUDNN Version: 8.2.1
Operating System + Version: Windows 10
Python Version (if applicable): N/A
TensorFlow Version (if applicable): N/A
PyTorch Version (if applicable): N/A
Baremetal or Container (if container which image + tag): N/A

Relevant Files

Please attach or include links to any models, data, files, or scripts necessary to reproduce your issue. (Github repo, Google Drive, Dropbox, etc.)

Steps To Reproduce

Please include:

Exact steps/commands to build your repro
Exact steps/commands to run your repro
Full traceback of errors encountered

spolisetty · December 3, 2021, 4:42pm

Hi,

Sorry for the delayed response. We can alter the Severity (level) of the logging. Similar to following, which may help you.
https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#build-phase-c

github.com

NVIDIA/TensorRT/blob/main/samples/common/logging.h#L182

    
      
          private:
              static std::ostream& severityOstream(Severity severity)
              {
                  return severity >= Severity::kINFO ? std::cout : std::cerr;
              }
          
          
    static std::string severityPrefix(Severity severity)
              {
                  switch (severity)
                  {
                  case Severity::kINTERNAL_ERROR: return "[F] ";
                  case Severity::kERROR: return "[E] ";
                  case Severity::kWARNING: return "[W] ";
                  case Severity::kINFO: return "[I] ";
                  case Severity::kVERBOSE: return "[V] ";
                  default: assert(0); return "";
                  }
              }
          
          
    bool mShouldLog;
              Severity mSeverity;

Thank you.

rapidsrik4 · January 21, 2022, 4:47am

I also seeing lot of console outputs after switching to TRTv8.2.2.1. I tried to suppress logs based on above solution byadding below lines in derived logger class.

class Logger : public ILogger           
{
    void log(Severity severity, const char* msg) override
    {
        // suppress info warning level messages
        if (severity < Severity::kWARNING)
            std::cout << msg << std::endl;
    }
} logger;

Nothing worked so far. I still see lots of console outputs. Any suggestion?