Memory leak creating nvinfer1::IRuntime

AI & Data Science Deep Learning (Training & Inference) TensorRT
,
1

Description

I used a sanitizer to analyze the memory usage of my app and discovered a memory leak occurring when invoking the createInferRuntime function. I have created a simple demo to replicate the memory leak. Could you please assist me in confirming whether I am using it incorrectly or if there is a bug present?

Environment

TensorRT Version: 8600
GPU Type: GTX-2080
Nvidia Driver Version: 510.47.03
CUDA Version: 11.6
CUDNN Version: 8.4.1
Operating System + Version: Ubuntu 18.04

Relevant Files

leak_test.cc

#include "NvInfer.h"
#include "NvInferRuntimeCommon.h"
#include <memory>

class Logger : public nvinfer1::ILogger {
public:
  explicit Logger(Severity severity = Severity::kWARNING)
      : reportable_severity(severity) {}

  void log(Severity severity, const char *msg) noexcept override {}

  Severity reportable_severity;
};

int main(int argc, char *argv[]) {
  Logger logger;
  auto runtime_ =
      std::unique_ptr<nvinfer1::IRuntime>(nvinfer1::createInferRuntime(logger));
  return 0;
}

CMakeLists.txt

cmake_minimum_required(VERSION 3.10)

project(leak_test)
set(CMAKE_CXX_FLAGS
    "--std=c++17 -Wno-unused-variable -Wno-unused-function -Wno-unused-parameter -Wno-deprecated-declarations -g -fsanitize=address"
)
include_directories(/usr/include/x86_64-linux-gnu/)
include_directories(/usr/local/cuda/include/)
link_directories(/usr/lib/x86_64-linux-gnu/)
list(APPEND LIBRARIES nvinfer)

# Add the executable target
add_executable(leak_test leak_test.cc)

# Link against the required libraries
target_link_libraries(leak_test ${LIBRARIES})

Steps To Reproduce

  • mkdir build && cd build
  • cmake ..
  • make
  • ./leak_test

Will report memory leak like this:

AddressSanitizer:DEADLYSIGNAL
=================================================================
==19489==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x7f83585762cb bp 0x7ffe5e3a7a70 sp 0x7ffe5e3a7a50 T0)
==19489==The signal is caused by a READ memory access.
==19489==Hint: address points to the zero page.
    #0 0x7f83585762cb in createInferRuntime_INTERNAL (/usr/lib/x86_64-linux-gnu/libnvinfer.so.8+0x142b2cb)
    #1 0x55f8044cb0c1 in createInferRuntime /usr/include/x86_64-linux-gnu/NvInferRuntime.h:3653
    #2 0x55f8044cb187 in main /usr/src/tensorrt/samples/leak_test/leak_test.cc:18
    #3 0x7f83563b8c86 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21c86)
    #4 0x55f8044cafe9 in _start (/usr/src/tensorrt/samples/leak_test/build/leak_test+0xfe9)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV (/usr/lib/x86_64-linux-gnu/libnvinfer.so.8+0x142b2cb) in createInferRuntime_INTERNAL
==19489==ABORTING
2

Hi,
Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:

  1. validating your model with the below snippet

check_model.py

import sys
import onnx
filename = yourONNXmodel
model = onnx.load(filename)
onnx.checker.check_model(model).
2) Try running your model with trtexec command.

In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging
Thanks!

3

Thanks for your response.

This thread was deprecated because of the incorrect description. I didn’t find the entrance to delete this post except to flag this post to the maintainers.

For developers reading this post, I have posted a new thread - Memory leak creating nvinfer1::IRuntime on Jetson Orin - Jetson & Embedded Systems / Jetson Orin NX - NVIDIA Developer Forums