Memory leak creating nvinfer1::IRuntime on Jetson Orin

Robotics & Edge Computing Jetson & Embedded Systems Jetson Orin NX
,
1

Description

I used a sanitizer to analyze the memory usage of my app on the Jetson Orin platform and discovered a memory leak occurring when invoking the createInferRuntime function.

Environment

TensorRT Version: 8502
GPU Type: Jetson Orin
Jetpack Version: R35 (release), REVISION: 3.1, GCID: 32827747, BOARD: t186ref, EABI: aarch64, DATE: Sun Mar 19 15:19:21 UTC 2023
CUDA Version: 11.4
CUDNN Version: 8.6.0
Operating System + Version: Ubuntu 20.04

Reproduce steps

I have created a simple demo to replicate the memory leak. Could you please assist me in confirming whether I am using it incorrectly or if there is a bug present?

leak_test.cc

#include <fstream>
#include <iostream>
#include <memory>
#include <vector>
#include "NvInfer.h"
#include "NvInferRuntimeCommon.h"
#include <unistd.h>

class Logger : public nvinfer1::ILogger {
public:
    explicit Logger(Severity severity = Severity::kWARNING) : reportable_severity(severity) {}

    void log(Severity severity, const char* msg) noexcept override {
        // suppress messages with severity enum value greater than the reportable
        if (severity > reportable_severity) { return; }

      switch (severity) {
      case Severity::kINTERNAL_ERROR:
          std::cerr << "INTERNAL_ERROR: ";
          break;
      case Severity::kERROR:
          std::cerr << "ERROR: ";
          break;
      case Severity::kWARNING:
          std::cerr << "WARNING: ";
          break;
      case Severity::kINFO:
          std::cerr << "INFO: ";
          break;
      default:
          std::cerr << "UNKNOWN: ";
          break;
      }
      std::cerr << msg << std::endl;
    }

    Severity reportable_severity;
};

int main (int argc, char *argv[]) {
  Logger logger;
  auto runtime_ = std::unique_ptr<nvinfer1::IRuntime>(nvinfer1::createInferRuntime(logger));
}

CMakeLists.txt

cmake_minimum_required(VERSION 3.10)

project(leak_test)
set(CMAKE_CXX_FLAGS
    "--std=c++17 -Wno-unused-variable -Wno-unused-function -Wno-unused-parameter -Wno-deprecated-declarations -g -fsanitize=address"
)
# Find and link against the TensorRT library
include_directories(/usr/include/aarch64-linux-gnu/)
include_directories(/usr/local/cuda/include/)
link_directories(/usr/lib/aarch64-linux-gnu/)
# add_definitions(${TensorRT_DEFINITIONS})
list(APPEND LIBRARIES nvinfer)

# Add the executable target
add_executable(leak_test leak_test.cc)

# Link against the required libraries
target_link_libraries(leak_test ${LIBRARIES})

Steps To Reproduce

  • mkdir build && cd build
  • cmake ..
  • make
  • ./leak_test

Will report memory leak like this:

==606632==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 392 byte(s) in 1 object(s) allocated from:
    #0 0xffff9efdd540 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cc:144
    #1 0xffff8a42d3b4  (/lib/aarch64-linux-gnu/libnvcucompat.so+0xe3b4)
    #2 0xffff919243b4 in __pthread_once_slow /build/glibc-RIFKjK/glibc-2.31/nptl/pthread_once.c:116
    #3 0xffff94afaf10  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d62f10)
    #4 0xffff94aa6c84  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d0ec84)
    #5 0xffff94ad3ab4  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d3bab4)
    #6 0xffff92f954dc  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x11fd4dc)
    #7 0xffff92f956fc in createInferRuntime_INTERNAL (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x11fd6fc)
    #8 0xaaaae5181890 in createInferRuntime /usr/include/aarch64-linux-gnu/NvInferRuntime.h:3440
    #9 0xaaaae518195c in main /usr/src/tensorrt/samples/leak_test/leak_test.cc:42
    #10 0xffff91991e0c in __libc_start_main ../csu/libc-start.c:308
    #11 0xaaaae51817a0  (/usr/src/tensorrt/samples/leak_test/build/leak_test+0x17a0)

Direct leak of 100 byte(s) in 7 object(s) allocated from:
    #0 0xffff9efdd540 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cc:144
    #1 0xffff8a42d460  (/lib/aarch64-linux-gnu/libnvcucompat.so+0xe460)
    #2 0xffff919243b4 in __pthread_once_slow /build/glibc-RIFKjK/glibc-2.31/nptl/pthread_once.c:116
    #3 0xffff94afaf10  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d62f10)
    #4 0xffff94aa6c84  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d0ec84)
    #5 0xffff94ad3ab4  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d3bab4)
    #6 0xffff92f954dc  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x11fd4dc)
    #7 0xffff92f956fc in createInferRuntime_INTERNAL (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x11fd6fc)
    #8 0xaaaae5181890 in createInferRuntime /usr/include/aarch64-linux-gnu/NvInferRuntime.h:3440
    #9 0xaaaae518195c in main /usr/src/tensorrt/samples/leak_test/leak_test.cc:42
    #10 0xffff91991e0c in __libc_start_main ../csu/libc-start.c:308
    #11 0xaaaae51817a0  (/usr/src/tensorrt/samples/leak_test/build/leak_test+0x17a0)

Direct leak of 96 byte(s) in 1 object(s) allocated from:
    #0 0xffff9efdf0c8 in operator new(unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cc:104
    #1 0xffff92f95714 in createInferRuntime_INTERNAL (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x11fd714)
    #2 0xaaaae5181890 in createInferRuntime /usr/include/aarch64-linux-gnu/NvInferRuntime.h:3440
    #3 0xaaaae518195c in main /usr/src/tensorrt/samples/leak_test/leak_test.cc:42
    #4 0xffff91991e0c in __libc_start_main ../csu/libc-start.c:308
    #5 0xaaaae51817a0  (/usr/src/tensorrt/samples/leak_test/build/leak_test+0x17a0)

Direct leak of 28 byte(s) in 1 object(s) allocated from:
    #0 0xffff9efdd540 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cc:144
    #1 0xffff8a42db1c  (/lib/aarch64-linux-gnu/libnvcucompat.so+0xeb1c)
    #2 0xffff919243b4 in __pthread_once_slow /build/glibc-RIFKjK/glibc-2.31/nptl/pthread_once.c:116
    #3 0xffff94afaf10  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d62f10)
    #4 0xffff94aa6c84  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d0ec84)
    #5 0xffff94ad3ab4  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d3bab4)
    #6 0xffff92f954dc  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x11fd4dc)
    #7 0xffff92f956fc in createInferRuntime_INTERNAL (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x11fd6fc)
    #8 0xaaaae5181890 in createInferRuntime /usr/include/aarch64-linux-gnu/NvInferRuntime.h:3440
    #9 0xaaaae518195c in main /usr/src/tensorrt/samples/leak_test/leak_test.cc:42
    #10 0xffff91991e0c in __libc_start_main ../csu/libc-start.c:308
    #11 0xaaaae51817a0  (/usr/src/tensorrt/samples/leak_test/build/leak_test+0x17a0)

Direct leak of 20 byte(s) in 1 object(s) allocated from:
    #0 0xffff9efdd540 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cc:144
    #1 0xffff8a42db5c  (/lib/aarch64-linux-gnu/libnvcucompat.so+0xeb5c)
    #2 0xffff919243b4 in __pthread_once_slow /build/glibc-RIFKjK/glibc-2.31/nptl/pthread_once.c:116
    #3 0xffff94afaf10  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d62f10)
    #4 0xffff94aa6c84  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d0ec84)
    #5 0xffff94ad3ab4  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d3bab4)
    #6 0xffff92f954dc  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x11fd4dc)
    #7 0xffff92f956fc in createInferRuntime_INTERNAL (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x11fd6fc)
    #8 0xaaaae5181890 in createInferRuntime /usr/include/aarch64-linux-gnu/NvInferRuntime.h:3440
    #9 0xaaaae518195c in main /usr/src/tensorrt/samples/leak_test/leak_test.cc:42
    #10 0xffff91991e0c in __libc_start_main ../csu/libc-start.c:308
    #11 0xaaaae51817a0  (/usr/src/tensorrt/samples/leak_test/build/leak_test+0x17a0)

Direct leak of 20 byte(s) in 1 object(s) allocated from:
    #0 0xffff9efdd540 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cc:144
    #1 0xffff8a42db7c  (/lib/aarch64-linux-gnu/libnvcucompat.so+0xeb7c)
    #2 0xffff919243b4 in __pthread_once_slow /build/glibc-RIFKjK/glibc-2.31/nptl/pthread_once.c:116
    #3 0xffff94afaf10  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d62f10)
    #4 0xffff94aa6c84  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d0ec84)
    #5 0xffff94ad3ab4  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d3bab4)
    #6 0xffff92f954dc  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x11fd4dc)
    #7 0xffff92f956fc in createInferRuntime_INTERNAL (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x11fd6fc)
    #8 0xaaaae5181890 in createInferRuntime /usr/include/aarch64-linux-gnu/NvInferRuntime.h:3440
    #9 0xaaaae518195c in main /usr/src/tensorrt/samples/leak_test/leak_test.cc:42
    #10 0xffff91991e0c in __libc_start_main ../csu/libc-start.c:308
    #11 0xaaaae51817a0  (/usr/src/tensorrt/samples/leak_test/build/leak_test+0x17a0)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0xffff9efdd540 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cc:144
    #1 0xffff8a42d890  (/lib/aarch64-linux-gnu/libnvcucompat.so+0xe890)
    #2 0xffff919243b4 in __pthread_once_slow /build/glibc-RIFKjK/glibc-2.31/nptl/pthread_once.c:116
    #3 0xffff94afaf10  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d62f10)
    #4 0xffff94aa6c84  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d0ec84)
    #5 0xffff94ad3ab4  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d3bab4)
    #6 0xffff92f954dc  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x11fd4dc)
    #7 0xffff92f956fc in createInferRuntime_INTERNAL (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x11fd6fc)
    #8 0xaaaae5181890 in createInferRuntime /usr/include/aarch64-linux-gnu/NvInferRuntime.h:3440
    #9 0xaaaae518195c in main /usr/src/tensorrt/samples/leak_test/leak_test.cc:42
    #10 0xffff91991e0c in __libc_start_main ../csu/libc-start.c:308
    #11 0xaaaae51817a0  (/usr/src/tensorrt/samples/leak_test/build/leak_test+0x17a0)

Direct leak of 12 byte(s) in 1 object(s) allocated from:
    #0 0xffff9efdd540 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cc:144
    #1 0xffff8a42db3c  (/lib/aarch64-linux-gnu/libnvcucompat.so+0xeb3c)
    #2 0xffff919243b4 in __pthread_once_slow /build/glibc-RIFKjK/glibc-2.31/nptl/pthread_once.c:116
    #3 0xffff94afaf10  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d62f10)
    #4 0xffff94aa6c84  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d0ec84)
    #5 0xffff94ad3ab4  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d3bab4)
    #6 0xffff92f954dc  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x11fd4dc)
    #7 0xffff92f956fc in createInferRuntime_INTERNAL (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x11fd6fc)
    #8 0xaaaae5181890 in createInferRuntime /usr/include/aarch64-linux-gnu/NvInferRuntime.h:3440
    #9 0xaaaae518195c in main /usr/src/tensorrt/samples/leak_test/leak_test.cc:42
    #10 0xffff91991e0c in __libc_start_main ../csu/libc-start.c:308
    #11 0xaaaae51817a0  (/usr/src/tensorrt/samples/leak_test/build/leak_test+0x17a0)

Direct leak of 12 byte(s) in 1 object(s) allocated from:
    #0 0xffff9efdd540 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cc:144
    #1 0xffff8a42da8c  (/lib/aarch64-linux-gnu/libnvcucompat.so+0xea8c)
    #2 0xffff919243b4 in __pthread_once_slow /build/glibc-RIFKjK/glibc-2.31/nptl/pthread_once.c:116
    #3 0xffff94afaf10  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d62f10)
    #4 0xffff94aa6c84  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d0ec84)
    #5 0xffff94ad3ab4  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d3bab4)
    #6 0xffff92f954dc  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x11fd4dc)
    #7 0xffff92f956fc in createInferRuntime_INTERNAL (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x11fd6fc)
    #8 0xaaaae5181890 in createInferRuntime /usr/include/aarch64-linux-gnu/NvInferRuntime.h:3440
    #9 0xaaaae518195c in main /usr/src/tensorrt/samples/leak_test/leak_test.cc:42
    #10 0xffff91991e0c in __libc_start_main ../csu/libc-start.c:308
    #11 0xaaaae51817a0  (/usr/src/tensorrt/samples/leak_test/build/leak_test+0x17a0)

Direct leak of 8 byte(s) in 1 object(s) allocated from:
    #0 0xffff9efdd540 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cc:144
    #1 0xffff8a42daac  (/lib/aarch64-linux-gnu/libnvcucompat.so+0xeaac)
    #2 0xffff919243b4 in __pthread_once_slow /build/glibc-RIFKjK/glibc-2.31/nptl/pthread_once.c:116
    #3 0xffff94afaf10  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d62f10)
    #4 0xffff94aa6c84  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d0ec84)
    #5 0xffff94ad3ab4  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d3bab4)
    #6 0xffff92f954dc  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x11fd4dc)
    #7 0xffff92f956fc in createInferRuntime_INTERNAL (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x11fd6fc)
    #8 0xaaaae5181890 in createInferRuntime /usr/include/aarch64-linux-gnu/NvInferRuntime.h:3440
    #9 0xaaaae518195c in main /usr/src/tensorrt/samples/leak_test/leak_test.cc:42
    #10 0xffff91991e0c in __libc_start_main ../csu/libc-start.c:308
    #11 0xaaaae51817a0  (/usr/src/tensorrt/samples/leak_test/build/leak_test+0x17a0)

Direct leak of 1 byte(s) in 1 object(s) allocated from:
    #0 0xffff9efdd540 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cc:144
    #1 0xffff8a42d4f4  (/lib/aarch64-linux-gnu/libnvcucompat.so+0xe4f4)
    #2 0xffff919243b4 in __pthread_once_slow /build/glibc-RIFKjK/glibc-2.31/nptl/pthread_once.c:116
    #3 0xffff94afaf10  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d62f10)
    #4 0xffff94aa6c84  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d0ec84)
    #5 0xffff94ad3ab4  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d3bab4)
    #6 0xffff92f954dc  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x11fd4dc)
    #7 0xffff92f956fc in createInferRuntime_INTERNAL (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x11fd6fc)
    #8 0xaaaae5181890 in createInferRuntime /usr/include/aarch64-linux-gnu/NvInferRuntime.h:3440
    #9 0xaaaae518195c in main /usr/src/tensorrt/samples/leak_test/leak_test.cc:42
    #10 0xffff91991e0c in __libc_start_main ../csu/libc-start.c:308
    #11 0xaaaae51817a0  (/usr/src/tensorrt/samples/leak_test/build/leak_test+0x17a0)

Indirect leak of 52 byte(s) in 6 object(s) allocated from:
    #0 0xffff9efdd540 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cc:144
    #1 0xffff8a42d460  (/lib/aarch64-linux-gnu/libnvcucompat.so+0xe460)
    #2 0xffff919243b4 in __pthread_once_slow /build/glibc-RIFKjK/glibc-2.31/nptl/pthread_once.c:116
    #3 0xffff94afaf10  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d62f10)
    #4 0xffff94aa6c84  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d0ec84)
    #5 0xffff94ad3ab4  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x2d3bab4)
    #6 0xffff92f954dc  (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x11fd4dc)
    #7 0xffff92f956fc in createInferRuntime_INTERNAL (/lib/aarch64-linux-gnu/libnvinfer.so.8+0x11fd6fc)
    #8 0xaaaae5181890 in createInferRuntime /usr/include/aarch64-linux-gnu/NvInferRuntime.h:3440
    #9 0xaaaae518195c in main /usr/src/tensorrt/samples/leak_test/leak_test.cc:42
    #10 0xffff91991e0c in __libc_start_main ../csu/libc-start.c:308
    #11 0xaaaae51817a0  (/usr/src/tensorrt/samples/leak_test/build/leak_test+0x17a0)

SUMMARY: AddressSanitizer: 757 byte(s) leaked in 23 allocation(s).
3

Hi,

Could you check the app with cuda-memcheck?

$ /usr/local/cuda/bin/cuda-memcheck --leak-check full ./leak_test

Thanks.

4

Hi,

There is no cuda-memcheck in my machine. However, there is compute-sanitizer under the folder /usr/local/cuda/bin/. I check the demo with the command sudo /usr/local/cuda/bin/compute-sanitizer --leak-check full ./leak_test. The result is:

========= COMPUTE-SANITIZER
========= LEAK SUMMARY: 0 bytes leaked in 0 allocations
========= ERROR SUMMARY: 0 errors

I’m not familiar with compute-sanitizer, I’m not sure whether it only detects the memory leak on the GPU. I added a line that obvious causes memory leaks such as:

float * test  = new float[10];

But compute-sanitizer didn’t find the memory leak.

To make sure there is memory leak in the demo, I modified the main function like this:

int main (int argc, char *argv[]) {
  Logger logger;
  while (1) {
    auto runtime_ = std::unique_ptr<nvinfer1::IRuntime>(nvinfer1::createInferRuntime(logger));
  }
}

Then I observed the RES column in htop, it was obvious that the size of RES increased as the for loop went on which is a sign of a memory leak on the CPU side.

5

Hi,

It looks like there is no leakage from the GPU part.
For the result of -fsanitize=address, we need to check it further.

Will get back to you soon.
Thanks.

7

Hi,

The leakage is a known issue from the CUDA side and cannot be freed.
But it won’t cause any harm since the leakage is a definite amount and won’t increase over time.

Thanks.

8

Thank you for your patience.

9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.