TensorRT engine cannot be safely deleted in deconstruction of a global or static instance

Description

app developer is complaining about the long loading time trt plan file. so i decided to put the loading part at the global init phase and they should be deleted at the end of global lifetime.
As described in title, with the minimum code in attachment. when calling delete m_trtEngine; a segment fault happens.
By the way. if i put gItem in the main function, i works good.

I want to Ask

whether i could delete the context/engine/runtime with C++ RAII mechanism in a global/static instance.

Environment

TensorRT Version: 8.6 8.4
GPU Type: RTX3090 A5000
Nvidia Driver Version: 540, 535
CUDA Version: 12.3 11.8
CUDNN Version: 8.9 8.6
Operating System + Version: Ubuntu 22.04

Relevant Files

#include <NvInfer.h>
#include <string>
#include <iostream>
#include <fstream>
#include <memory>
#include <iterator>
#include <chrono>
#include <thread>
class TrtLogger : public nvinfer1::ILogger
{
public:
    void log(Severity severity, const char *msg) noexcept override {
        std::cerr << msg << std::endl;
    }
};
class Engine {
  public:
    Engine(){}
    Engine(const std::string& trt){
      std::string value;
      std::ifstream stream(trt.c_str(), std::ios::binary);
      stream >> std::noskipws;
      std::copy(std::istream_iterator<char>(stream), std::istream_iterator<char>(), back_inserter(value));
      m_trtRunTime = nvinfer1::createInferRuntime(gLogger);
      m_trtEngine = m_trtRunTime->deserializeCudaEngine(value.data(), value.size());

    }
    ~Engine() {
      if (m_trtEngine) {
        delete m_trtEngine;
        m_trtEngine = nullptr;
      }

      if (m_trtRunTime) {
        delete m_trtRunTime;
        m_trtRunTime = nullptr;
      }
      
    } 
  private:
    TrtLogger gLogger;
    nvinfer1::ICudaEngine *m_trtEngine{nullptr};
    nvinfer1::IRuntime *m_trtRunTime{nullptr};

};
class gItem {
public:
  std::shared_ptr<Engine> ee;
};
gItem gitem;
int main(int argc, char** argv) {
  gitem.ee = std::make_shared<Engine>("../out.trt");
  return 0;
}

Hi @ahawelcome ,
Can you please share your onnx model and repro script and steps.

Thanks

This problem is not related to specific onnx file. I use the Resnet50.onnx provided by the TensorRT example package.
Use trtexec to generate trt plan file out.trt

trtexec --onnx=TensorRT/data/resnet50/ResNet50.onnx   --saveEngine=out.trt 

To build the cpp program, i use CMakeLists.txt below

cmake_minimum_required(VERSION 3.10)
project(prog)

find_package(CUDA REQUIRED)

link_directories(TensorRT/lib)
cuda_add_executable(main main.cpp)
target_include_directories(main PRIVATE TensorRT/include)
target_link_libraries(main nvinfer nvparsers)

project structure like
proj/
|–TensorRT/
|
|–build/
|
|–main.cpp
|
|–CMakeLists.txt
|
|–out.trt

TensorRT/ contain the the libs and headers of tensorrt

then,

cd build
cmake ..
make
./main

which produce

Loaded engine size: 109 MiB
Deserialization required 17090 microseconds.
[MemUsageChange] TensorRT-managed allocation in engine deserialization: CPU +0, GPU +107, now: CPU 0, GPU 107 (MiB)
[1]    2215031 segmentation fault (core dumped)  ./main

error massage may change every time you run the exacutable, Sometimes massage below may appear several times

1: [defaultAllocator.cpp::deallocate::61] Error Code 1: Cuda Runtime (driver shutting down)

Hi @ahawelcome ,
Can you please try TRT 9,and try a repro.
Thanks

I am experiencing the same issue. Have you found a solution yet?