TensorRT engine cannot be safely deleted in deconstruction of a global or static instance


app developer is complaining about the long loading time trt plan file. so i decided to put the loading part at the global init phase and they should be deleted at the end of global lifetime.
As described in title, with the minimum code in attachment. when calling delete m_trtEngine; a segment fault happens.
By the way. if i put gItem in the main function, i works good.

I want to Ask

whether i could delete the context/engine/runtime with C++ RAII mechanism in a global/static instance.


TensorRT Version: 8.6 8.4
GPU Type: RTX3090 A5000
Nvidia Driver Version: 540, 535
CUDA Version: 12.3 11.8
CUDNN Version: 8.9 8.6
Operating System + Version: Ubuntu 22.04

Relevant Files

#include <NvInfer.h>
#include <string>
#include <iostream>
#include <fstream>
#include <memory>
#include <iterator>
#include <chrono>
#include <thread>
class TrtLogger : public nvinfer1::ILogger
    void log(Severity severity, const char *msg) noexcept override {
        std::cerr << msg << std::endl;
class Engine {
    Engine(const std::string& trt){
      std::string value;
      std::ifstream stream(trt.c_str(), std::ios::binary);
      stream >> std::noskipws;
      std::copy(std::istream_iterator<char>(stream), std::istream_iterator<char>(), back_inserter(value));
      m_trtRunTime = nvinfer1::createInferRuntime(gLogger);
      m_trtEngine = m_trtRunTime->deserializeCudaEngine(value.data(), value.size());

    ~Engine() {
      if (m_trtEngine) {
        delete m_trtEngine;
        m_trtEngine = nullptr;

      if (m_trtRunTime) {
        delete m_trtRunTime;
        m_trtRunTime = nullptr;
    TrtLogger gLogger;
    nvinfer1::ICudaEngine *m_trtEngine{nullptr};
    nvinfer1::IRuntime *m_trtRunTime{nullptr};

class gItem {
  std::shared_ptr<Engine> ee;
gItem gitem;
int main(int argc, char** argv) {
  gitem.ee = std::make_shared<Engine>("../out.trt");
  return 0;

Hi @ahawelcome ,
Can you please share your onnx model and repro script and steps.


This problem is not related to specific onnx file. I use the Resnet50.onnx provided by the TensorRT example package.
Use trtexec to generate trt plan file out.trt

trtexec --onnx=TensorRT/data/resnet50/ResNet50.onnx   --saveEngine=out.trt 

To build the cpp program, i use CMakeLists.txt below

cmake_minimum_required(VERSION 3.10)

find_package(CUDA REQUIRED)

cuda_add_executable(main main.cpp)
target_include_directories(main PRIVATE TensorRT/include)
target_link_libraries(main nvinfer nvparsers)

project structure like

TensorRT/ contain the the libs and headers of tensorrt


cd build
cmake ..

which produce

Loaded engine size: 109 MiB
Deserialization required 17090 microseconds.
[MemUsageChange] TensorRT-managed allocation in engine deserialization: CPU +0, GPU +107, now: CPU 0, GPU 107 (MiB)
[1]    2215031 segmentation fault (core dumped)  ./main

error massage may change every time you run the exacutable, Sometimes massage below may appear several times

1: [defaultAllocator.cpp::deallocate::61] Error Code 1: Cuda Runtime (driver shutting down)

Hi @ahawelcome ,
Can you please try TRT 9,and try a repro.

I am experiencing the same issue. Have you found a solution yet?