Recommended method to share logger between methods in C++

AI & Data Science Deep Learning (Training & Inference) TensorRT
#1

Description

I would like to make a TensorRT inference module that is callable from the outside. I would like to make two methods:

  1. An initialization method to load the engine from a file
  2. An inference method to actually run the inference

The idea is that the initialization should occur once after my application is started, but there will thereupon be many inference calls.

I am using the “sampleOnnxMNIST” as an example.

Original example has everything in one method:

int main(int argc, char** argv)
{
    samplesCommon::Args args;
    bool argsOK = samplesCommon::parseArgs(args, argc, argv);
    if (!argsOK)
    {
        sample::gLogError << "Invalid arguments" << std::endl;
        printHelpInfo();
        return EXIT_FAILURE;
    }
    if (args.help)
    {
        printHelpInfo();
        return EXIT_SUCCESS;
    }

    auto sampleTest = sample::gLogger.defineTest(gSampleName, argc, argv);

    sample::gLogger.reportTestStart(sampleTest);

    SampleOnnxMNIST sample(initializeSampleParams(args));

    sample::gLogInfo << "Building and running a GPU inference engine for Onnx MNIST" << std::endl;

    if (!sample.build())
    {
        return sample::gLogger.reportFail(sampleTest);
    }
    if (!sample.infer())
    {
        return sample::gLogger.reportFail(sampleTest);
    }

    return sample::gLogger.reportPass(sampleTest);
}

I basically want to do something like the following;
bool BuildTensorRTEngine(char* inputLayerName, char* outputLayerName, char* engineFile)
{

	// Do some initialization stuff here...

    SampleOnnxMNIST sample(initializeSampleParams(inputLayerName, outputLayerName));

    sample::gLogInfo << "Building and running a GPU inference engine for Onnx MNIST" << std::endl;

    if (!sample.build(engineFile))
    {
        return sample::gLogger.reportFail(sampleTest);
    }

    return sample::gLogger.reportPass(sampleTest);
}

bool RunInference(float* input)
{
    if (!sample.infer(input))
    {
        return sample::gLogger.reportFail(sampleTest);
    }
    return sample::gLogger.reportPass(sampleTest);
}

What’s missing above is the creation and sharing of the logger between the two callable methods. What is the recommended approach in this scenario?

Environment

TensorRT Version: 7.2 C++ API
CUDA Version: 11.1
CUDNN Version: 8.0.5
Operating System + Version: Windows 10

#3

Hi @solarflarefx,
Apologies for delayed response,
I think you can just use one logger for the two methods .
Can you please try that?

Thanks!

#4

Hi @AakankshaS ,
Actually this is what I am trying to do. I want to be able to access an initialize and inference method from the outside. My thought was to declare a logger object accessible to both of type:

sample::Logger::TestAtom Logger;

I thought perhaps I could initialize this object in the initialize method and then continue to use it in the inference method. Within the initialize method, I tried to do:

Logger = sample::gLogger.defineTest(AppName, "TensorRTLogger");

However, I get:
function “sample::Logger::TestAtom::operator=(const sample::Logger::TestAtom &)” (declared implicitly) cannot be referenced – it is a deleted function

I believe I should be using a different approach?