Issue of using CUPTI Profiling API refer to callback_profiling.cu sample

[gulatuoba.cn]

Hi,
I want to profile metric values of AI model, and I refer to callback_profiling.cu sample code. environment: jetson orin with JetPack6.1
First, the sample code can be run. But I want to start profiling after the model has been running for a while (warming up). I tried to change the sample code, which is to simply add a line(just DoVectorAddition function) at the beginning of main(), and then found that the domain CUPTI_CB_DOMAIN_RESOURCE and the callbak id CUPTI_CBID_RESOURCE_CONTEXT_CREATED in the callback function cannot be obtained. , so SetupProfiling cannot be executed.
Is there any way to customize the start and end moments of profiling?

original code result:

modified code and result:

Thank you!

[gulatuoba.cn]

another question, when I add disable subscriber, it will report error.
I just add two lines code on the same sample of callback_profiling.cu.

run result error:

[Sanjiv.Satoor]

Regarding your first query - “But I want to start profiling after the model has been running for a while (warming up).”

When we call DoVectorAddition() at the very beginning - none of the CUPTI API calls have been made yet - so SetupProfiling() is not called on context create. This results in the CUPTI_ERROR_INVALID_OPERATION error.

You can change the code as follows:

1. Add a bSkipKernelLaunch flag in the ProfilingData struct

typedef struct ProfilingData_t
{
int numRanges = 2;
bool bProfiling = false;
bool bSkipKernelLaunch = true;
.
.
std::vector<uint8_t> counterDataScratchBuffer;
} ProfilingData;

2. Move the SetupProfiling() call from context create to kernel launch

Modify the code:

    case CUPTI_CB_DOMAIN_DRIVER_API:
    {
        switch (callbackId)
        {
            case CUPTI_DRIVER_TRACE_CBID_cuLaunchKernel:
            {
                if (pCallbackInfo->callbackSite == CUPTI_API_ENTER)
                {
                    if(!pProfilingData->bSkipKernelLaunch)
                    {
                        if(!pProfilingData->bProfiling)
                        {
                            SetupProfiling(pProfilingData);
                            pProfilingData->bProfiling = true;
                        }
                        EnableProfiling(pProfilingData);
                    }
                }
                else
                {
                    if(!pProfilingData->bSkipKernelLaunch)
                          DisableProfiling(pProfilingData);
                }
            }
            break;
            default:
                break;
        }
        break;
    }

Remove the code:

    case CUPTI_CB_DOMAIN_RESOURCE:
    {
        switch (callbackId)
        {
            case CUPTI_CBID_RESOURCE_CONTEXT_CREATED:
            {
                SetupProfiling(pProfilingData);
                pProfilingData->bProfiling = true;
            }
            break;
            default:
                break;
        }
        break;
    }

3. Add call to DoVectorAddition() at the start for warm-up - as you had done.

4. After warm-up disable the bSkipKernelLaunch flag

pProfilingData->bSkipKernelLaunch = false; // start profiling
do
{
    DoVectorAddition();
}
while (!pProfilingData->allPassesSubmitted);

[Sanjiv.Satoor]

Regarding the second query - “when I add disable subscriber, it will report error.”

The StopProfiling() call calls cuptiProfilerDeInitialize(), which removes all the resources and unsubscribes all the callbacks and resets the subscriber handle.

Now on call to cuptiEnableCallback() CUPTI reinitializes with new structures and handles. Due to the old subscriber handle an CUPTI_ERROR_INVALID_PARAMETER error is reported.

[gulatuoba.cn]

Thank you!