OptiX SDK 7.2.0 out-of-host-memory error 7002

I’m getting this error on my project with optixModuleCreateFromPTX().
Same code before worked with 7.1.
I’ve also tested Optix7.2 on example 2 from the Optix7 course which uses optixModuleCreateFromPTX() and I get the same error…

#osc: setting up module …
[ 2][ ERROR]: Unknown std::bad_alloc error
Optix call (optixModuleCreateFromPTX(optixContext, &moduleCompileOptions, &pipelineCompileOptions, ptxCode.c_str(), ptxCode.size(), log,&sizeof_log, &module )) failed with code 7002 (line 171)

System Config
Ubuntu 18.04
2x RTX 2080Ti
128GB Ram
Cuda 11.1
gcc-6/g+±6
CMake 3.10.2
Driver version 455.28

1 Like

Thanks for the report.
I filed a bug report internally to investigate this.

That turned out to be a problem in these examples.
OptiX applications should meticulously default-initialize OptiX structures before setting the necessary parameters.
That is not the case in Ingo’s code.

In this case OptiX SDK 7.2.0 added a new feature
https://raytracing-docs.nvidia.com/optix7/guide/index.html#program_pipeline_creation#parameter-specialization
which is controlled by these two additional fields inside the OptixModuleCompileOptions:

/// Ingored if numBoundValues is set to 0
const OptixModuleCompileBoundValueEntry* boundValues;

/// set to 0 if unused
unsigned int numBoundValues;

Now if the OptixModuleCompileOptions have not been default-initialized when switching to the OptiX SDK 7.2.0 version, these two fields can contain random values and with a huge number of numBoundValues there will be a huge amount of memory allocated to manage them.
So either you get an OPTIX_ERROR_HOST_OUT_OF_MEMORY or an OPTIX_ERROR_UNKNOWN during optixModuleCreateFromPTX() if the boundValues is also random garbage.

These changes should fix it:

void SampleRenderer::createModule()
{
  moduleCompileOptions = {}; // Was missing

  moduleCompileOptions.maxRegisterCount  = OPTIX_COMPILE_DEFAULT_MAX_REGISTER_COUNT; // Do not limit the amount of registers.
  moduleCompileOptions.optLevel          = OPTIX_COMPILE_OPTIMIZATION_DEFAULT; // or OPTIX_COMPILE_OPTIMIZATION_LEVEL_3;
  moduleCompileOptions.debugLevel        = OPTIX_COMPILE_DEBUG_LEVEL_NONE; // or OPTIX_COMPILE_DEBUG_LEVEL_LINEINFO;

  pipelineCompileOptions = {};

  pipelineCompileOptions.traversableGraphFlags = OPTIX_TRAVERSABLE_GRAPH_FLAG_ALLOW_SINGLE_GAS;
  pipelineCompileOptions.usesMotionBlur     = 0; // Type is int.
  pipelineCompileOptions.numPayloadValues   = 2;
  pipelineCompileOptions.numAttributeValues = 2;
  pipelineCompileOptions.exceptionFlags     = OPTIX_EXCEPTION_FLAG_NONE;
  pipelineCompileOptions.pipelineLaunchParamsVariableName = "optixLaunchParams";

  pipelineLinkOptions = {}; // Was missing.

  //pipelineLinkOptions.overrideUsesMotionBlur = 0; // Type is int. Removed in OptiX 7.1.0: 
  pipelineLinkOptions.maxTraceDepth = 2;
  pipelineLinkOptions.debugLevel    = OPTIX_COMPILE_DEBUG_LEVEL_NONE; // or OPTIX_COMPILE_DEBUG_LEVEL_LINEINFO; Should match with moduleCompileOptions.debugLevel  // Was missing.

  const std::string ptxCode = embedded_ptx_code;
  
  char log[2048];
  size_t sizeof_log = sizeof( log );

  OPTIX_CHECK(optixModuleCreateFromPTX(optixContext,
                                       &moduleCompileOptions,
                                       &pipelineCompileOptions,
                                       ptxCode.c_str(),
                                       ptxCode.size(),
                                       log,&sizeof_log,
                                       &module
                                       ));
  if (sizeof_log > 1)
    PRINT(log);
}
4 Likes

Please try pulling the OptiX 7 Course repository again.
The relevant OptiX structs are default-initialized now.