Hi, I’m currently using OptiX 5 on Windows 10 64bit, Geforce 1080 Ti.
I wrote a simple mesh viewer using OptiX. (The structure is almost same as optixMeshViewer example in the SDK. Only some parts(ex. Custom mesh loader, Removed texture support, Single material only) are different.) It works well for 508MB .ply model. But when I put 1.48GB .ply model, I get the following error after calling context->launch().
Memory allocation failed (Details: Function "_rtContextLaunch2D" caught exception: Out of memory)
If I just copy the vertices and faces to optix buffers and do nothing in optix programs, I don’t get this error. But when I create a geometry group for this, set the acceleration structure(Trbvh) and do rtTrace() on that geometry group, I get this error. And sum of the sizes(buffer->getSize() x buffer->getElementSize()) of optix buffers I use is less than 2GB. So my guess is that geometry group or acceleration structure may use a lot of memory… but I’m not sure.
Could you tell me how to solve this problem?