Issues while loading big textures (~2.4GB)

Hi all,

during loading of a texture which size is equal to / bigger than 2.4GB I encountered an error:

“Unknown error (Details: Function “_rtContextLaunch2D” caught exception: Assertion failed: “!m_enteredFromAPI : Memory manager already entered from API”, file: C:\u\workspace\rel5.1-win64-cuda90-VS2015-build-Release\sw\wsapps\raytracing\rtsdk\rel5.1\src\Memory\MemoryManager.cpp, line: 963)”

You should be able to easily reproduce this by loading an .ppm file bigger than 2.4 gigs (mine was a 24576 by 24576 texture with 4 unsigned byte channels) in the optix advanced sample 07 (https://github.com/nvpro-samples/optix_advanced_samples/tree/master/src/optixIntroduction/optixIntro_07)

I use a GeForce GTX 980M with 4GB RAM which should be capable of handling this size of texture (I guess) during this small example…

If you have any idea where I maybe missed some point or how I can track down what exactly is going here, please let me know. (USE_DEBUG_EXCEPTIONS activated didn’t help.)

EDIT: Is there maybe any known issue as this size exceeds INT_MAX?

Thanks in advance!

That memory manager assertion normally happens if you continue after an out-of-memory (OOM) error.
I would have expected that to be reported as an OptiX exception shortly before that assertion though.
I do not know of a way to recover from OOM errors in OptiX. You would need to setup the OptiX context from scratch after that.

Loading a single 2.4 GB texture onto a 4 GB VRAM board is rather ambitious in any 3D API. That is most likely not going to find a sufficiently sized contiguous block of memory due to VRAM fragmentation by the operating system or other graphics uses.

Other than that, the texture size limit is probably 16384 on that architecture. Query GL_MAX_TEXTURE_SIZE in OpenGL for example.