optix calling D3D11 CreateTexture2D for unknown reason and exhausting memory

Gaming and Visualization Technologies Visualization OptiX
1

Hi,

I’m currently developing a normal/displacement map baker (very similar to xNormal) using optix. It works very well, but it crashes in rtContextLaunch2D as soon as i’m baking maps bigger than about 3000x3000.

First-chance exception at 0x7640c41f in yNormal.exe: Microsoft C++ exception: _com_error at memory location 0x0035cc2c…
First-chance exception at 0x7640c41f in yNormal.exe: Microsoft C++ exception: _com_error at memory location 0x0035d068…
First-chance exception at 0x7640c41f in yNormal.exe: Microsoft C++ exception: _com_error at memory location 0x0035d16c…
First-chance exception at 0x7640c41f in yNormal.exe: Microsoft C++ exception: _com_error at memory location 0x0035e2dc…
D3D11 ERROR: ID3D11Device::CreateTexture2D: Returning E_OUTOFMEMORY, meaning memory was exhausted. [ STATE_CREATION ERROR #105: CREATETEXTURE2D_OUTOFMEMORY_RETURN]
D3D11 WARNING: Using ID3D11Debug::ReportLiveDeviceObjects with D3D11_RLDO_DETAIL will help drill into object lifetimes. Objects with Refcount=0 and IntRef=0 will be eventually destroyed through typical Immediate Context usage. However, if the application requires these objects to be destroyed sooner, ClearState followed by Flush on the Immediate Context will realize their destruction.
[ STATE_CREATION WARNING #422: LIVE_OBJECT_SUMMARY]
D3D11 WARNING: Live ID3D11Device at 0x00456A64, Refcount: 20 [ STATE_CREATION WARNING #441: LIVE_DEVICE]
First-chance exception at 0x7640c41f in yNormal.exe: Microsoft C++ exception: optix::CudaError at memory location 0x0035f0fc…

It seems rtContextLaunch2D does some CreateTexture2D in D3D11 for an unknown reason and exhausts the memory (I run on a 6Gb titan)

I tried replacing rtContextLaunch2D( context, 0, resolution, resolution ) by rtContextLaunch2D( context, 0, 10, 10 ) it still crashes.

Some more specifics on my optix/D3D11 usage:

  • first I generate 2 FLOAT4 textures (render targets) using D3D11. Let’s say their resolution is 4096x4096. One contains the low resolution mesh normals (arranged in a UV atlas), the other contains low-res mesh positions.

  • I bind these 2 textures in optix using rtTextureSamplerCreateFromD3D11Resource, they are used to generate the rays that are tested against the high resolution mesh

  • I also have a 4096x4096 FLOAT4 optix output buffer that receive hi-res normal in xyz and displacement in w

doing the math my input and output textures/buffers (not counting mesh and other minor things) take 40964096sizeof(float)*12 = 768Mb of video memory (far from the 6Gb of the Titan)

2 questions:

  • Is there another problem than memory exhaustion ? because I seem to have exceptions before the D3D11 E_OUTOFMEMORY error.

  • Why does optix calls CreateTexture2D himself ? how often does it do so ? how can I fix this ?
    I should be able to handle far bigger maps than that without resorting to tiling.

PS: for legal reasons I’m unfortunately not able to send you (nvidia) the source code. However I might be able to come up with a minimal repro case.

Thanks in advance.
Florent Tournade

2

Hi Florent,

If you can provide a repro case that would be great. Can you also include more info in addition to the GPU HW about your test system? CUDA and driver version, platform, dev environment?

Best,
GM

3

I circumvented the problem by implementing a tiled approach (doing the computations on small chunks at a time) I’m now able to render up to 8192*8192.

I don’t think I’ll have the time to make a repro case, but I’ll still be curious as to why optix in certain circunstances silently allocates D3D textures.

cheers