Driver Crash when creating Root Signature for D3D12 resource with shared heap in Unreal Engine 4

Gaming and Visualization Technologies General Topics and Other SDKs
1

Hi,

I’m currently working on a game streaming project where we are using the CUDA video encoder to encode images captured in a Unreal Engine 4 application at runtime.

In order to do this, we need to share the graphics API texture resource. For D3D11, this works fine because we are using the CUDA CUgraphicsResource type that is not available for D3D12. For D3D12, we need to create a shared handle to the resource to be able to load it from external memory for use with CUDA. The D3D12 CreateSharedHandle function will fail if the heap flag is not set to D3D12_HEAP_FLAG_SHARED when using the
D3D12 CreateCommittedResource function to create the D3D12Resource.

However Unreal Engine hard codes this flag in its D3D12 texture creation code to be D3D12_HEAP_FLAG_NONE and changing it to D3D12_HEAP_FLAG_SHARED successfully creates the texture. However we then set up a compute shader that writes to this texture and when setting the params for this shader, Unreal tries to create a D3D12RootSignature and the CreateRootSignature D3D12 api call causes the driver to crash.

My drivers are the up to date.

Is there a way to create a shared D3D12 resource with CUDA without using the D3D12_HEAP_FLAG_SHARED flag or does anyone have any other recommendations on how this could be resolved?

2

I resolved this issue. When debugging with -d3d12 flag in Unreal, I got more information. A change I made to the engine was resulting in a mismatch between the format of the texture and the texture format of its associated UAV and this caused the crash. Works fine now. This topic can be closed. Apologies for not closing sooner.