Crash: Shader Compilation Failure When Creating Primitives in Modeling Mode

Gaming and Visualization Technologies Engines Unreal
1

Engine Version: Unreal Engine NVRTX 5.5 Caustics Branch

GPU: NVIDIA GeForce RTX 4090 Laptop GPU

Driver Version: 580.97

Description: The editor crashes 100% of the time when attempting to create any new primitive (e.g., Cube, Sphere) using the in-editor Modeling Tools. The crash occurs in all viewport modes (Lit, Unlit, Wireframe).

Steps to Reproduce:

  1. Launch the editor and open any project.

  2. Select “Modeling” from the “Modes” dropdown.

  3. Under the “Create” section, click on “Cube” or any other primitive.

  4. Move the mouse into the viewport to place the primitive.

  5. The editor crashes immediately.

Fatal Error Message (from log): Fatal error: [File:C:\GameDev\UnrealEngine-NvRTX_Caustics-5.5\Engine\Source\Runtime\RHI\Private\PipelineStateCache.cpp] [Line: 528] Shader compilation failures are Fatal.

Debugging Analysis: The crash is a shader compilation failure triggered by the hit proxy rendering pass.

  • Trigger: The crash occurs when the editor calls FEditorViewportClient::GetCursor to determine what the mouse is hovering over. This flushes rendering commands and forces the compilation of a shader for the newly created primitive’s hit proxy.

  • Shader Permutation: The failing shader’s DDC key includes the string RTXDITS. The C++ code in Shader.cpp adds this permutation when Data.IsRTXDITranslucencyEnabled() is true, setting the FRTXDITSTranslucency permutation.

  • Problematic Shader Code: This seems to enable the USE_RTXDI_TRANSPARENT_SHADOW define in HLSL. The code under this define does not appear to be guarded against running in the hit proxy pass (i.e., it is missing a !MATERIAL_SHADER_TYPE_HITPROXY check), causing the compile to fail.

    Call Stack (Visual Studio attached to process) Game Thread:

    ntdll.dll!00007ffb079c2934() Unknown
    KernelBase.dll!00007ffb04f5cbbf() Unknown

    UnrealEditor-Core.dll!FEventWin::Wait(unsigned int WaitTime, const bool bIgnoreThreadIdleStats) Line 1726 C++
    [Inline Frame] UnrealEditor-RenderCore.dll!FEvent::Wait(const FTimespan &) Line 89 C++
    [Inline Frame] UnrealEditor-RenderCore.dll!GameThreadWaitForTask(const UE::Tasks::Private::FTaskHandle &) Line 1264 C++
    [Inline Frame] UnrealEditor-RenderCore.dll!FRenderCommandFence::Wait(bool) Line 1306 C++
    UnrealEditor-RenderCore.dll!FlushRenderingCommands() Line 1350 C++
    UnrealEditor-Engine.dll!FViewport::GetRawHitProxyData(UE::Math::TIntRect InRect) Line 1972 C++
    UnrealEditor-Engine.dll!FViewport::GetHitProxyMap(UE::Math::TIntRect InRect, TArray<HHitProxy *,TSizedDefaultAllocator<32>> & OutMap) Line 1980 C++
    UnrealEditor-Engine.dll!FViewport::GetHitProxy(int X, int Y) Line 2029 C++
    UnrealEditor-UnrealEd.dll!FEditorViewportClient::GetCursor(FViewport * InViewport, int X, int Y) Line 1722 C++
    UnrealEditor-UnrealEd.dll!FLevelEditorViewportClient::GetCursor(FViewport * InViewport, int X, int Y) Line 4614 C++
    UnrealEditor-Engine.dll!FSceneViewport::OnCursorQuery(const FGeometry & MyGeometry, const FPointerEvent & CursorEvent) Line 490 C++
    UnrealEditor-Slate.dll!SViewport::OnCursorQuery(const FGeometry & MyGeometry, const FPointerEvent & CursorEvent) Line 233 C++
    UnrealEditor-Slate.dll!FSlateUser::QueryCursor() Line 815 C++
    UnrealEditor-Slate.dll!FSlateUser::UpdateCursor() Line 917 C++
    [Inline Frame] UnrealEditor-Slate.dll!FSlateApplication::TickPlatform::__l14::<lambda_1>::operator()(FSlateUser &) Line 1633 C++
    [Inline Frame] UnrealEditor-Slate.dll!Invoke(FSlateApplication::TickPlatform::l14::<lambda_1> &) Line 47 C++
    UnrealEditor-Slate.dll!UE::Core::Private::Function::TFunctionRefCaller<FSlateApplication::TickPlatform'::14’::<lambda_1>,void,FSlateUser &>::Call(void * Obj, FSlateUser & <Params_0>) Line 315 C++
    [Inline Frame] UnrealEditor-Slate.dll!UE::Core::Private::Function::TFunctionRefBase<UE::Core::Private::Function::FFunctionRefStoragePolicy,void cdecl(FSlateUser &)>::operator()(FSlateUser &) Line 470 C++
    [Inline Frame] UnrealEditor-Slate.dll!FSlateApplication::ForEachUser(TFunctionRef<void cdecl(FSlateUser &)>) Line 4423 C++
    UnrealEditor-Slate.dll!FSlateApplication::TickPlatform(float DeltaTime) Line 1632 C++
    UnrealEditor-Slate.dll!FSlateApplication::Tick(ESlateTickType TickType) Line 1561 C++
    UnrealEditor.exe!FEngineLoop::Tick() Line 5922 C++
    [Inline Frame] UnrealEditor.exe!EngineTick() Line 69 C++
    UnrealEditor.exe!GuardedMain(const wchar_t * CmdLine) Line 188 C++
    UnrealEditor.exe!GuardedMainWrapper(const wchar_t * CmdLine) Line 123 C++
    UnrealEditor.exe!LaunchWindowsStartup(HINSTANCE     * hInInstance, HINSTANCE * hPrevInstance, char * formal, int nCmdShow, const wchar_t * CmdLine) Line 277 C++
    UnrealEditor.exe!WinMain(HINSTANCE     * hInInstance, HINSTANCE * hPrevInstance, char * pCmdLine, int nCmdShow) Line 317 C++
    [Inline Frame] UnrealEditor.exe!invoke_main() Line 102 C++
    UnrealEditor.exe!__scrt_common_main_seh() Line 288 C++
    kernel32.dll!00007ffb05ebe8d7() Unknown
    ntdll.dll!00007ffb0789c34c() Unknown