Crash in nvogl64.dll creating a vulkan ray-tracing pipeline with raygen shader using ray queries

Dioxyd · December 26, 2021, 7:02pm

Using drivers 497.09:

Gettting a crash in driver :

 	nvoglv64.dll!00007ffe5b675c85()	Unknown
 	nvoglv64.dll!00007ffe5b675166()	Unknown
 	nvoglv64.dll!00007ffe5b6c747f()	Unknown
 	nvoglv64.dll!00007ffe5b6bf1d0()	Unknown
 	nvoglv64.dll!00007ffe5b6be210()	Unknown
 	nvoglv64.dll!00007ffe5c7c08c1()	Unknown
 	nvoglv64.dll!00007ffe5c7c0b83()	Unknown
 	nvoglv64.dll!00007ffe5c7cca12()	Unknown
 	nvoglv64.dll!00007ffe5c7cb648()	Unknown

Full spir-v listing of the raygen shader:

OpCapability RayTracingKHR
OpCapability RuntimeDescriptorArray
OpCapability RayQueryKHR
OpCapability RayTraversalPrimitiveCullingKHR
OpExtension "SPV_KHR_ray_tracing"
OpExtension "SPV_EXT_descriptor_indexing"
OpExtension "SPV_KHR_ray_query"
%1 = OpExtInstImport "GLSL.std.450"
OpMemoryModel Logical GLSL450
OpEntryPoint RayGenerationNV %raygen_main "raygen_main" %g_rtas_table %g_views_table %g_constants %g_rw_textures %7 %8 %g_packed_views_table %g_render_constantsTable
OpSource HLSL 660
OpName %type_PushConstant_Constants "type.PushConstant.Constants"
OpMemberName %type_PushConstant_Constants 0 "v"
OpName %g_constants "g_constants"
OpName %type_2d_image "type.2d.image"
OpName %g_rw_textures "g_rw_textures"
OpName %type_StructuredBuffer_uint "type.StructuredBuffer.uint"
OpName %type_StructuredBuffer_View "type.StructuredBuffer.View"
OpName %View "View"
OpMemberName %View 0 "translation"
OpMemberName %View 1 "near"
OpMemberName %View 2 "rotation"
OpMemberName %View 3 "inv_translation"
OpMemberName %View 4 "inv_rotation"
OpMemberName %View 5 "view"
OpMemberName %View 6 "inv_view"
OpMemberName %View 7 "projection"
OpMemberName %View 8 "inv_projection"
OpMemberName %View 9 "view_projection"
OpMemberName %View 10 "inv_view_projection"
OpMemberName %View 11 "frustum"
OpName %g_views_table "g_views_table"
OpName %accelerationStructureNV "accelerationStructureNV"
OpName %g_rtas_table "g_rtas_table"
OpName %type_ConstantBuffer_RenderConstants "type.ConstantBuffer.RenderConstants"
OpMemberName %type_ConstantBuffer_RenderConstants 0 "min_shader_index"
OpMemberName %type_ConstantBuffer_RenderConstants 1 "shader_index_range"
OpMemberName %type_ConstantBuffer_RenderConstants 2 "shaders_total"
OpMemberName %type_ConstantBuffer_RenderConstants 3 "shaders_draw_counts_start"
OpMemberName %type_ConstantBuffer_RenderConstants 4 "shaders_instances_start"
OpMemberName %type_ConstantBuffer_RenderConstants 5 "shaders_draw_commands_start"
OpMemberName %type_ConstantBuffer_RenderConstants 6 "shaders_draw_parameters_start"
OpMemberName %type_ConstantBuffer_RenderConstants 7 "packed_mesh_instance_count"
OpMemberName %type_ConstantBuffer_RenderConstants 8 "packed_mesh_instances"
OpMemberName %type_ConstantBuffer_RenderConstants 9 "packed_view_count"
OpMemberName %type_ConstantBuffer_RenderConstants 10 "packed_views"
OpMemberName %type_ConstantBuffer_RenderConstants 11 "packed_mesh_shader_count"
OpMemberName %type_ConstantBuffer_RenderConstants 12 "packed_mesh_shaders"
OpMemberName %type_ConstantBuffer_RenderConstants 13 "packed_light_count"
OpMemberName %type_ConstantBuffer_RenderConstants 14 "packed_lights"
OpMemberName %type_ConstantBuffer_RenderConstants 15 "mesh_section_shader_index_channel"
OpMemberName %type_ConstantBuffer_RenderConstants 16 "mesh_shader_data_channel"
OpMemberName %type_ConstantBuffer_RenderConstants 17 "mesh_data"
OpMemberName %type_ConstantBuffer_RenderConstants 18 "ray_tracing_supported"
OpName %g_render_constantsTable "g_render_constantsTable"
OpName %g_packed_views_table "g_packed_views_table"
OpName %raygen_main "raygen_main"
OpName %rayQueryKHR "rayQueryKHR"
OpDecorate %8 BuiltIn LaunchIdNV
OpDecorate %7 BuiltIn LaunchSizeNV
OpDecorate %g_rw_textures DescriptorSet 2
OpDecorate %g_rw_textures Binding 0
OpDecorate %g_views_table DescriptorSet 6
OpDecorate %g_views_table Binding 0
OpDecorate %g_rtas_table DescriptorSet 7
OpDecorate %g_rtas_table Binding 0
OpDecorate %g_render_constantsTable DescriptorSet 5
OpDecorate %g_render_constantsTable Binding 0
OpDecorate %g_packed_views_table DescriptorSet 6
OpDecorate %g_packed_views_table Binding 0
OpDecorate %_arr_uint_uint_4 ArrayStride 4
OpMemberDecorate %type_PushConstant_Constants 0 Offset 0
OpDecorate %type_PushConstant_Constants Block
OpDecorate %_runtimearr_uint ArrayStride 4
OpMemberDecorate %type_StructuredBuffer_uint 0 Offset 0
OpMemberDecorate %type_StructuredBuffer_uint 0 NonWritable
OpDecorate %type_StructuredBuffer_uint Block
OpDecorate %_arr_v4float_uint_5 ArrayStride 16
OpMemberDecorate %View 0 Offset 0
OpMemberDecorate %View 1 Offset 12
OpMemberDecorate %View 2 Offset 16
OpMemberDecorate %View 3 Offset 32
OpMemberDecorate %View 4 Offset 44
OpMemberDecorate %View 5 Offset 60
OpMemberDecorate %View 5 MatrixStride 16
OpMemberDecorate %View 5 RowMajor
OpMemberDecorate %View 6 Offset 124
OpMemberDecorate %View 6 MatrixStride 16
OpMemberDecorate %View 6 RowMajor
OpMemberDecorate %View 7 Offset 188
OpMemberDecorate %View 7 MatrixStride 16
OpMemberDecorate %View 7 RowMajor
OpMemberDecorate %View 8 Offset 252
OpMemberDecorate %View 8 MatrixStride 16
OpMemberDecorate %View 8 RowMajor
OpMemberDecorate %View 9 Offset 316
OpMemberDecorate %View 9 MatrixStride 16
OpMemberDecorate %View 9 RowMajor
OpMemberDecorate %View 10 Offset 380
OpMemberDecorate %View 10 MatrixStride 16
OpMemberDecorate %View 10 RowMajor
OpMemberDecorate %View 11 Offset 444
OpDecorate %_runtimearr_View ArrayStride 524
OpMemberDecorate %type_StructuredBuffer_View 0 Offset 0
OpMemberDecorate %type_StructuredBuffer_View 0 NonWritable
OpDecorate %type_StructuredBuffer_View Block
OpMemberDecorate %type_ConstantBuffer_RenderConstants 0 Offset 0
OpMemberDecorate %type_ConstantBuffer_RenderConstants 1 Offset 4
OpMemberDecorate %type_ConstantBuffer_RenderConstants 2 Offset 8
OpMemberDecorate %type_ConstantBuffer_RenderConstants 3 Offset 12
OpMemberDecorate %type_ConstantBuffer_RenderConstants 4 Offset 16
OpMemberDecorate %type_ConstantBuffer_RenderConstants 5 Offset 20
OpMemberDecorate %type_ConstantBuffer_RenderConstants 6 Offset 24
OpMemberDecorate %type_ConstantBuffer_RenderConstants 7 Offset 28
OpMemberDecorate %type_ConstantBuffer_RenderConstants 8 Offset 32
OpMemberDecorate %type_ConstantBuffer_RenderConstants 9 Offset 36
OpMemberDecorate %type_ConstantBuffer_RenderConstants 10 Offset 40
OpMemberDecorate %type_ConstantBuffer_RenderConstants 11 Offset 44
OpMemberDecorate %type_ConstantBuffer_RenderConstants 12 Offset 48
OpMemberDecorate %type_ConstantBuffer_RenderConstants 13 Offset 52
OpMemberDecorate %type_ConstantBuffer_RenderConstants 14 Offset 56
OpMemberDecorate %type_ConstantBuffer_RenderConstants 15 Offset 60
OpMemberDecorate %type_ConstantBuffer_RenderConstants 16 Offset 64
OpMemberDecorate %type_ConstantBuffer_RenderConstants 17 Offset 68
OpMemberDecorate %type_ConstantBuffer_RenderConstants 18 Offset 72
OpDecorate %type_ConstantBuffer_RenderConstants Block
%uint = OpTypeInt 32 0
%uint_0 = OpConstant %uint 0
%uint_1 = OpConstant %uint 1
%float = OpTypeFloat 32
%uint_4 = OpConstant %uint 4
%uint_5 = OpConstant %uint 5
%v3float = OpTypeVector %float 3
%float_1 = OpConstant %float 1
%float_0 = OpConstant %float 0
%int = OpTypeInt 32 1
%int_0 = OpConstant %int 0
%int_10 = OpConstant %int 10
%int_2 = OpConstant %int 2
%float_0_5 = OpConstant %float 0.5
%v2float = OpTypeVector %float 2
%38 = OpConstantComposite %v2float %float_0_5 %float_0_5
%float_2 = OpConstant %float 2
%float_n2 = OpConstant %float -2
%41 = OpConstantComposite %v2float %float_2 %float_n2
%float_n1 = OpConstant %float -1
%43 = OpConstantComposite %v2float %float_n1 %float_1
%float_0_99000001 = OpConstant %float 0.99000001
%int_1 = OpConstant %int 1
%_arr_uint_uint_4 = OpTypeArray %uint %uint_4
%type_PushConstant_Constants = OpTypeStruct %_arr_uint_uint_4
%_ptr_PushConstant_type_PushConstant_Constants = OpTypePointer PushConstant %type_PushConstant_Constants
%type_2d_image = OpTypeImage %float 2D 2 0 0 2 Rgba32f
%_runtimearr_type_2d_image = OpTypeRuntimeArray %type_2d_image
%_ptr_UniformConstant__runtimearr_type_2d_image = OpTypePointer UniformConstant %_runtimearr_type_2d_image
%_runtimearr_uint = OpTypeRuntimeArray %uint
%type_StructuredBuffer_uint = OpTypeStruct %_runtimearr_uint
%_runtimearr_type_StructuredBuffer_uint = OpTypeRuntimeArray %type_StructuredBuffer_uint
%_ptr_StorageBuffer__runtimearr_type_StructuredBuffer_uint = OpTypePointer StorageBuffer %_runtimearr_type_StructuredBuffer_uint
%v4float = OpTypeVector %float 4
%mat4v4float = OpTypeMatrix %v4float 4
%_arr_v4float_uint_5 = OpTypeArray %v4float %uint_5
%View = OpTypeStruct %v3float %float %v4float %v3float %v4float %mat4v4float %mat4v4float %mat4v4float %mat4v4float %mat4v4float %mat4v4float %_arr_v4float_uint_5
%_runtimearr_View = OpTypeRuntimeArray %View
%type_StructuredBuffer_View = OpTypeStruct %_runtimearr_View
%_runtimearr_type_StructuredBuffer_View = OpTypeRuntimeArray %type_StructuredBuffer_View
%_ptr_StorageBuffer__runtimearr_type_StructuredBuffer_View = OpTypePointer StorageBuffer %_runtimearr_type_StructuredBuffer_View
%accelerationStructureNV = OpTypeAccelerationStructureKHR
%_runtimearr_accelerationStructureNV = OpTypeRuntimeArray %accelerationStructureNV
%_ptr_UniformConstant__runtimearr_accelerationStructureNV = OpTypePointer UniformConstant %_runtimearr_accelerationStructureNV
%type_ConstantBuffer_RenderConstants = OpTypeStruct %uint %uint %uint %uint %uint %uint %uint %uint %uint %uint %uint %uint %uint %uint %uint %uint %uint %uint %uint
%_runtimearr_type_ConstantBuffer_RenderConstants = OpTypeRuntimeArray %type_ConstantBuffer_RenderConstants
%_ptr_Uniform__runtimearr_type_ConstantBuffer_RenderConstants = OpTypePointer Uniform %_runtimearr_type_ConstantBuffer_RenderConstants
%v3uint = OpTypeVector %uint 3
%_ptr_Input_v3uint = OpTypePointer Input %v3uint
%void = OpTypeVoid
%62 = OpTypeFunction %void
%v2uint = OpTypeVector %uint 2
%_ptr_Function_accelerationStructureNV = OpTypePointer Function %accelerationStructureNV
%_ptr_PushConstant_uint = OpTypePointer PushConstant %uint
%_ptr_Uniform_uint = OpTypePointer Uniform %uint
%_ptr_StorageBuffer_uint = OpTypePointer StorageBuffer %uint
%_ptr_StorageBuffer_View = OpTypePointer StorageBuffer %View
%_ptr_UniformConstant_accelerationStructureNV = OpTypePointer UniformConstant %accelerationStructureNV
%bool = OpTypeBool
%_ptr_UniformConstant_type_2d_image = OpTypePointer UniformConstant %type_2d_image
%rayQueryKHR = OpTypeRayQueryKHR
%_ptr_Function_rayQueryKHR = OpTypePointer Function %rayQueryKHR
%g_constants = OpVariable %_ptr_PushConstant_type_PushConstant_Constants PushConstant
%g_rw_textures = OpVariable %_ptr_UniformConstant__runtimearr_type_2d_image UniformConstant
%g_views_table = OpVariable %_ptr_StorageBuffer__runtimearr_type_StructuredBuffer_View StorageBuffer
%g_rtas_table = OpVariable %_ptr_UniformConstant__runtimearr_accelerationStructureNV UniformConstant
%g_render_constantsTable = OpVariable %_ptr_Uniform__runtimearr_type_ConstantBuffer_RenderConstants Uniform
%g_packed_views_table = OpVariable %_ptr_StorageBuffer__runtimearr_type_StructuredBuffer_uint StorageBuffer
%8 = OpVariable %_ptr_Input_v3uint Input
%7 = OpVariable %_ptr_Input_v3uint Input
%uint_645 = OpConstant %uint 645
%raygen_main = OpFunction %void None %62
%74 = OpLabel
%75 = OpVariable %_ptr_Function_rayQueryKHR Function
%76 = OpVariable %_ptr_Function_accelerationStructureNV Function
%77 = OpLoad %v3uint %8
%78 = OpVectorShuffle %v2uint %77 %77 0 1
%79 = OpAccessChain %_ptr_PushConstant_uint %g_constants %int_0 %int_2
%80 = OpLoad %uint %79
%81 = OpAccessChain %_ptr_PushConstant_uint %g_constants %int_0 %int_0
%82 = OpLoad %uint %81
%83 = OpAccessChain %_ptr_Uniform_uint %g_render_constantsTable %82 %int_10
%84 = OpLoad %uint %83
%85 = OpAccessChain %_ptr_StorageBuffer_uint %g_packed_views_table %84 %int_0 %80
%86 = OpLoad %uint %85
%87 = OpAccessChain %_ptr_StorageBuffer_View %g_views_table %uint_1 %int_0 %86
%88 = OpLoad %View %87
%89 = OpCompositeExtract %mat4v4float %88 6
%90 = OpCompositeExtract %mat4v4float %88 8
%91 = OpConvertUToF %v2float %78
%92 = OpFAdd %v2float %91 %38
%93 = OpLoad %v3uint %7
%94 = OpVectorShuffle %v2uint %93 %93 0 1
%95 = OpConvertUToF %v2float %94
%96 = OpFDiv %v2float %92 %95
%97 = OpFMul %v2float %96 %41
%98 = OpFAdd %v2float %97 %43
%99 = OpCompositeExtract %float %98 0
%100 = OpCompositeExtract %float %98 1
%101 = OpCompositeConstruct %v4float %99 %100 %float_1 %float_1
%102 = OpVectorTimesMatrix %v4float %101 %90
%103 = OpVectorShuffle %v3float %102 %102 0 1 2
%104 = OpCompositeExtract %float %102 3
%105 = OpCompositeConstruct %v3float %104 %104 %104
%106 = OpFDiv %v3float %103 %105
%107 = OpCompositeConstruct %v4float %99 %100 %float_0_99000001 %float_1
%108 = OpVectorTimesMatrix %v4float %107 %90
%109 = OpVectorShuffle %v3float %108 %108 0 1 2
%110 = OpCompositeExtract %float %108 3
%111 = OpCompositeConstruct %v3float %110 %110 %110
%112 = OpFDiv %v3float %109 %111
%113 = OpCompositeExtract %float %106 0
%114 = OpCompositeExtract %float %106 1
%115 = OpCompositeExtract %float %106 2
%116 = OpCompositeConstruct %v4float %113 %114 %115 %float_1
%117 = OpVectorTimesMatrix %v4float %116 %89
%118 = OpVectorShuffle %v3float %117 %117 0 1 2
%119 = OpCompositeExtract %float %112 0
%120 = OpCompositeExtract %float %112 1
%121 = OpCompositeExtract %float %112 2
%122 = OpCompositeConstruct %v4float %119 %120 %121 %float_1
%123 = OpVectorTimesMatrix %v4float %122 %89
%124 = OpVectorShuffle %v3float %123 %123 0 1 2
%125 = OpFSub %v3float %124 %118
%126 = OpExtInst %v3float %1 Normalize %125
%127 = OpAccessChain %_ptr_UniformConstant_accelerationStructureNV %g_rtas_table %uint_0
%128 = OpLoad %accelerationStructureNV %127
OpStore %76 %128
%129 = OpLoad %accelerationStructureNV %76
OpRayQueryInitializeKHR %75 %129 %uint_645 %uint_1 %118 %float_0 %126 %float_1
%130 = OpRayQueryProceedKHR %bool %75
%131 = OpRayQueryGetIntersectionTypeKHR %uint %75 %uint_1
%132 = OpIEqual %bool %131 %uint_1
%133 = OpSelect %float %132 %float_1 %float_0
%134 = OpCompositeConstruct %v4float %133 %133 %133 %133
%135 = OpAccessChain %_ptr_PushConstant_uint %g_constants %int_0 %int_1
%136 = OpLoad %uint %135
%137 = OpAccessChain %_ptr_UniformConstant_type_2d_image %g_rw_textures %136
%138 = OpLoad %type_2d_image %137
OpImageWrite %138 %78 %134 None
OpReturn
OpFunctionEnd

Same shader without ray queries (for comparison):
(does not crash)

OpCapability RayTracingKHR
OpCapability RuntimeDescriptorArray
OpExtension "SPV_KHR_ray_tracing"
OpExtension "SPV_EXT_descriptor_indexing"
%1 = OpExtInstImport "GLSL.std.450"
OpMemoryModel Logical GLSL450
OpEntryPoint RayGenerationNV %raygen_main "raygen_main" %3 %g_constants %5 %g_rw_textures
OpSource HLSL 660
OpName %type_PushConstant_Constants "type.PushConstant.Constants"
OpMemberName %type_PushConstant_Constants 0 "v"
OpName %g_constants "g_constants"
OpName %type_2d_image "type.2d.image"
OpName %g_rw_textures "g_rw_textures"
OpName %raygen_main "raygen_main"
OpDecorate %5 BuiltIn LaunchIdNV
OpDecorate %3 BuiltIn LaunchSizeNV
OpDecorate %g_rw_textures DescriptorSet 2
OpDecorate %g_rw_textures Binding 0
OpDecorate %_arr_uint_uint_4 ArrayStride 4
OpMemberDecorate %type_PushConstant_Constants 0 Offset 0
OpDecorate %type_PushConstant_Constants Block
%uint = OpTypeInt 32 0
%float = OpTypeFloat 32
%uint_4 = OpConstant %uint 4
%float_0 = OpConstant %float 0
%int = OpTypeInt 32 1
%int_0 = OpConstant %int 0
%v4float = OpTypeVector %float 4
%17 = OpConstantComposite %v4float %float_0 %float_0 %float_0 %float_0
%int_1 = OpConstant %int 1
%_arr_uint_uint_4 = OpTypeArray %uint %uint_4
%type_PushConstant_Constants = OpTypeStruct %_arr_uint_uint_4
%_ptr_PushConstant_type_PushConstant_Constants = OpTypePointer PushConstant %type_PushConstant_Constants
%type_2d_image = OpTypeImage %float 2D 2 0 0 2 Rgba32f
%_runtimearr_type_2d_image = OpTypeRuntimeArray %type_2d_image
%_ptr_UniformConstant__runtimearr_type_2d_image = OpTypePointer UniformConstant %_runtimearr_type_2d_image
%v3uint = OpTypeVector %uint 3
%_ptr_Input_v3uint = OpTypePointer Input %v3uint
%void = OpTypeVoid
%25 = OpTypeFunction %void
%v2uint = OpTypeVector %uint 2
%_ptr_PushConstant_uint = OpTypePointer PushConstant %uint
%_ptr_UniformConstant_type_2d_image = OpTypePointer UniformConstant %type_2d_image
%g_constants = OpVariable %_ptr_PushConstant_type_PushConstant_Constants PushConstant
%g_rw_textures = OpVariable %_ptr_UniformConstant__runtimearr_type_2d_image UniformConstant
%5 = OpVariable %_ptr_Input_v3uint Input
%3 = OpVariable %_ptr_Input_v3uint Input
%raygen_main = OpFunction %void None %25
%29 = OpLabel
%30 = OpLoad %v3uint %5
%31 = OpVectorShuffle %v2uint %30 %30 0 1
%32 = OpAccessChain %_ptr_PushConstant_uint %g_constants %int_0 %int_1
%33 = OpLoad %uint %32
%34 = OpAccessChain %_ptr_UniformConstant_type_2d_image %g_rw_textures %33
%35 = OpLoad %type_2d_image %34
OpImageWrite %35 %31 %17 None
OpReturn
OpFunctionEnd

Dioxyd · December 26, 2021, 7:04pm

To be clear, this is a RayGen shader (compiled using DXC). Crashes in vkCreateRayTracingPipelinesKHR related to the use of “SPV_KHR_ray_query” as disabling the related pieces of code does not crash.

g.kaerts · January 5, 2022, 9:04pm

I’m hitting this exact same problem. Similar circumstances: ray gen shader compiled with DXC using ray queries. Crashes with a similar callstack.

nbickford · January 15, 2022, 12:54am

Hi Dioxyd and Gijs,

I had a go at reproducing this by assembling the SPIR-V shader and using it as the raygen shader in this sample. I managed to get a crash in vkCreateRayTracingPipelinesKHR, but it turned out that was due to me accidentally specifying the wrong entrypoint name (I was using the entrypoint name “main”, but the SPIR-V module uses the “raygen_main” entrypoint name!) Also, it looks like the call stack for that crash is slightly different.

I have a couple of questions that might help narrow things down:

Are you able to use the Vulkan Validation Layers, and do you get any validation errors from them? For instance, with the crash due to the wrong entrypoint name, I got the VUID-VkPipelineShaderStageCreateInfo-pName-00707 error (“No entrypoint found named main for stage VK_SHADER_STAGE_RAYGEN_BIT_KHR…”).
If you’re able to reproduce the crash, is it possible to get the address of the nvoglv64.dll module in memory (in Visual Studio, this is visible in the Debug > Windows > Modules window), and the new call stack? This is because modules are assigned roughly random locations in the 0x00007f… address space, so with the module starting address we can figure out what functions in the driver are being called.

Thanks!