NVIDIA Shader Compiler Consumes too much memory

Hey, so I’m working on a shader compiler that takes shader microcode from another language and converts it to SPIR-V.

I’ve noticed that when dealing with code that does a lot of branching (jump statements and such), NVIDIA’s shader compiler will eat up a lot of memory.
I’ve even had plenty of cases where it would consume so much that it would deadlock my entire computer, prompting a restart.

From the compiled shader code, I could see that NVIDIA’s compiler was taking the dumb route and duplicating a lot of code when taking branches. This can be fixed by including OpSelectionMerges, but unfortunately, there’s no metadata in the source shader code that describes high-level control flow constructs.
We could add a compiler that analyzes the code and adds in OpSelectionMerges, but my question is: Is NVIDIA going to address this behavior?

Could you provide the SPIR-V shader in question so we can take a look?


Mathias Schott

Disasm: http://pastebin.com/VtSyfViy

At the top is the source microcode, and then the SPIR-V disassembly follows.
For the jmp instructions, we generate an OpBranchConditional without any OpSelectionMerges, which seems to be problematic for the compiler.

Binary: https://dl.dropboxusercontent.com/u/5619434/Development/Xenia/shader_vk_4ED842B1EDA370D7.bin.frag

Thanks. We’ll take a look.


Some intermediate observations from our shader compiler team:

  • Our SPIR-V consumer requires "structured control flow" which was intended for all code originating from GLSL. We could probably do a work-around of sorts but ideally you'd extend your code generator since that would also reduce the risk of running into similar issues on other SPIR-V consumers. Speaking of which, have you tried running this on other Vulkan implementations?
  • spirv-disam (SPIRV-Tools v2016.0-dev spirv-1.1-rev1-26-g0d512bb)tool fails with
    %2573 = OpTypeVector %512 1024
       error: 319: Invalid opcode: 11008
  • Minor: .spv seems to be the canonical file extension for files containing raw SPIR-V content
  • Minor: The generator number is set to -1. Getting a “generator number” from the SPIR-V registry might be interesting at some point


  1. Yeah - I figured :P. Like I said though - this code doesn’t originate from GLSL but instead originates from microcode intended to run on another graphics chip entirely. And no - I haven’t been able to try this on other implementations due to the lack of alternate hardware.

It’s a bit odd though that the compiler is consuming enough memory to hang my computer. I’ve got 8GB of ram and I am currently idling at 5GB used. Xenia uses around 1GB during emulation putting me at around 6-7GB used - so if the compiler gets a wonky shader it’ll use up the rest of my free ram and hang my entire computer.

  1. Hmm - our copy of spirv-disasm appears to be working OK (source of the pastebin disasm).

Thanks Mathias and the compiler team

You call it odd, the spec calls it undefined behavior, so all bets are off ;)


Loops must be structured, having an OpLoopMerge instruction in their header.
Selections must be structured, having an OpSelectionMerge instruction in their header.

On a more serious note, a SPIR-V for Vulkan validation layer should catch this.

SPIR-V does “open the door” to non-structured control flow. Our compiler doesn’t allow anything but structured flow. So if this is really shader code it is required to be in CFG form.

Btw https://github.com/KhronosGroup/SPIRV-Cross also uses 10GB+ of memory when working on this .spv file…