Stencil HBAO+ on DX12

Is it possible to enable stencil test in HBAO with DX12?

I work on a VR game that uses HBAO+. The pixels in the corners are not visible to the player and we use a stencil mesh to early exit the shaders. Looking at the capture, HBAO+ contains mostly fullscreen pixel shaders. If we could early exit in these locations with stencil tests or early z, we could save 10% gpu time.


Hello @lmira and welcome to the NVIDIA developer forums!

I cannot answer this from the top of my head, but I will try to find it out for you.


Hi again!

HBAO happens on driver level after early-z w/ depth test and similar, so any pixels excluded at that stage should also not be affected by the later stage pixel shaders.

What capture do you mean?

Hi Markus, I did a PIX capture. In this example HBAO takes ~1ms per eye.
It runs ~20 full screen passes with 1 large triangle covering the whole viewport.
I believe HBAO does not take advantage of the stencil mesh:

  • Depth and Stencil test are disable in all HBAO passes
  • I don’t see any performance improvement with/without stencil mesh enabled.

Looking at the file HBAOPlus/GFSDK_SSAO.h at master · NVIDIAGameWorks/HBAOPlus · GitHub,
it seems we can set a custom depth stencil state for DX11, but not for DX12?