Random grey squares flickering on DX11

I’m working on a DX11 project where we sometime render small 3D scenes to textures that we later display as thumbnails in a UI. When we bake more than one scene to texture in the same frame, some of the thumbnails will appear with many random 4x4 or 8x8 grey squares.

This problem seems to be specific to Nvidia GPUs. I was able to see the problem on a GeForce 1070 and a GeForce 1060 with recent drivers. When using the reference rasterizer, the warp device or an AMD GPU, the thumbnails are baked properly.

I did a capture of the baking process with Nsight and exported it to C++. When I launch the generated C++ code, the location of random grey squares will changes every frame.

I did a pass to strip most of the code that didn’t look necessary to reproduce the bug. I was able to remove every operation involving compute shaders and unordered access views and the problem would still be present. At this point, it looks like the driver is missing an implicit synchronization or cache operation when we do a specific sequence of render target/depth stencil changes and draw calls.

Due to the nature of the project, I can’t really share the code and data here but I can send them privately.