Anaconda keeps freezing with driver versions 5xx

Hi, I have upgraded the nvidia driver to 535 and my anaconda started freezing. By freezing I mean any anaconda command (conda create, conda activate, etc) would cause a ~30s freeze on the whole system. This in especially annoying in VS code, where each run opens a new terminal and freezes the system. If I revert to 470 or uninstall the nvidia drivers, conda works perfectly fine.

Specs:

Ubuntu 20
2x 2080Ti
AMD® Ryzen threadripper 2950x 16-core processor × 32
64GB RAM

nvidia-bug-report.log.gz (664.6 KB)

Xid 109, seems to be a bug introduced with driver 525:
https://forums.developer.nvidia.com/t/multiple-cuda-rtx-vulkan-application-crashing-with-xid-13-109-errors/235459