The CUDA Toolkit installer seems to have installed a new driver without removing the old one. Now what?

After updating from CUDA Toolkit version 11.5.0 to 11.5.1 with the installer on Windows 10, my development environment is completely broken. I cannot get anything to build with CMake in CLion, and I get a ton of strange PTX runtime errors in Visual Studio. I think this may be because there are two versions of the Nvidia Graphics Driver installed. I only see version 496.13 in Apps & Features (and Programs and Features), but in the Nvidia Control Panel, I see version 472.42 instead. How do I proceed from here?

Update: I reinstalled every Nvidia program (including drivers) at once with the Toolkit installer. While this did fix the discrepancy between the Nvidia Control Panel and Apps/Programs and Features, it did not fix the issue with my environment. As of now, it seems I have no choice but to reinstall Windows.

NVIDIA CUDA setup (and even NVIDIA driver setup because they are based on the same proprietary installer written using .Net Framework) will mess up your PATH environment variable. I have reported this bug to NVIDIA in April 2019 if I remember correctly. I am not sure if they ever fixed it or if there is a regression.

Before reinstalling Windows, try fixing your PATH enviroment variable. You can also try repairing / reinstalling CMake, and reinstalling Visual Studio integration.