We have been tending to “side-by-side” install all the CUDA versions of a given major series - for instance, for CUDA 11, we install 11.0, 11.1, …, 11.8. Do we really need to do that, or is just the latest CUDA version in a major release all we need (anotherwords, are they backwards compatible?) We sometimes need to run old code compiled with an older version of CUDA, so at a minimum, we’d need 10.x, 11.x and now 12.x. Can I just install the latest/last version of 10.x, 11.x and 12.x and be covered?
For an application that is statically linked to needed libraries, this should be a non-issue. The only thing that application “needs” is a recent enough GPU driver. If you are installing the latest CUDA toolkit (e.g. 12.x currently), and doing a full/proper install, you are installing the latest GPU driver. So you are covered there.
For applications that are dynamically linked to one or more libraries, I think this should generally work at least for CUDA 11.x and CUDA 12.x. I have tested those myself to confirm that this works when libcudart.so is dynamically linked (dynamic linking of libcudart is no longer the default for nvcc compilation).
It’s tough to be exhaustive for all possible scenarios. CUDA can get used in a variety of different ways, and some people assume that libraries like cudnn, TensorRT, NVML, and many others, are part of “CUDA”. They aren’t, and I don’t know of claims made about those.
This is the relevant doc section. Note that most of the claims there start with CUDA 11 and forward.