I’m looking into using the CUDA platform for a project. We have multiple workstations with different Nvidia cards (at the moment a Tesla K20C, a GeForce GTX 750 Ti, a GeForce GTX 690, and a Quadro FX 1800), and I’ve been trying to determine compatibility. I know the Quadro FX is too old (compute capability of 1.1) to run the latest CUDA toolkit, but my understanding is that the others (compute capabilities of between 3.0 and 5.0) should run the latest version.
My question is, will they all execute/compile common code the same way? Are individual libraries (e.g. cuFFT or cuBLAS) dependent on compute capability, rather than just the version of the CUDA toolkit installed?
compiling code is not a function of which GPU you use. But in general, you should be able to write codes that run in a similar fashion on all of those (except the QFX). cufft and cublas for the most part don’t depend on a specific compute capability.
Last I checked, some CUDA libraries include architecture-specific code paths. As a consequence the results of some floating-point computations may not always be bit-identical across all supported architectures, for the same library version. The same applies to Intel’s optimized CPU libraries in my experience.
In practice any architecture-specific numerical differences that exist should be benign in the context of real-life use cases (e.g. error bounds are maintained across all platforms).
As noted, numerical differences between different versions of the same library are common, and a natural consequence of ongoing optimizations for performance as well as accuracy.