I was just wondering if people buy these things specifically for CUDA or for double precision work, that’s why I am asking.
We specifically buy the Titans for our CUDA work as we wanted the power of full Maxwell as well as the extra memory. The jump up to Teslas is hard to justify until we hit problems or industries that need to guarantee memory correctness, but speed is very important so we go for a sensible balance. We primarily use single precision which is why I am so interested in the Titan X.
Unfortunately being in the UK I will have to wait till the resellers get them (thanks for the direction to AnandTech Gert-Jan) but I think I can manage that.
None of the titans are maxwell based except Titan X.
I put 4 x GK110 Titan into a 128GB Supermicro box specifically for single precision Maxwell’s and Navier-Stokes. Occasionally we run in double precision to check for SP truncation issues; so far single precision has been ideal for developing the kernels. The cluster can crunch doubles, if required.
Oops I meant Full Kepler, I’ve Maxwell on the brain at the moment.
Here is an observation I have found puzzling and that I hope folks here can shed some light on. In my mind, scientific computing and HPC have always been synonymous with double-precision computation. And in the early years of CUDA the message was loud and clear that without serious double precision performance, GPUs would not go very far in those markets.
However, now that GPUs and CUDA are established in these fields, most publications and most people I talk to seem to indicate that production codes running in the field are actually mostly or exclusively using single precision, and now certain fields are even clamoring for half precision (at increased throughput).
I am left to wonder what happened here. Was there a major shift across the totality of markets that are the big consumer of FLOPS, causing the focus to shift away from segments that required and still require double-precision computation and towards segments that do not really need more than single precision (possibly augmented by a few compensated sums, products, dot products in select places)?
Did people just believe that they needed double precision to generate valid results, but found that this actually was not the case upon conducting detailed analysis? I would have expected that as problem sizes increase, the need for increased precision would grow to counterbalance the accumulation of rounding errors in larger and longer-running simulations, and I do recall a number of publications in the early 2000s that seemed to indicate that there would be non-trivial demand for quadruple-precision computation by about now.
to throw a penny in your pond:
a) i think hpc as construct has changed - evolved
i think these days one needs to talk of hpc and super hpc perhaps
or then: hpc on a pc and hpc on a cluster/ supercomputer
b) i would like to think that (true) supercomputers are still measured and ranked in terms of DP, not SP
given the costs of these systems, i would certainly think that they would discard DP, if it mattered not
c) because of a), i think the sphere of applications has changed, and through this, the ‘distribution’ of SP/DP based applications
Ordered a Titan X online via Nivida today, and hopefully will get it by the weekend.
Will run a battery of tests and post the results.
As far as 32-bit vs. 64-bit, much depends on the application. For image processing/reconstruction 32 bit is just fine, and memory bandwidth is the main consideration.
For Monte Carlo simulations we use a combination, with most of the work done in 32 bit.
Talking to some of the individuals who submitted posters at GTC, quite a number of them use 64 bit for their calculations. They made it clear that for their applications they need that level of precision, and all used either Tesla K20 or Tesla K40.
"Talking to some of the individuals who submitted posters at GTC, quite a number of them use 64 bit for their calculations. They made it clear that for their applications they need that level of precision, and all used either Tesla K20 or Tesla K40. "
i would be interested in knowing whether these and such individuals purchase k20/ k40 for the sake of ecc, or for the supposed extra bells and whistles that comes with tesla or ‘server-class’ gpus
i do not see how it can be for the sake of DP flops
We do image reconstruction (ptychography) so after a bit of investigation we decided that single precision was sufficient (other factors were limiting accuracy). If we could get to where double precision was needed we would be laughing as that would give us ridiculous accuracy, but currently an iterative algorithm works fine with single precision. The cost in speed to go to double precision means far less iterations per unit time, meaning less convergence which leads to less accuracy.
Here’s my take on it. HPC originally settled on double precision because of two main factors. 1) From an application perspective, it is almost always safer to work in higher precision. 2) Double precision was cheap to implement in big OOO CPUs because the cost of a couple functional units was dwarfed by caches/scheduling logic.
- still applies, but 2) is no longer true. When 52-bit multipliers get replicated thousands of times on a big GPU, they start to take up a significant amount of the area and power budget, even if they are never used. This manifests itself indirectly in the pricing and performance characteristics of GPUs from all vendors, even if no one says it explicitly. I think that many developers have started looking carefully at their applications and started asking whether they are willing to pay more and run slower for full-rate double precision.
I understand NVIDIA has to shift product where the growth markets are, and it seems those are primarily dominated by single-precision computation at this time. I guess I have to look for good DP performance at reasonable prices elsewhere.
“…where the growth markets are, and it seems those are primarily dominated by single-precision computation at this time”
the 80/ 20 rule may refute the argument
take the latest announcements around nvlink at gtc, related to hpc and new super computers to be commissioned for example; in particular the partnership between ibm and nvidia for a new super computer
i believe this is the link:
[url]http://www.nvidia.com/object/exascale-supercomputing.html[/url]
one such super-computer is the equivalent of many lower-end customers
also, i simply can not conceive of such an installation being SP-based in principle; in essence, my hypothesis is that true (many node) super-computers would continue to the DP based, and DP ranked, for the foreseeable future
Oh, I completely agree on the need for DP in general-purpose super computers, which are mostly financed by governments and have diverse uses. As far as I can tell, that isn’t exactly a growth market, looking at the revenues of companies dedicated to that market segment. So I don’t think we are in disagreement.
In talking with Scott Le Grand of Amazon, he seems to be making do with single precision in his molecular simulation work. He’s using tricks like accumulating into a float and an int for when precision demands it.
As far as the neural network community goes, the trend seems to be in the direction of sub 32bit precision. So I wouldn’t be surprised to see for example in Pascal the ability to do a vector fp16 multiply but a 32 bit accumulate. So something like:
mad.v2.f32x2.f16x2 {f32a, f32b}, f16ab, f16cd, {f32a, f32b};
Then you could add a swizzle flag to avoid having to shift things around to get an outer product permutation. This kind of instruction would probably heavily tax the register banks, but is probably workable. This one instruction would turn a 7 Tflop chip into a 14 Tflop chip for deep learning applications.
Oh, and by the way, my fastest gemm kernels working at low entropy are able to run at about 7050 Gflops on TitanX. This is probably a first that Nvidia marketing actually matches the performance you can get out of the chip with usable code. If I had the above instruction, all memory operations could stay in 16 bit (including shared memory traffic) and that would reduce chip power enough for it to run at these speeds even with fully random input data.
As I recall Scott L. introduced the approach of single-precision computation combined with fixed-point accumulation into AMBER when sm_30 first appeared on the scene and DP performance in the form of sm_35 lagged time-wise.
At the time I did not really appreciate the foresight of this change but from today’s perspective it looks like a brilliant move. The AMBER folks get to publish new performance records whenever the latest SP-monster from NVIDIA is released. The AMBER benchmarking page was updated with Titan X results right after GTC.