It’s a good idea that would work, but the interesting question is whether it’d be worth the complexity.
It’d require hardware level changes… since if different SMs have different clock rates (idle versus full power) they would need to separate their own signal lines into a new chip level domain. Merely disabling the cores sounds good too but that means you also need independent POWER domains on the chip… and the ability for signal lanes to deal with neighbors which may or may not be active. It gets complex.
However, it’s not impossible and in the latest i7 chips from Intel do many of these tricks with special power gating per core.
GPUs will likely do more clock/power gating in the future too. Likely the priority will stay chip-wide for a long while (not per-SM), since it’s much more important to control idle power than it is to try to fine-grain optimize partial shutdowns. And in fact the parallelism of GPU tasks means that it’s almost always easier to use the whole chip for a shorter time than it is to dynamically shut down variable numbers of SMs. On the CPU, threads can’t be broken up and shared among cores… GPU tasks are fundamentally designed to massively parallellize.
We can see that the GPUs already do shut down part of themselves when running CUDA apps… likely the rasterizers and graphics-specific hardware is quiescent even when the SMs and memory are cranking at full clock rate running CUDA programs. This hypthesis is strongly supported by measuring GPU wattage during heavy graphics tasks (for example, worst case FurMark on a GTX480, using 300 watts) and heavy full power CUDA programs on the same GTX480 using only 200 watts.
You can also see optimizations like Optimus for laptops which dynamically power down the GPU completely (0 watts!). That’s extremely elegant, getting a great power savings for the common idle case, by letting a separate low capability, but low-wattage chip handle 2D display.