The problem is that inference is memory bandwidth bound. And a Mac Studio M2 Ultra is equipped with 800 GB/s memory bandwidth. Contrast this to the LPDDR5 on Spark providing 273 GB/s.
That being said 300 t/s is truly remarkable and leaves me a bit hesitant, because, assuming there is enough prompt processing compute on Spark (and there is), 300 t/s on 800 GB/s memory would imply, everything else the same, that we could approach on Spark roughly 100 t/s for the same model, which is in reality almost one full order or magnitude off, even when accounting for speculative decoding. More than glad to be proven wrong.