Fast enough to think.

Fitting the model was never the problem. It's a dense 27-billion-parameter build, and with that much unified memory it loads with room to spare. Speed was the only question, and a reasoning turn costs about 221 seconds. Play a hundred games at that rate and the run is measured in days. So the whole serving stack became its own small project: how fast can one Apple-Silicon machine make a 27B think?

Multi-token prediction is speculative decoding: a small draft head proposes the next k=3 tokens, and the full model verifies all three in a single forward pass. When the guesses are good you get two-to-three tokens for the price of one pass, and the measured rate — ~60 tok/s — legitimately beats the memory-bandwidth roofline of ~51, because the roofline assumes one token per pass and MTP averages ~2.3–2.9.

The part I did not expect: the draft head gets more accurate as the context grows — acceptance rises from ~70% at 8K tokens to ~80% at 15.6K. Long agentic reasoning prompts, the ones I feared would be slowest, are its best case.

The numbers are almost rude about it: stock mlx-lm at 4-bit decodes 40.7 tok/s, and oMLX with MTP switched off is slower — about 35. You pay to install the fancy engine and go backward until you flip one setting. And it made the setup unforgiving. Stock mlx-lm cannot do MTP for this model at all — there is no draft-head module for it — so the speed lives entirely in oMLX, the Apple-Silicon server that supports it. And oMLX only loads the engine author's own quantisations: I learned that by downloading a 158 GB third-party build and watching it fail to load, not on bad weights but on a parameter-naming convention (shard_0 where the parser wanted shards.0). The pre-flight check is free — one look at the weight index — and I ran it only after the download. Now I run it first.

And the lever is only as strong as the hardware under it — which is what the table up top is really saying. MTP earns its ~1.4× on the M4 Pro mini too, but its ceiling is ~18 tok/s, not ~60, because decode is bound by memory bandwidth and the mini has roughly a third of the Ultra's. Neither MTP nor the ANE lifts that roof. The trick did not get worse; the roof came down.

Once decode was fast, the wait moved. At long context, prefill dominates — reading a 15.6K-token prompt costs about 39 seconds before the first token, against ~23 seconds of actually generating. Time-to-first-token is what you feel. So the second lever hands prefill to the Apple Neural Engine, the coprocessor that otherwise sits idle.

Honest about the size of it: with thinking on, a 16K call is decode-dominated, so the end-to-end win is about −13.5% wall time (51.0 s → 44.1 s), not the headline 21%. And the 0.5 offload fraction I set is not what runs — the engine caps it to 0.375 because the recurrent path stays off the approximate kernel. Two small lies the numbers told me, both worth catching.

None of the above arrives clean. The stack fell over in a dozen small ways, and each one looked, at first, like a result about the model.