Mixture of experts
A sparse model architecture: the network holds many expert sub-networks and routes each token through a few of them, so only a small share of the parameters is active per token. A dense model reads all of its parameters for every token.
On local hardware this changes the speed maths, because generation is limited by how many bytes are read per token (memory-bandwidth-bound-generation). Kai's figures (Sept 2026), from one M4 Pro machine: a 35B mixture-of-experts model with about 3B active ran at 78.8 tokens per second, and a dense 27B at 14.7, so the bigger model was about five times faster. The 35B sparse model at 4-bit takes about 17 GB, which is why Kai puts the floor for local AI on a Mac mini at 24 GB and recommends 32.
Memory still has to hold the whole model, so the 35B sparse model needs its full 17 GB even though each token reads only a fraction of it.
Related: model-parameters, open-weights.
Source: report