Cerebras has published a detailed explanation of how it deploys trillion-parameter AI models across its wafer-scale hardware, using the open source model Kimi K3 as its example. In a post dated October 22, 2026, the company said it splits a large frontier model across multiple WSE-3T wafers to serve multi-trillion-parameter models at 1,000–2,000+ tokens per second.
The engineering challenge is memory: a single WSE-3T wafer carries 44 GB of on-chip memory, while trillion-parameter models need terabytes just for their weights. Cerebras' answer is to spread the model layer by layer over dozens of wafers and connect them into a pipeline.
How Cerebras splits Kimi K3 across 59 wafers
Kimi K3 is an open source model with roughly 3 trillion parameters, made up of 69 linear-attention layers and 24 dense layers. Stored at 4 bits each, the raw parameter payload comes to about 1.5 TB across 93 layers, or roughly 16 GB per layer, according to Cerebras.
Because each wafer has 44 GB of on-chip SRAM and a layer needs only about 16 GB, most layers fit entirely on a chip. Cerebras says that means model weights generally never move and layers do not have to be split across wafers — only activations travel between them.
Mapping those layers onto wafers gives 35 wafers for the linear-attention layers and 24 for the dense layers, for a total of 59 wafers. The runtime then routes tokens between wafers, passing activations over RDMA over Ethernet at roughly 2 microseconds per hop.
- Kimi K3: ~3 trillion parameters, 69 linear-attention layers, 24 dense layers
- Each WSE-3T wafer: 44 GB of on-chip SRAM
- Each model layer: roughly 16 GB at 4-bit precision
