AI chip · Cerebras Systems
Cerebras WSE-3 (Wafer-Scale Engine 3)
Third-generation wafer-scale AI processor; 900,000 cores on a single ~46,000 mm² die. The largest chip made. Fabricated by TSMC.
Market position
The wafer-scale engine: one 46,225 mm² chip that IS the die. 900,000 cores, 44 GB on-die SRAM. 4 trillion transistors, 57x an H100. Used by G42, Mayo Clinic, and GSK for training and inference-at-scale. Not a NVIDIA alternative in the same sense; a completely different architecture that only makes sense at pod scale.
Efficiency and power
How much work you get per watt and per dollar, and how much power a full rack draws.
- Perf per watt
- 2.72 FP16 TFLOPS/W62,500 TFLOPS ÷ 23000 W = 2.72
What fits in 44 GB
Which open-source LLMs run on one Cerebras WSE-3 (Wafer-Scale Engine 3), by precision. Weights only: add roughly 15% headroom for real serving. If a model does not fit at FP16, try INT8 or INT4 (smaller quality trade-off than most people expect).
| Model | Params | FP16 | INT8 | INT4 |
|---|---|---|---|---|
| Llama 3.1 8B dense | 8 B | 16 GB ✓ | 8 GB ✓ | 4 GB ✓ |
| Llama 3.1 70B dense | 70 B | 140 GB ✗ | 70 GB ✗ | 35 GB ✓ |
| Llama 3.1 405B dense | 405 B | 810 GB ✗ | 405 GB ✗ | 203 GB ✗ |
| Llama 3.3 70B dense | 70 B | 140 GB ✗ | 70 GB ✗ | 35 GB ✓ |
| DeepSeek V3 MoE (671B total, 37B active per token) | 671 B | 1342 GB ✗ | 671 GB ✗ | 336 GB ✗ |
| DeepSeek R1 MoE (671B total, 37B active per token) | 671 B | 1342 GB ✗ | 671 GB ✗ | 336 GB ✗ |
| Qwen 2.5 7B dense | 7 B | 14 GB ✓ | 7 GB ✓ | 4 GB ✓ |
| Qwen 2.5 72B dense | 72 B | 144 GB ✗ | 72 GB ✗ | 36 GB ✓ |
| Mixtral 8x7B MoE (46.7B total, 12.9B active per token) | 46.7 B | 93 GB ✗ | 47 GB ✗ | 23 GB ✓ |
| Mixtral 8x22B MoE (141B total, 39B active per token) | 141 B | 282 GB ✗ | 141 GB ✗ | 71 GB ✗ |
| Gemma 2 27B dense | 27 B | 54 GB ✗ | 27 GB ✓ | 14 GB ✓ |
| Command R+ dense | 104 B | 208 GB ✗ | 104 GB ✗ | 52 GB ✗ |
| Kimi K2 MoE (1T total, 32B active per token) | 1000 B | 2000 GB ✗ | 1000 GB ✗ | 500 GB ✗ |
Math: FP16 = params × 2 bytes; INT8 = params × 1 byte; INT4 = params × 0.5 bytes. MoE models sum every expert (full weights on disk), not the per-token active subset.
Common questions
How much does Cerebras WSE-3 (Wafer-Scale Engine 3) cost?
Cerebras WSE-3 (Wafer-Scale Engine 3) doesn't have a public launch list price. Vendors like this one usually price through direct sales rather than a public sheet. See list price per TFLOP for chips that do publish.
How much memory does Cerebras WSE-3 (Wafer-Scale Engine 3) have?
44 GB of on-die SRAM, running at 21,000,000 GB/s. Straight from the vendor datasheet. See chips with the most memory for context.
How much power does one Cerebras WSE-3 (Wafer-Scale Engine 3) draw?
23000 W at the chip. A full server draws more once you add CPU, memory, networking and cooling: see the rack power number above. Compare to other chips on perf-per-watt.
Which open-source LLMs fit on one Cerebras WSE-3 (Wafer-Scale Engine 3)?
Of 13 open-source LLMs we track, 2 fit at FP16, 3 at INT8, and 7 at INT4 (for example Llama 3.1 8B, Llama 3.1 70B and Llama 3.3 70B). Full table above with each model's memory need.
Key facts
- Class
- Compute SoC (AI accelerator)
- Designer
- Cerebras Systems
- Safety-critical?
- No (data-center inference / training)
- Record as of
- 2026-09-27
- Most recent source
- 2024-03-01 (across 1 source on this page)
Specifications (10 fields, click to expand)
Straight from the vendor datasheet. Dense throughput shown first; sparse (2:4) numbers in parentheses where the vendor publishes them. Full datasheet linked below.
- Process node
- TSMC N5
- Transistors
- 4000 B
- Die size
- 46225 mm²
- TDP
- 23000 W
- Memory
- 44 GB on-die SRAM
- Memory bandwidth
- 21,000,000 GB/s
- FP16 (dense)
- 62,500 TFLOPS
- Form factor
- wafer-scale (single-wafer system)
- Announced
- 2024-03-13
- Released
- 2024-04-01
Source: vendor datasheet
Foundry & process
- Fabbed at
- TSMC (all chips TSMC makes →)
- Process node
- TSMC N5
- Transistors
- 4000 B on 46225 mm² die
Compare with
Benchmarks
Published performance numbers by workload. Vendor datasheet figures where noted; independent measurements otherwise.
| Workload | Value | Unit | Source | As of |
|---|---|---|---|---|
| peak tflops fp16(Wafer-scale; 900k cores) | 125000 | TFLOPS | vendor | 2024-03-01 |
Sources
- Cerebras WSE-3Cerebras
Adoption rows appear as we confirm each chip-in-robot pairing from a public source. See every chip we track for the full catalog or the Cerebras Systems page.