AI chip · AMD
AMD Instinct MI300X
CDNA 3 data-center accelerator with 192 GB HBM3, positioned as the primary alternative to Nvidia H100/H200 for LLM inference workloads. Fabricated by TSMC.
Part of 1 rack design.
Market position
MI300X is the chip that gave AMD real inference share against H100/H200: 192 GB HBM3 (2.4x H100 80GB) at $10-15K street. Meta, Microsoft (Copilot inference), Oracle and Vultr became public buyers. First real dual-source cloud GPU since Volta.
Efficiency and power
How much work you get per watt and per dollar, and how much power a full rack draws.
- Perf per watt
- 3.49 FP8 TFLOPS/W2,614 TFLOPS ÷ 750 W = 3.49
- List $ per FP8 TFLOP
- $5.74$15,000 ÷ 2,614 TFLOPS = $5.74
- Rack power
- 6 kW per AMD Instinct MI300X Platform (8-GPU OAM baseboard) (~750 W per chip)6 kW ÷ 8 chips (whole-system, includes CPU, memory, NICs, cooling)
- Cooling
- airDepends on the rack design
What fits in 192 GB
Which open-source LLMs run on one AMD Instinct MI300X, by precision. Weights only: add roughly 15% headroom for real serving. If a model does not fit at FP16, try INT8 or INT4 (smaller quality trade-off than most people expect).
| Model | Params | FP16 | INT8 | INT4 |
|---|---|---|---|---|
| Llama 3.1 8B dense | 8 B | 16 GB ✓ | 8 GB ✓ | 4 GB ✓ |
| Llama 3.1 70B dense | 70 B | 140 GB ✓ | 70 GB ✓ | 35 GB ✓ |
| Llama 3.1 405B dense | 405 B | 810 GB ✗ | 405 GB ✗ | 203 GB ✗ |
| Llama 3.3 70B dense | 70 B | 140 GB ✓ | 70 GB ✓ | 35 GB ✓ |
| DeepSeek V3 MoE (671B total, 37B active per token) | 671 B | 1342 GB ✗ | 671 GB ✗ | 336 GB ✗ |
| DeepSeek R1 MoE (671B total, 37B active per token) | 671 B | 1342 GB ✗ | 671 GB ✗ | 336 GB ✗ |
| Qwen 2.5 7B dense | 7 B | 14 GB ✓ | 7 GB ✓ | 4 GB ✓ |
| Qwen 2.5 72B dense | 72 B | 144 GB ✓ | 72 GB ✓ | 36 GB ✓ |
| Mixtral 8x7B MoE (46.7B total, 12.9B active per token) | 46.7 B | 93 GB ✓ | 47 GB ✓ | 23 GB ✓ |
| Mixtral 8x22B MoE (141B total, 39B active per token) | 141 B | 282 GB ✗ | 141 GB ✓ | 71 GB ✓ |
| Gemma 2 27B dense | 27 B | 54 GB ✓ | 27 GB ✓ | 14 GB ✓ |
| Command R+ dense | 104 B | 208 GB ✗ | 104 GB ✓ | 52 GB ✓ |
| Kimi K2 MoE (1T total, 32B active per token) | 1000 B | 2000 GB ✗ | 1000 GB ✗ | 500 GB ✗ |
Math: FP16 = params × 2 bytes; INT8 = params × 1 byte; INT4 = params × 0.5 bytes. MoE models sum every expert (full weights on disk), not the per-token active subset.
Common questions
How much does AMD Instinct MI300X cost?
Launch list price was $15,000 when it shipped in 2024-01-01. Street prices swing with supply and how old the generation is. Cloud rental rates vary widely by provider. See list price per TFLOP for the full ranking.
How much memory does AMD Instinct MI300X have?
192 GB of HBM3, running at 5,300 GB/s. Straight from the vendor datasheet. See chips with the most memory for context.
How much power does one AMD Instinct MI300X draw?
750 W at the chip. A full server draws more once you add CPU, memory, networking and cooling: see the rack power number above. Compare to other chips on perf-per-watt.
Which open-source LLMs fit on one AMD Instinct MI300X?
Of 13 open-source LLMs we track, 7 fit at FP16, 9 at INT8, and 9 at INT4 (for example Llama 3.1 8B, Llama 3.1 70B and Llama 3.3 70B). Full table above with each model's memory need.
Key facts
- Class
- Compute SoC (AI accelerator)
- Designer
- AMD
- Safety-critical?
- No (data-center inference / training)
- Record as of
- 2026-09-27
- Most recent source
- 2023-12-01 (across 3 sources on this page)
Specifications (14 fields, click to expand)
Straight from the vendor datasheet. Dense throughput shown first; sparse (2:4) numbers in parentheses where the vendor publishes them. Full datasheet linked below.
- Process node
- TSMC N5 + N6 (chiplet)
- Transistors
- 153 B
- CUDA cores
- 19,456
- TDP
- 750 W
- Memory
- 192 GB HBM3
- Memory bandwidth
- 5,300 GB/s
- PCIe
- Gen 5 x16 (896 GB/s)
- FP16 (dense)
- 1,307 TFLOPS
- FP8 (dense)
- 2,614 TFLOPS
- INT8 (dense)
- 2,614 TOPS
- Form factor
- OAM
- Announced
- 2023-12-06
- Released
- 2024-01-01
- Launch price
- $15,000 (list)
Source: vendor datasheet
Generation
Foundry & process
- Fabbed at
- TSMC (all chips TSMC makes →)
- Process node
- TSMC N5 + N6 (chiplet)
- Transistors
- 153 B
Rack designs that use it
Whole-rack systems buyers order in bulk (NVL72, HGX, TPU pods) that ship with AMD Instinct MI300X inside.
Compare with
Benchmarks
Published performance numbers by workload. Vendor datasheet figures where noted; independent measurements otherwise.
Sources
Adoption rows appear as we confirm each chip-in-robot pairing from a public source. See every chip we track for the full catalog or the AMD page.