Open-weight LLM comparison
Llama 3.1 405B vs Mixtral 8x7B
Side-by-side
Straight from each model's release page and Hugging Face card. Memory shown is for the weights alone: add roughly 15% for real serving. Bold column marks the more parameters and the smaller memory footprint.
| Field | Llama 3.1 405B | Mixtral 8x7B |
|---|---|---|
| Total parameters | 405 B | 46.7 B |
| Active parameters (MoE) | — | 12.9 B |
| Architecture | dense | MoE (46.7B total, 12.9B active per token) |
| Vendor | Meta | Mistral AI |
| License | Llama 3.1 Community License | Apache 2.0 |
| Released | 2024-07-23 | 2023-12-11 |
| Weights @ FP16 | 810 GB | 93 GB |
| Weights @ INT8 | 405 GB | 47 GB |
| Weights @ INT4 | 203 GB | 23 GB |
Which is smarter (published benchmarks)
Vendor-reported quality scores on the standard leaderboards. Bold column marks the higher score on the same test.
| Benchmark | Llama 3.1 405B | Mixtral 8x7B |
|---|---|---|
| MMLU (5-shot) | 88.6 | 70.6 |
| HumanEval | 89.0 | 40.2 |
| MATH (0-shot) | 73.8 | not published |
| GPQA | 51.1 | not published |
| MATH (maj@4) | not published | 28.4 |
Sources: Llama 3.1 405B model card · Mixtral 8x7B model card. Benchmark methodology and prompt template can shift these numbers by several points, so treat these as relative rankings, not absolute scores.
Common questions
Llama 3.1 405B vs Mixtral 8x7B: which is bigger?
Llama 3.1 405B has more parameters (Llama 3.1 405B: 405 B; Mixtral 8x7B: 46.7 B). More parameters usually means higher ceiling on capability and higher memory requirement, though MoE architectures decouple total parameters from per-token compute.
Llama 3.1 405B vs Mixtral 8x7B: which is newer?
Llama 3.1 405B released 2024-07-23; Mixtral 8x7B released 2023-12-11.
Llama 3.1 405B vs Mixtral 8x7B: which needs less memory to serve?
Mixtral 8x7B needs less HBM. Weights-only footprint at FP16: Llama 3.1 405B 810 GB; Mixtral 8x7B 93 GB. Half those numbers at INT8, quarter at INT4. Real serving adds 10-30% for KV cache.
Llama 3.1 405B vs Mixtral 8x7B: which license is more permissive?
Llama 3.1 405B: Llama 3.1 Community License. Mixtral 8x7B: Apache 2.0. Apache 2.0 and MIT allow unrestricted commercial use; Llama Community License allows commercial use but restricts training larger models on outputs; CC-BY-NC and vendor-specific licenses (Qwen 72B, Gemma) have narrower terms. Check the model card for the exact clauses.
See also: every LLM comparison · Llama 3.1 405B full page · Mixtral 8x7B full page.