AI chip · NVIDIA
NVIDIA A100 Tensor Core GPU
Ampere-generation data-center GPU (SXM4 / PCIe), 40/80 GB HBM2e. The workhorse of AI training 2020-2023 and still dominant in the deployed base. TSMC 7nm.
Market position
A100 was the GPU GPT-3, ChatGPT training, Stable Diffusion and the entire 2020-2022 model-scaling era ran on. Its 80 GB HBM2e SKU made it the first GPU where a large language model actually fit. Superseded by H100 for training; still shipping for older Kubernetes clusters and academic labs.
Efficiency and power
How much work you get per watt and per dollar, and how much power a full rack draws.
- Perf per watt
- 0.78 FP16 TFLOPS/W312 TFLOPS ÷ 400 W = 0.78
- List $ per FP16 TFLOP
- $32.05$10,000 ÷ 312 TFLOPS = $32.05
What fits in 80 GB
Which open-source LLMs run on one NVIDIA A100 Tensor Core GPU, by precision. Weights only: add roughly 15% headroom for real serving. If a model does not fit at FP16, try INT8 or INT4 (smaller quality trade-off than most people expect).
| Model | Params | FP16 | INT8 | INT4 |
|---|---|---|---|---|
| Llama 3.1 8B dense | 8 B | 16 GB ✓ | 8 GB ✓ | 4 GB ✓ |
| Llama 3.1 70B dense | 70 B | 140 GB ✗ | 70 GB ✓ | 35 GB ✓ |
| Llama 3.1 405B dense | 405 B | 810 GB ✗ | 405 GB ✗ | 203 GB ✗ |
| Llama 3.3 70B dense | 70 B | 140 GB ✗ | 70 GB ✓ | 35 GB ✓ |
| DeepSeek V3 MoE (671B total, 37B active per token) | 671 B | 1342 GB ✗ | 671 GB ✗ | 336 GB ✗ |
| DeepSeek R1 MoE (671B total, 37B active per token) | 671 B | 1342 GB ✗ | 671 GB ✗ | 336 GB ✗ |
| Qwen 2.5 7B dense | 7 B | 14 GB ✓ | 7 GB ✓ | 4 GB ✓ |
| Qwen 2.5 72B dense | 72 B | 144 GB ✗ | 72 GB ✓ | 36 GB ✓ |
| Mixtral 8x7B MoE (46.7B total, 12.9B active per token) | 46.7 B | 93 GB ✗ | 47 GB ✓ | 23 GB ✓ |
| Mixtral 8x22B MoE (141B total, 39B active per token) | 141 B | 282 GB ✗ | 141 GB ✗ | 71 GB ✓ |
| Gemma 2 27B dense | 27 B | 54 GB ✓ | 27 GB ✓ | 14 GB ✓ |
| Command R+ dense | 104 B | 208 GB ✗ | 104 GB ✗ | 52 GB ✓ |
| Kimi K2 MoE (1T total, 32B active per token) | 1000 B | 2000 GB ✗ | 1000 GB ✗ | 500 GB ✗ |
Math: FP16 = params × 2 bytes; INT8 = params × 1 byte; INT4 = params × 0.5 bytes. MoE models sum every expert (full weights on disk), not the per-token active subset.
Export controls
US Commerce Department rules that restrict where NVIDIA A100 Tensor Core GPU can be sold. Effective dates and full text below, with the official Federal Register filings and news coverage for each.
The US Bureau of Industry and Security's Oct 7 2022 interim final rule restricted export of advanced AI accelerators to China, keyed to a performance threshold (>=4,800 TOPS or >=600 GB/s interconnect). Directly blocked NVIDIA A100 and H100 sales to Chinese entities without a license, triggering the A800 and H800 China-specific variants NVIDIA released within weeks.
Sources: BIS press release, Oct 7 2022 · Federal Register, Oct 13 2022 (rule text)
Common questions
How much does NVIDIA A100 Tensor Core GPU cost?
Launch list price was $10,000 when it shipped in 2020-05-14. Street prices swing with supply and how old the generation is. Cloud rental rates vary widely by provider. See list price per TFLOP for the full ranking.
How much memory does NVIDIA A100 Tensor Core GPU have?
80 GB of HBM2e, running at 2,039 GB/s. Straight from the vendor datasheet. See chips with the most memory for context.
How much power does one NVIDIA A100 Tensor Core GPU draw?
400 W at the chip. A full server draws more once you add CPU, memory, networking and cooling: see the rack power number above. Compare to other chips on perf-per-watt.
Which open-source LLMs fit on one NVIDIA A100 Tensor Core GPU?
Of 13 open-source LLMs we track, 3 fit at FP16, 7 at INT8, and 9 at INT4 (for example Llama 3.1 8B, Llama 3.1 70B and Llama 3.3 70B). Full table above with each model's memory need.
Is NVIDIA A100 Tensor Core GPU subject to US export controls?
Yes. 1 US Commerce Department rule restrict where this chip can be sold, most recently BIS Oct 7 2022 export controls on advanced AI chips to China (2022-10-07). Details and effective dates in the export-control section above.
Key facts
- Class
- Compute SoC (AI accelerator)
- Designer
- NVIDIA
- Safety-critical?
- No (data-center inference / training)
- Record as of
- 2026-09-27
- Most recent source
- 2020-11-01 (across 2 sources on this page)
Specifications (17 fields, click to expand)
Straight from the vendor datasheet. Dense throughput shown first; sparse (2:4) numbers in parentheses where the vendor publishes them. Full datasheet linked below.
- Process node
- TSMC N7
- Transistors
- 54 B
- Die size
- 826 mm²
- CUDA cores
- 6,912
- Tensor cores
- 432 (3rd gen (Ampere))
- TDP
- 400 W
- Memory
- 80 GB HBM2e
- Memory bandwidth
- 2,039 GB/s
- PCIe
- Gen 4 x16 (600 GB/s)
- FP32
- 19.5 TFLOPS
- TF32 (dense)
- 156 TFLOPS
- FP16 (dense)
- 312 TFLOPS (624 sparse)
- INT8 (dense)
- 624 TOPS (1,248 sparse)
- Form factor
- SXM4
- Announced
- 2020-05-14
- Released
- 2020-05-14
- Launch price
- $10,000 (list)
Source: vendor datasheet
Generation
Foundry & process
- Fabbed at
- TSMC (all chips TSMC makes →)
- Process node
- TSMC N7
- Transistors
- 54 B on 826 mm² die
Compare with
Benchmarks
Published performance numbers by workload. Vendor datasheet figures where noted; independent measurements otherwise.
Sources
- NVIDIA A100Nvidia
Adoption rows appear as we confirm each chip-in-robot pairing from a public source. See every chip we track for the full catalog or the NVIDIA page.