Chip registry

AI rack designs

When a hyperscaler buys AI compute, they usually buy it a rack at a time. These are the named designs (NVL72, HGX, OAM platforms, TPU pods, Trainium UltraServers), and the chips inside each.

NVIDIA GB200 NVL72

rack

72 Blackwell GPUs + 36 Grace CPUs in one liquid-cooled rack, sharing a single NVLink 5 domain. The current flagship training rack.

36 chips per instance · 132 kW · water-cooled · GA 2024-12-01

NVIDIA HGX H100 (8-GPU baseboard)

platform

The 8-GPU H100 SXM5 baseboard that anchored the entire 2023-2024 AI-training build-out. The generation the world learned to buy in bulk.

8 chips per instance · 10.2 kW · air-cooled · GA 2022-10-13

NVIDIA HGX H200 (8-GPU baseboard)

platform

H100's memory-refresh sibling: same footprint, 141 GB HBM3e per GPU (+76%) and 4.8 TB/s bandwidth. Inference-oriented shops upgraded here.

8 chips per instance · 10.2 kW · air-cooled · GA 2024-03-01

NVIDIA HGX B200 (8-GPU baseboard)

platform

The Blackwell-generation 8-GPU baseboard for buyers who wanted a familiar HGX form-factor instead of committing to NVL72's rack-scale.

8 chips per instance · 14.3 kW · hybrid-cooled · GA 2024-11-01

NVIDIA DGX SuperPOD H100 (32-node reference)

supercluster

32 DGX H100 nodes = 256 GPUs, 32 InfiniBand switches, one full-fat rail-optimised training pod. The AI startup unit-of-currency in 2023.

256 chips per instance · 480 kW · air-cooled · GA 2023-01-01

AMD Instinct MI300X Platform (8-GPU OAM baseboard)

platform

AMD's 8-way OAM answer to HGX H100/H200: 1.5 TB HBM3 total, priced to unseat NVIDIA on memory-bound inference. The Meta / Microsoft / Oracle purchase.

8 chips per instance · 6 kW · air-cooled · GA 2024-01-01

AMD Instinct MI325X Platform (8-GPU OAM baseboard)

platform

The MI325X refresh: 256 GB HBM3e per GPU, 2 TB per baseboard. AMD's tightest inference-per-dollar pitch heading into 2025.

8 chips per instance · 8 kW · air-cooled · GA 2025-01-01

Google TPU v5p Pod (8,960 chips)

supercluster

Google's single-pod ceiling under v5p: 8,960 accelerators in one all-to-all ICI domain. The unit-of-scale for Gemini-generation training.

8,960 chips per instance · water-cooled · GA 2024-01-01

Google TPU v6e (Trillium) Pod-256

pod

The inference-optimised Trillium pod slice at 256 chips. Google Cloud's default Gemini-serving unit.

256 chips per instance · water-cooled · GA 2024-12-01

AWS Trainium 2 UltraServer

ultraserver

AWS's answer to NVL72: 64 Trainium 2 chips lashed together with NeuronLink. Anthropic's Project Rainier compute unit.

64 chips per instance · water-cooled · GA 2025-01-01

Huawei Atlas 900 A3 SuperCluster (Ascend 910B)

supercluster

The Chinese ceiling under US export controls: 8,192 domestically fabbed Ascend 910B accelerators in one all-to-all HCCS fabric.

8,192 chips per instance · water-cooled · GA 2025-01-01