NVIDIA · Blackwell · Layer 2
NVIDIA B200
The default frontier accelerator. If you have no specific reason to choose otherwise, this is what the market chose.
rent Rentable by the hour from multiple clouds; purchasable at scale.
Specifications
- Memory
- 192 GB HBM3E
- Bandwidth
- 8 TB/s
- Compute
- FP4 and FP8 sparse; vendor figures vary by configuration
- Power
- Up to 1,000 W per GPU in liquid-cooled configurations
- Interconnect
- NVLink 5
- Software
- CUDA
- Form factor
- accelerator
- Workload
- both
Verified 2026-09-06. Fields reading “not published” are exactly that — we do not estimate a figure a vendor withholds.
The catch
192 GB is less memory than AMD's MI355X at 288 GB, and for memory-bound inference that gap is the whole argument.
The economics
CUDA maturity is the real product. The chip is competitive; the software moat is why it wins procurement.
Cost per million tokens is the metric that decides most real purchases, and there is no neutral benchmark for it — every published figure comes from a company selling one side of the comparison. Read all of them, ours included, as claims rather than measurements.
The power question underneath this
A rack of frontier accelerators draws 120–200 kW against a 2026 average of about 27 kW, and the US grid interconnection queue exceeds 2,600 GW with waits approaching five years. Whether you can energise this part is now a harder question than whether you can buy it.
Layer 1 — Energy · The interconnection queue · Rack power density · Tokens per watt
Compared against
- NVIDIA B200 vs AMD Instinct MI355X — The memory argument, stated cleanly: 192 GB of CUDA against 288 GB of ROCm.Compare with AMD Instinct MI355X
- Meta MTIA v2 vs NVIDIA B200 — The custom-silicon question in one line: every MTIA Meta deploys is a B200 NVIDIA does not sell.Compare with Meta MTIA v2
Related silicon
- NVIDIA GB200 NVL72 — 13.5 TB HBM3E across the rack, rent
- NVIDIA GB300 NVL72 — Higher HBM3E capacity than GB200; per-rack figure not consistently published, rent
- NVIDIA H100 — 80 GB HBM3, rent
- NVIDIA H200 — 141 GB HBM3E, rent
- NVIDIA RTX PRO 6000 Blackwell — 96 GB GDDR7, buy
- AMD Instinct MI300X — 192 GB HBM3, rent