NVIDIA · Hopper · Layer 2
NVIDIA H100
The reference point everything else is benchmarked against, and still the most rentable accelerator on earth.
rent Rentable by the hour from multiple clouds; purchasable at scale.
Specifications
- Memory
- 80 GB HBM3
- Bandwidth
- 3.35 TB/s
- Compute
- Hopper-generation FP8
- Power
- 700 W
- Interconnect
- NVLink 4
- Software
- CUDA
- Form factor
- accelerator
- Workload
- both
Verified 2026-09-06. Fields reading “not published” are exactly that — we do not estimate a figure a vendor withholds.
The catch
80 GB is the binding limit. Large models need multi-GPU sharding that a 141 GB or 288 GB part would not, and sharding costs you latency and complexity.
The economics
The benchmark denominator. When a vendor claims '2.6x an H100', this is the H100 they mean.
Cost per million tokens is the metric that decides most real purchases, and there is no neutral benchmark for it — every published figure comes from a company selling one side of the comparison. Read all of them, ours included, as claims rather than measurements.
The power question underneath this
A rack of frontier accelerators draws 120–200 kW against a 2026 average of about 27 kW, and the US grid interconnection queue exceeds 2,600 GW with waits approaching five years. Whether you can energise this part is now a harder question than whether you can buy it.
Layer 1 — Energy · The interconnection queue · Rack power density · Tokens per watt
Compared against
- NVIDIA H100 vs NVIDIA H200 — The same architecture with 61 GB and 1.45 TB/s more. Whether that matters is decided entirely by your model size.Compare with NVIDIA H200
- AMD Instinct MI300X vs NVIDIA H100 — The generation where AMD first became a real answer for memory-bound inference.Compare with AMD Instinct MI300X
- Intel Gaudi 3 vs NVIDIA H100 — Standard Ethernet and a lower price against the deepest software ecosystem in computing.Compare with Intel Gaudi 3
- Groq LPU vs NVIDIA H100 — Specialist inference silicon against the general-purpose default.Compare with Groq LPU
- Huawei Ascend 910C vs NVIDIA H100 — A jurisdiction comparison, not a performance one.Compare with Huawei Ascend 910C
Related silicon
- NVIDIA B200 — 192 GB HBM3E, rent
- NVIDIA GB200 NVL72 — 13.5 TB HBM3E across the rack, rent
- NVIDIA GB300 NVL72 — Higher HBM3E capacity than GB200; per-rack figure not consistently published, rent
- NVIDIA H200 — 141 GB HBM3E, rent
- NVIDIA RTX PRO 6000 Blackwell — 96 GB GDDR7, buy
- AMD Instinct MI300X — 192 GB HBM3, rent