Groq · LPU · Layer 2
Groq LPU
Cheap, fast token generation via API. The lowest cost per million tokens among the specialist inference parts.
rent Rentable by the hour from multiple clouds; purchasable at scale.
Specifications
- Memory
- 230 MB SRAM per chip, no HBM
- Bandwidth
- 80 TB/s on-chip
- Compute
- Deterministic inference architecture
- Power
- Not published per chip
- Interconnect
- Chip-to-chip fabric
- Software
- GroqCloud API, GroqWare
- Form factor
- accelerator
- Workload
- inference
Verified 2026-09-09. Fields reading “not published” are exactly that — we do not estimate a figure a vendor withholds.
The catch
230 MB per chip means the model is split across many chips — Llama2-70B at Groq's headline speed required 576 chips, roughly eight racks. The per-chip spec is misleading unless you read it as a per-cluster architecture.
The economics
Cheaper per million tokens than Cerebras while being materially slower. If your product is priced per token rather than per millisecond, that is the right trade.
Cost per million tokens is the metric that decides most real purchases, and there is no neutral benchmark for it — every published figure comes from a company selling one side of the comparison. Read all of them, ours included, as claims rather than measurements.
The power question underneath this
A rack of frontier accelerators draws 120–200 kW against a 2026 average of about 27 kW, and the US grid interconnection queue exceeds 2,600 GW with waits approaching five years. Whether you can energise this part is now a harder question than whether you can buy it.
Layer 1 — Energy · The interconnection queue · Rack power density · Tokens per watt
Where you actually rent this
Choosing the part is the easy half. The same accelerator rents for wildly different money depending only on who you rent it from — the cheapest and dearest published on-demand price for an H100 sit 5.6× apart — and the provider, not the chip, is what decides whether you can leave later.
Compared against
- Cerebras WSE-3 vs Groq LPU — Speed against cost. Cerebras is 3-8x faster; Groq is cheaper per million tokens.Compare with Cerebras WSE-3
- Groq LPU vs NVIDIA H100 — Specialist inference silicon against the general-purpose default.Compare with NVIDIA H100
- SambaNova SN40L vs Groq LPU — Many models on one system against one model served cheaply.Compare with SambaNova SN40L
Related silicon
- Cerebras WSE-3 — 44 GB on-chip SRAM, rent
- SambaNova SN40L — 520 MiB SRAM + 64 GiB HBM + up to 1.5 TiB DDR per socket, rent
- AMD Helios (Instinct MI455X) — 31 TB of HBM4 per rack across 72 MI455X GPUs; 432 GB of HBM4 per GPU., rent
- AMD Instinct MI300X — 192 GB HBM3, rent
- AMD Instinct MI350X — 288 GB HBM3E, rent
- AMD Instinct MI355X — 288 GB HBM3E, rent
Sources
Want this whole layer as one table?
The accelerators on these pages as a single CSV — memory, bandwidth, compute, power, interconnect, software stack, availability, who each part is for and the catch — ready to sort, filter or load into a notebook.
Get the silicon table — free for research and evaluation
Four short lines and the download starts at once (CC-BY-NC-4.0, built from these pages the moment you ask). What you type is used only to answer the request and is never sold. Prefer Hugging Face? Request it there.
Using it inside a product, a client report or anything you sell? That needs a commercial licence.