Layer 2 · head to head
SambaNova SN40L vs Groq LPU
Many models on one system against one model served cheaply.
| SambaNova SN40L | Groq LPU | |
|---|---|---|
| Can you get it | Rentable by the hour from multiple clouds; purchasable at scale. | Rentable by the hour from multiple clouds; purchasable at scale. |
| Memory | 520 MiB SRAM + 64 GiB HBM + up to 1.5 TiB DDR per socket | 230 MB SRAM per chip, no HBM |
| Bandwidth | Over 1 TB/s DDR-to-HBM model streaming | 80 TB/s on-chip |
| Compute | Reconfigurable dataflow | Deterministic inference architecture |
| Power | Not published | Not published per chip |
| Interconnect | Socket-to-socket fabric | Chip-to-chip fabric |
| Software | SambaFlow | GroqCloud API, GroqWare |
| Workload | inference | inference |
SambaNova SN40L
For: Serving many large models from one system. The three-tier memory design exists to swap models in and out rather than to hold one resident.
The catch: The most unusual architecture here, so the least portable. Independently verified at 1,084 tokens/s for Llama3-8B — but on 16 sockets, which is the number that matters.
Economics: Its argument is models-per-dollar rather than tokens-per-dollar. If you serve one model, this is the wrong shape.
Groq LPU
For: Cheap, fast token generation via API. The lowest cost per million tokens among the specialist inference parts.
The catch: 230 MB per chip means the model is split across many chips — Llama2-70B at Groq's headline speed required 576 chips, roughly eight racks. The per-chip spec is misleading unless you read it as a per-cluster architecture.
Economics: Cheaper per million tokens than Cerebras while being materially slower. If your product is priced per token rather than per millisecond, that is the right trade.
Before either — can you power it?
Not published against Not published per chip. In 2026 that comparison usually matters more than the FLOPS one: the US interconnection queue exceeds 2,600 GW with waits approaching five years, and roughly 80% of projects withdraw before energising.
Want this whole layer as one table?
The accelerators on these pages as a single CSV — memory, bandwidth, compute, power, interconnect, software stack, availability, who each part is for and the catch — ready to sort, filter or load into a notebook.
Get the silicon table — free for research and evaluation
Four short lines and the download starts at once (CC-BY-NC-4.0, built from these pages the moment you ask). What you type is used only to answer the request and is never sold. Prefer Hugging Face? Request it there.
Using it inside a product, a client report or anything you sell? That needs a commercial licence.