serverless inference · api · Layer 2.5
Baseten
Teams deploying their own model weights who want autoscaling without operating Kubernetes.
low portability Sticky. Leaving means rewriting against a different interface, not changing a hostname.
What you are buying
- Business model
- You never see a GPU. You send a request and pay per token or per second of execution.
- How you reach it
- You call an endpoint. There is no infrastructure to see, and no infrastructure to move.
- Accelerators
- H100, A100, L4 and others behind managed deployments
- Regions
- US
- Pricing model
- Per minute of compute for dedicated deployments; autoscaling to zero.
- Getting started
- Self-serve.
- Capacity
- Managed on top of other people's capacity.
- Ownership
- Private.
Verified 2026-09-09. Fields reading “not published” are exactly that — we do not estimate a figure a vendor withholds.
The counterparty
You inherit whoever they buy from, which is not always disclosed. Ask.
A multi-year GPU commitment is a credit decision wearing a cloud contract. This is the section no benchmark covers and the one that decides what happens to your workload in 2028.
The catch
Scale-to-zero is the headline and cold starts are the cost. For interactive products the cold-start number matters more than the hourly rate.
The economics
Pay for what you serve. Excellent for spiky traffic, poor value at steady high utilisation where a reserved GPU is cheaper.
No rate is quoted on this page on purpose. Published list prices at this layer move weekly and essentially nobody signing a real contract pays them. Dated, sourced figures live in the GPU rental price index, where the spread between the cheapest and dearest seller of the identical chip runs to roughly 9x.
The layers underneath this one
Whatever you rent here is a chip in a building that needs power. In 2026 megawatts, not silicon, are the binding constraint on the whole industry — a frontier rack draws 120–200 kW against a 2026 average near 27 kW, and the US interconnection queue exceeds 2,600 GW.
Layer 2 — Silicon · Layer 1 — Energy · The interconnection queue · Tokens per watt
Compared against
- Modal vs Baseten — Scale-to-zero, and which one's cold start you can live with.Compare with Modal
Related providers
- Fireworks AI — serverless inference, low portability
- Replicate — serverless inference, low portability
- Together AI — serverless inference, low portability
- Modal — serverless inference, low portability
- Akash Network — marketplace, medium portability
- AWS (P5, P6, G6) — hyperscaler, medium portability