serverless inference · api · Layer 2.5
Baseten
Teams deploying their own model weights who want autoscaling without operating Kubernetes.
low portability Sticky. Leaving means rewriting against a different interface, not changing a hostname.
What you are buying
- Business model
- You never see a GPU. You send a request and pay per token or per second of execution.
- How you reach it
- You call an endpoint. There is no infrastructure to see, and no infrastructure to move.
- Accelerators
- H100, A100, L4 and others behind managed deployments
- Regions
- US
- Pricing model
- Per minute of compute for dedicated deployments; autoscaling to zero.
- Getting started
- Self-serve.
- Capacity
- Managed on top of other people's capacity.
- Ownership
- Private.
Verified 2026-09-09. Fields reading “not published” are exactly that — we do not estimate a figure a vendor withholds.
The counterparty
You inherit whoever they buy from, which is not always disclosed. Ask.
A multi-year GPU commitment is a credit decision wearing a cloud contract. This is the section no benchmark covers and the one that decides what happens to your workload in 2028.
The catch
Scale-to-zero is the headline and cold starts are the cost. For interactive products the cold-start number matters more than the hourly rate.
The economics
Pay for what you serve. Excellent for spiky traffic, poor value at steady high utilisation where a reserved GPU is cheaper.
No rate is quoted on this page on purpose. Published list prices at this layer move weekly and essentially nobody signing a real contract pays them. Dated, sourced figures live in the GPU cost calculator, where the cheapest and dearest published on-demand price for the identical H100 sit 5.6× apart.
The layers underneath this one
Whatever you rent here is a chip in a building that needs power. In 2026 megawatts, not silicon, are the binding constraint on the whole industry — a frontier rack draws 120–200 kW against a 2026 average near 27 kW, and the US interconnection queue exceeds 2,600 GW.
Layer 2 — Silicon · Layer 1 — Energy · The interconnection queue · Tokens per watt
Compared against
- Modal vs Baseten — Scale-to-zero, and which one's cold start you can live with.Compare with Modal
Related providers
- Fireworks AI — serverless inference, low portability
- Replicate — serverless inference, low portability
- Together AI — serverless inference, low portability
- Modal — serverless inference, low portability
- Akash Network — marketplace, medium portability
- AWS (P5, P6, G6) — hyperscaler, medium portability