serverless inference · api · Layer 2.5
Replicate
Product teams shipping a model-backed feature who do not want to think about GPUs at all.
low portability Sticky. Leaving means rewriting against a different interface, not changing a hostname.
What you are buying
- Business model
- You never see a GPU. You send a request and pay per token or per second of execution.
- How you reach it
- You call an endpoint. There is no infrastructure to see, and no infrastructure to move.
- Accelerators
- A100, H100, L40S, T4 behind hosted models
- Regions
- US
- Pricing model
- Per second of prediction time, per hardware class.
- Getting started
- API key.
- Capacity
- Managed.
- Ownership
- Private.
Verified 2026-09-09. Fields reading “not published” are exactly that — we do not estimate a figure a vendor withholds.
The counterparty
Low. Models are containerised with an open packaging format, so the exit is unusually clean for this layer.
A multi-year GPU commitment is a credit decision wearing a cloud contract. This is the section no benchmark covers and the one that decides what happens to your workload in 2028.
The catch
Cold starts on unpopular models are long. The convenience premium over renting the same GPU directly is large and worth calculating once.
The economics
The most expensive way to buy a GPU-second and often the cheapest way to ship a feature. Both are true; which one matters depends on your volume.
No rate is quoted on this page on purpose. Published list prices at this layer move weekly and essentially nobody signing a real contract pays them. Dated, sourced figures live in the GPU cost calculator, where the cheapest and dearest published on-demand price for the identical H100 sit 5.6× apart.
The layers underneath this one
Whatever you rent here is a chip in a building that needs power. In 2026 megawatts, not silicon, are the binding constraint on the whole industry — a frontier rack draws 120–200 kW against a 2026 average near 27 kW, and the US interconnection queue exceeds 2,600 GW.
Layer 2 — Silicon · Layer 1 — Energy · The interconnection queue · Tokens per watt
Compared against
- Replicate vs Modal — Ship a feature in an afternoon, or own the deployment model.Compare with Modal
Related providers
- Baseten — serverless inference, low portability
- Fireworks AI — serverless inference, low portability
- Together AI — serverless inference, low portability
- Modal — serverless inference, low portability
- Akash Network — marketplace, medium portability
- AWS (P5, P6, G6) — hyperscaler, medium portability