Modal → vLLM
What it saves, what actually moves, what you rebuild — and the thing that catches people.
What it costs, and what it saves
Modal's H100 is about $3.95/hr against roughly $1.99 renting directly and running vLLM yourself — you stop paying the orchestration premium. For an endpoint busy 12 hours a day that is roughly $1,400/month against $700.
Moves cleanly
The model and the inference code. Both are ordinary Python and PyTorch.
You rebuild
Everything Modal was doing for you: autoscaling, scale-to-zero, request queuing and the deployment pipeline.
What Modal costs you today
Per-second billing with no minimum and scale-to-zero when idle. H100 capacity runs about $3.95/hr, A100 80GB about $2.50/hr, and L4 about $0.80/hr, with CPU and memory billed separately. For comparison, Replicate's H100 is around $5.49/hr and Baseten's around $6.50/hr per replica-hour billed continuously, while raw rental at RunPod is near $1.99/hr. Rates checked 2026-07-30; GPU pricing moves quickly, so confirm before planning around it.
What actually holds you in
Higher than it looks, and it is worth naming precisely. The model weights and inference code are portable — that part is just Python and PyTorch. What is not portable is Modal's own layer: `@app.function`, image definitions, volumes, secrets and scheduling all live inside your source files, so leaving means unpicking the platform from the application rather than redeploying it elsewhere. Teams that keep the model logic in a plain module and confine Modal decorators to a thin entry point keep the exit cheap. Teams that do not, do not.
What you are moving to
vLLM is the high-throughput LLM inference engine that effectively set the category standard, and its PagedAttention memory management is why it serves far more concurrent requests per GPU than a naive implementation. It exposes an OpenAI-compatible API, so an application already talking to OpenAI can be pointed at a vLLM endpoint by changing a base URL. It is Apache-2.0 and now sits under the PyTorch Foundation rather than a single company. The important thing to understand about the whole category: a large share of the managed platforms you might pay for are running vLLM underneath, so choosing it directly is not a downgrade from the commercial option — it is the commercial option without the margin.
Free and open source. You pay only for the GPUs you rent or own.
vLLM strengths
- Highest throughput per GPU in general open benchmarks — PagedAttention is the reason
- OpenAI-compatible API: swap a base URL, keep the application
- Apache-2.0 under the PyTorch Foundation, not a single vendor
- Runs the same on a rented H100, your own box, or a Kubernetes cluster
What you give up
- You provide the GPU, the autoscaling and the uptime
- No scale-to-zero — an idle GPU still costs whatever you rent it for
- Tuning memory and batching well takes real understanding
- Focused on text models; multimodal support lags the frontier
The migration, step by step
- 1Run vLLM locally against your model first and confirm throughput on one GPU
- 2Rent a persistent GPU and serve with vLLM's OpenAI-compatible endpoint
- 3Change your application's base URL — if you were calling an OpenAI-shaped API this is a one-line change
- 4Add a process supervisor and health checks, because nothing restarts it for you now
- 5Measure cost across a full week before cancelling Modal
The gotcha
You lose scale-to-zero. A rented GPU costs the same at 3am with no traffic as at midday. If utilisation is under about 40%, Modal was genuinely cheaper and this migration loses money.
When to stay on Modal
Traffic is bursty or unpredictable, or nobody wants to own uptime for an inference endpoint.
More Layer 3 migrations
Figures verified 2026-07-30 against vendor pricing pages. Prices change and migrations differ by estate — treat the cost delta as a starting model, not a quote. Rankings and recommendations here are merit-only; affiliate income never changes a verdict. See our methodology.