vLLM vs Ray Serve
Both are alternatives to Modal. Here's how they stack up — verified facts, no spin.
Also searched as Ray Serve vs vLLM — same comparison, one verdict.
vLLM and Ray Serve are closely matched on ownership (95 vs 89) — this one comes down to pricing and to which trade-offs below you can live with.
vLLM
TOP PICKThe standard open inference engine — the thing under most serving platforms.
vLLM is the high-throughput LLM inference engine that effectively set the category standard, and its PagedAttention memory management is why it serves far more concurrent requests per GPU than a naive implementation. It exposes an OpenAI-compatible API, so an application already talking to OpenAI can be pointed at a vLLM endpoint by changing a base URL. It is Apache-2.0 and now sits under the PyTorch Foundation rather than a single company. The important thing to understand about the whole category: a large share of the managed platforms you might pay for are running vLLM underneath, so choosing it directly is not a downgrade from the commercial option — it is the commercial option without the margin.
Ray Serve
Multi-node, multi-model serving for when one GPU is not the problem.
Ray Serve is the serving layer of Ray, the distributed computing framework, and it is the answer when the hard part is not throughput on one card but coordinating many models across many machines. It has first-class vLLM support, does autoscaling and back-pressure properly, and composes pipelines where a request touches several models in sequence. It is heavier than the others and that weight is the point — it is aimed at the case where you are building serving infrastructure rather than deploying an endpoint. If you only need one model behind one URL, this is more machinery than the job requires.
Side by side
10 points of comparison, every one read from a verified field. Green marks the side that wins a row outright. A dash means we do not hold that fact — never that it is zero.
| vLLM | Ray Serve | |
|---|---|---|
| Sovereignty ScoreOur transparent 0–100 composite for data ownership and exit cost. | 95 | 89 |
| Open source | Yes | Yes |
| Self-hostable | Yes | Yes |
| Local-first data | Yes | Yes |
| License | Apache-2.0 | Apache-2.0 |
| Pricing | Free and open source. You pay only for the GPUs you rent or own. | Free and open source. Anyscale sells a managed Ray platform. |
| RAM to run it wellThe figure that actually matters, not the vendor's minimum. | 24 GB VRAM minimum for useful production serving | — |
| Realistic running costWhat the box costs each month if you run it yourself. | $300–900/mo for a rented A100 or L40S, against managed inference with a platform margin on every token | — |
| Setup timeHonest first-install estimate, not the marketing quickstart. | A day including CUDA | — |
| Ongoing maintenanceThe part nobody budgets for. | Moderate. CUDA and driver versions are the recurring pain, not vLLM itself. | — |
vLLM is Macrostack's recommended Modal alternative, so it's our pick here.
vLLM
Strengths
- +Highest throughput per GPU in general open benchmarks — PagedAttention is the reason
- +OpenAI-compatible API: swap a base URL, keep the application
- +Apache-2.0 under the PyTorch Foundation, not a single vendor
- +Runs the same on a rented H100, your own box, or a Kubernetes cluster
Trade-offs
- −You provide the GPU, the autoscaling and the uptime
- −No scale-to-zero — an idle GPU still costs whatever you rent it for
- −Tuning memory and batching well takes real understanding
- −Focused on text models; multimodal support lags the frontier
Ray Serve
Strengths
- +Genuine multi-node, multi-model orchestration with autoscaling and back-pressure
- +First-class vLLM integration — the engine underneath is the same
- +Composes multi-stage pipelines where a request hits several models
- +Same framework covers training, batch inference and serving
Trade-offs
- −Heaviest option here; a Ray cluster is a system to operate
- −Overkill for a single model behind a single endpoint
- −Debugging distributed failures is genuinely hard
- −Steepest learning curve of the four
Which one fits you
The trade-offs above, turned into a decision. Find the line that describes your team.
Choose vLLM
if a lower exit cost matters more to you than any single feature, and highest throughput per GPU in general open benchmarks — PagedAttention is the reason.
Choose Ray Serve
if genuine multi-node, multi-model orchestration with autoscaling and back-pressure.
Neither, yet
if both carry a real cost you should weigh first — you provide the GPU, the autoscaling and the uptime, and heaviest option here; a Ray cluster is a system to operate. If either of those is a dealbreaker for your team, the shortlist is wrong rather than the choice.
What it takes to run these yourself
Real requirements and honest running costs, not the vendor quickstart.
vLLM vs Ray Serve — common questions
Is vLLM a better fit than Ray Serve for model serving & inference?
It depends on what you are optimising for, and the honest split is this: vLLM scores 95 to Ray Serve's 89 on data ownership and exit cost, so it is the safer choice if you care about being able to leave. Ray Serve earns its place on a different axis — genuine multi-node, multi-model orchestration with autoscaling and back-pressure. Neither is a wrong answer for every team; the table above is the actual comparison.
What happens if we want to switch later?
vLLM keeps its data local or in open formats, so leaving is an export rather than a negotiation. Ray Serve is still self-hostable, so the files stay on your server either way — but it is not local-first by design, so check what its export produces before you rely on it.
Can I self-host vLLM or Ray Serve?
Both can be self-hosted. The difference is what it costs you in time rather than whether it is possible — see the setup and maintenance rows above.
Are vLLM and Ray Serve both alternatives to Modal?
Yes — both appear in our Modal comparison, which is why they are worth putting side by side. People usually arrive here already having decided to move off Modal and now choosing between the two replacements, which is a narrower and much easier question.
More model serving & inference comparisons
Related alternative guides
Facts verified 2026-07-30. Licenses and pricing change — spotted something out of date? That's a correction we want.