vLLM
Top pickThe standard open inference engine — the thing under most serving platforms.
vLLM is the high-throughput LLM inference engine that effectively set the category standard, and its PagedAttention memory management is why it serves far more concurrent requests per GPU than a naive implementation. It exposes an OpenAI-compatible API, so an application already talking to OpenAI can be pointed at a vLLM endpoint by changing a base URL. It is Apache-2.0 and now sits under the PyTorch Foundation rather than a single company. The important thing to understand about the whole category: a large share of the managed platforms you might pay for are running vLLM underneath, so choosing it directly is not a downgrade from the commercial option — it is the commercial option without the margin.
What it does well
- +Highest throughput per GPU in general open benchmarks — PagedAttention is the reason
- +OpenAI-compatible API: swap a base URL, keep the application
- +Apache-2.0 under the PyTorch Foundation, not a single vendor
- +Runs the same on a rented H100, your own box, or a Kubernetes cluster
Where it falls short
- −You provide the GPU, the autoscaling and the uptime
- −No scale-to-zero — an idle GPU still costs whatever you rent it for
- −Tuning memory and batching well takes real understanding
- −Focused on text models; multimodal support lags the frontier
vLLM as an alternative to
Where vLLM shows up in our comparisons, and how it ranked.
vLLM head-to-head
Straight comparisons against the tools people weigh it against.