Layer 4 · self-hosting reality check
What it actually takes to self-host LlamaIndex
The docs say 1 GB. In practice you want 4 GB for the app; your vector store is the real cost. Here is the honest version — real requirements, real monthly cost, what you will be maintaining, and the one thing that catches people out.
Usually reached from LangChain alternatives, where LlamaIndex is one of the picks.
Wondering whether you need to at all? Is LangChain free? — what the free tier actually allows, and where the wall is.
| RAM — documented minimum | 1 GB |
|---|---|
| RAM — what it really needs | 4 GB for the app; your vector store is the real cost |
| CPU | 2 vCPU |
| Disk | Small — it is a library, not a datastore |
| Monthly cost | $0 for the framework. The bill is embeddings and the vector store underneath. |
| Setup time | An hour to a working pipeline, weeks to a good one |
| How you install it | pip install; it is a library you build with, not a service you deploy |
| Ongoing maintenance | Version churn is real — the API has moved fast. Pin versions on anything long-lived. |
| Where it stops scaling | As far as your vector store and embedding budget go. The framework is not the bottleneck. |
The thing that catches people out
Retrieval quality is decided by chunking, and chunking is decided before you ever call a model. A chunk boundary through the middle of a definition, or a table flattened into a line of numbers, cannot be rescued by a better retriever or a bigger model — and it fails silently, retrieving confidently and answering wrongly. Inspect your actual chunks before blaming anything else.
When not to self-host LlamaIndex
You want a product rather than a framework. R2R or RAGFlow deploy as services; LlamaIndex is parts you assemble.
Every guide here carries this section. A site that only ever tells you to self-host is selling something — the useful answer is sometimes no.
Other Layer 4 self-hosting guides
- Self-hosting Ollama8 GB VRAM for a 7B model at usable speed; 24 GB for 30B-class
- Self-hosting vLLM24 GB VRAM minimum for useful production serving
- Self-hosting QdrantVectors × dimensions × 4 bytes, in RAM, plus overhead — 1M × 768d is roughly 3 GB
- Self-hosting pgvector8 GB — the HNSW index wants to be resident
- Self-hosting faster-whisper5 GB VRAM for large-v3 in float16; 2 GB with int8
- Self-hosting Langfuse4 GB
Common questions
- How much RAM does LlamaIndex actually need?
- 4 GB for the app; your vector store is the real cost in practice. The documented minimum is 1 GB, which is the figure at which the process starts rather than the figure at which it works under real use. 2 vCPU alongside it.
- What does self-hosting LlamaIndex cost per month?
- $0 for the framework. The bill is embeddings and the vector store underneath. This is commodity VPS pricing and excludes your time, which is the larger cost for most people — budget for version churn is real — the API has moved fast. Pin versions on anything long-lived.
- How long does it take to set up LlamaIndex?
- An hour to a working pipeline, weeks to a good one, via pip install; it is a library you build with, not a service you deploy.
- When should I NOT self-host LlamaIndex?
- You want a product rather than a framework. R2R or RAGFlow deploy as services; LlamaIndex is parts you assemble.
- What is the most common mistake when self-hosting LlamaIndex?
- Retrieval quality is decided by chunking, and chunking is decided before you ever call a model. A chunk boundary through the middle of a definition, or a table flattened into a line of numbers, cannot be rescued by a better retriever or a bigger model — and it fails silently, retrieving confidently and answering wrongly. Inspect your actual chunks before blaming anything else.