Layer 4 · self-hosting reality check
What it actually takes to self-host LlamaIndex
The docs say 1 GB. In practice you want 4 GB for the app; your vector store is the real cost. Here is the honest version — real requirements, real monthly cost, what you will be maintaining, and the one thing that catches people out.
Usually reached from LangChain alternatives, where LlamaIndex is one of the picks.
| RAM — documented minimum | 1 GB |
|---|---|
| RAM — what it really needs | 4 GB for the app; your vector store is the real cost |
| CPU | 2 vCPU |
| Disk | Small — it is a library, not a datastore |
| Monthly cost | $0 for the framework. The bill is embeddings and the vector store underneath. |
| Setup time | An hour to a working pipeline, weeks to a good one |
| How you install it | pip install; it is a library you build with, not a service you deploy |
| Ongoing maintenance | Version churn is real — the API has moved fast. Pin versions on anything long-lived. |
| Where it stops scaling | As far as your vector store and embedding budget go. The framework is not the bottleneck. |
The thing that catches people out
Retrieval quality is decided by chunking, and chunking is decided before you ever call a model. A chunk boundary through the middle of a definition, or a table flattened into a line of numbers, cannot be rescued by a better retriever or a bigger model — and it fails silently, retrieving confidently and answering wrongly. Inspect your actual chunks before blaming anything else.
When not to self-host LlamaIndex
You want a product rather than a framework. R2R or RAGFlow deploy as services; LlamaIndex is parts you assemble.
Every guide here carries this section. A site that only ever tells you to self-host is selling something — the useful answer is sometimes no.
Other Layer 4 self-hosting guides
- Self-hosting Ollama8 GB VRAM for a 7B model at usable speed; 24 GB for 30B-class
- Self-hosting vLLM24 GB VRAM minimum for useful production serving
- Self-hosting QdrantVectors × dimensions × 4 bytes, in RAM, plus overhead — 1M × 768d is roughly 3 GB
- Self-hosting pgvector8 GB — the HNSW index wants to be resident
- Self-hosting faster-whisper5 GB VRAM for large-v3 in float16; 2 GB with int8
- Self-hosting Langfuse4 GB
Common questions
- How much RAM does LlamaIndex actually need?
- 4 GB for the app; your vector store is the real cost in practice. The documented minimum is 1 GB, which is the figure at which the process starts rather than the figure at which it works under real use. 2 vCPU alongside it.
- What does self-hosting LlamaIndex cost per month?
- $0 for the framework. The bill is embeddings and the vector store underneath. This is commodity VPS pricing and excludes your time, which is the larger cost for most people — budget for version churn is real — the API has moved fast. Pin versions on anything long-lived.
- How long does it take to set up LlamaIndex?
- An hour to a working pipeline, weeks to a good one, via pip install; it is a library you build with, not a service you deploy.
- When should I NOT self-host LlamaIndex?
- You want a product rather than a framework. R2R or RAGFlow deploy as services; LlamaIndex is parts you assemble.
- What is the most common mistake when self-hosting LlamaIndex?
- Retrieval quality is decided by chunking, and chunking is decided before you ever call a model. A chunk boundary through the middle of a definition, or a table flattened into a line of numbers, cannot be rescued by a better retriever or a bigger model — and it fails silently, retrieving confidently and answering wrongly. Inspect your actual chunks before blaming anything else.