Layer 4 · self-hosting reality check
What it actually takes to self-host Chroma
The docs say 512 MB. In practice you want 4 GB. Here is the honest version — real requirements, real monthly cost, what you will be maintaining, and the one thing that catches people out.
Usually reached from Pinecone alternatives, where Chroma is one of the picks.
| RAM — documented minimum | 512 MB |
|---|---|
| RAM — what it really needs | 4 GB |
| CPU | 2 vCPU |
| Disk | Vectors plus metadata |
| Monthly cost | $12–24/mo, or nothing at all in embedded mode |
| Setup time | 10 minutes |
| How you install it | pip install and run embedded, or a container for client-server mode |
| Ongoing maintenance | Low. |
| Where it stops scaling | Low millions of vectors comfortably. It is the fastest thing here to get started with and honest about its ceiling. |
The thing that catches people out
Embedded mode is a local file, and people prototype against it then deploy several application replicas pointing at the same volume — which does not work and corrupts in confusing ways. Move to client-server mode before you scale past one process. The transition is easy if you plan it and unpleasant if you discover it.
When not to self-host Chroma
You are heading for tens of millions of vectors. Chroma optimises for developer experience at prototype scale; Qdrant or Milvus are the production answer.
Every guide here carries this section. A site that only ever tells you to self-host is selling something — the useful answer is sometimes no.
Other Layer 4 self-hosting guides
- Self-hosting Ollama8 GB VRAM for a 7B model at usable speed; 24 GB for 30B-class
- Self-hosting vLLM24 GB VRAM minimum for useful production serving
- Self-hosting QdrantVectors × dimensions × 4 bytes, in RAM, plus overhead — 1M × 768d is roughly 3 GB
- Self-hosting pgvector8 GB — the HNSW index wants to be resident
- Self-hosting LlamaIndex4 GB for the app; your vector store is the real cost
- Self-hosting faster-whisper5 GB VRAM for large-v3 in float16; 2 GB with int8
Common questions
- How much RAM does Chroma actually need?
- 4 GB in practice. The documented minimum is 512 MB, which is the figure at which the process starts rather than the figure at which it works under real use. 2 vCPU alongside it.
- What does self-hosting Chroma cost per month?
- $12–24/mo, or nothing at all in embedded mode This is commodity VPS pricing and excludes your time, which is the larger cost for most people — budget for low.
- How long does it take to set up Chroma?
- 10 minutes, via pip install and run embedded, or a container for client-server mode.
- When should I NOT self-host Chroma?
- You are heading for tens of millions of vectors. Chroma optimises for developer experience at prototype scale; Qdrant or Milvus are the production answer.
- What is the most common mistake when self-hosting Chroma?
- Embedded mode is a local file, and people prototype against it then deploy several application replicas pointing at the same volume — which does not work and corrupts in confusing ways. Move to client-server mode before you scale past one process. The transition is easy if you plan it and unpleasant if you discover it.