Layer 4 · self-hosting reality check
What it actually takes to self-host Chroma
The docs say 512 MB. In practice you want 4 GB. Here is the honest version — real requirements, real monthly cost, what you will be maintaining, and the one thing that catches people out.
Usually reached from Pinecone alternatives, where Chroma is one of the picks.
Wondering whether you need to at all? Is Pinecone free? — what the free tier actually allows, and where the wall is.
| RAM — documented minimum | 512 MB |
|---|---|
| RAM — what it really needs | 4 GB |
| CPU | 2 vCPU |
| Disk | Vectors plus metadata |
| Monthly cost | $12–24/mo, or nothing at all in embedded mode |
| Setup time | 10 minutes |
| How you install it | pip install and run embedded, or a container for client-server mode |
| Ongoing maintenance | Low. |
| Where it stops scaling | Low millions of vectors comfortably. It is the fastest thing here to get started with and honest about its ceiling. |
The thing that catches people out
Embedded mode is a local file, and people prototype against it then deploy several application replicas pointing at the same volume — which does not work and corrupts in confusing ways. Move to client-server mode before you scale past one process. The transition is easy if you plan it and unpleasant if you discover it.
When not to self-host Chroma
You are heading for tens of millions of vectors. Chroma optimises for developer experience at prototype scale; Qdrant or Milvus are the production answer.
Every guide here carries this section. A site that only ever tells you to self-host is selling something — the useful answer is sometimes no.
Other Layer 4 self-hosting guides
- Self-hosting Ollama8 GB VRAM for a 7B model at usable speed; 24 GB for 30B-class
- Self-hosting vLLM24 GB VRAM minimum for useful production serving
- Self-hosting QdrantVectors × dimensions × 4 bytes, in RAM, plus overhead — 1M × 768d is roughly 3 GB
- Self-hosting pgvector8 GB — the HNSW index wants to be resident
- Self-hosting LlamaIndex4 GB for the app; your vector store is the real cost
- Self-hosting faster-whisper5 GB VRAM for large-v3 in float16; 2 GB with int8
Common questions
- How much RAM does Chroma actually need?
- 4 GB in practice. The documented minimum is 512 MB, which is the figure at which the process starts rather than the figure at which it works under real use. 2 vCPU alongside it.
- What does self-hosting Chroma cost per month?
- $12–24/mo, or nothing at all in embedded mode This is commodity VPS pricing and excludes your time, which is the larger cost for most people — budget for low.
- How long does it take to set up Chroma?
- 10 minutes, via pip install and run embedded, or a container for client-server mode.
- When should I NOT self-host Chroma?
- You are heading for tens of millions of vectors. Chroma optimises for developer experience at prototype scale; Qdrant or Milvus are the production answer.
- What is the most common mistake when self-hosting Chroma?
- Embedded mode is a local file, and people prototype against it then deploy several application replicas pointing at the same volume — which does not work and corrupts in confusing ways. Move to client-server mode before you scale past one process. The transition is easy if you plan it and unpleasant if you discover it.