Models & tooling
The models themselves, plus the gateways, vector stores, frameworks and observability that make them usable.
The fastest-moving layer, and the one where open weights have most closed the gap on closed ones. A decision made here in 2024 is almost certainly wrong now.
7 decisions · 35 verified alternatives · 7 categories
Every decision at this layer
Ranked by how self-sufficient the leading alternative is. The score is a published rubric, not an opinion — see methodology.
LangChain → Direct SDKs (no framework)
The 2026 consensus: official SDKs + a few hundred lines you own.
Free — you pay only your model provider; no framework tier, no per-seat tooling
OpenAI API (ChatGPT) → Ollama
Run Llama, Mistral, Qwen and more with one command.
Free / self-host (you pay only for your own hardware + power)
OpenAI AgentKit → LangGraph
Agents as an explicit state graph — the closest thing to what you are leaving.
Free and open source. LangGraph Platform is an optional paid hosted tier.
Pinecone → Qdrant
Fast, open-source, one binary — the default self-hosted vector DB.
Free self-hosted (no limits); Qdrant Cloud managed tiers with a free 1GB cluster
OpenAI Embeddings API → Nomic Embed
The sovereign default — Apache-2.0, runs in Ollama, 8k context.
Free — runs locally via Ollama or sentence-transformers; your hardware is the cost
Portkey → LiteLLM
One OpenAI-compatible API in front of 100+ providers, running on your own box.
Free and self-hosted — you pay the model providers directly with no markup, plus hosting for a small VM (roughly $20–50/month). A paid enterprise tier exists for SSO, audit logs and support.
LangSmith → Langfuse
The self-hosted default for LLM tracing — MIT core, framework-agnostic.
Free to self-host — Docker Compose or Kubernetes, costs are your own infrastructure. A managed cloud with a free tier and paid plans exists if you would rather not run it.
Categories in this layer
Where this sits in the stack
L3 below · Infrastructure — The clouds, orchestration and plumbing that turn silicon into something you can rent by the hour.
L5 above · Applications — The software people actually open — the layer where AI stops being infrastructure and becomes a product.