Models & tooling
The models themselves, plus the gateways, vector stores, frameworks and observability that make them usable.
The fastest-moving layer, and the one where open weights have most closed the gap on closed ones. A decision made here in 2024 is almost certainly wrong now.
14 decisions · 70 verified alternatives · 14 categories
Every decision at this layer
Ranked by how self-sufficient the leading alternative is. The score is a published rubric, not an opinion — see methodology.
Deepgram → faster-whisper
Whisper, several times faster, on less memory. The practical default.
Free. Hardware you already own; a laptop handles the smaller models.
Braintrust → Promptfoo
Prompt regressions fail the build. Declarative evals that live in CI.
Free and MIT, unlimited seats. An enterprise tier exists for larger organisations.
AWS Textract → Docling
IBM's document converter. Layout-aware PDF to clean Markdown, MIT.
Free, MIT. Runs on your own hardware, CPU or GPU.
LangChain → Direct SDKs (no framework)
The 2026 consensus: official SDKs + a few hundred lines you own.
Free — you pay only your model provider; no framework tier, no per-seat tooling
OpenAI Fine-Tuning → Axolotl
Fine-tune most open models from one YAML file. The community default.
Free. You rent or own the GPU; a small LoRA can cost a few dollars of rented time.
Vectara → LlamaIndex
The most complete open RAG framework. Every stage, under your control.
Free and MIT licensed. A paid managed parsing/ingest service exists separately.
Weights & Biases → MLflow
The open standard. Tracking, registry, projects and deployment in one.
Free, Apache-2.0. Managed versions are sold by the clouds if you want one.
OpenAI API (ChatGPT) → Ollama
Run Llama, Mistral, Qwen and more with one command.
Free / self-host (you pay only for your own hardware + power)
OpenAI AgentKit → LangGraph
Agents as an explicit state graph — the closest thing to what you are leaving.
Free and open source. LangGraph Platform is an optional paid hosted tier.
Pinecone → Qdrant
Fast, open-source, one binary — the default self-hosted vector DB.
Free self-hosted (no limits); Qdrant Cloud managed tiers with a free 1GB cluster
OpenAI Embeddings API → Nomic Embed
The sovereign default — Apache-2.0, runs in Ollama, 8k context.
Free — runs locally via Ollama or sentence-transformers; your hardware is the cost
Azure AI Content Safety → NVIDIA NeMo Guardrails
Write your policy as rails, in a language built for it.
Free and Apache-2.0. Runs wherever you run it.
Portkey → LiteLLM
One OpenAI-compatible API in front of 100+ providers, running on your own box.
Free and self-hosted — you pay the model providers directly with no markup, plus hosting for a small VM (roughly $20–50/month). A paid enterprise tier exists for SSO, audit logs and support.
LangSmith → Langfuse
The self-hosted default for LLM tracing — MIT core, framework-agnostic.
Free to self-host — Docker Compose or Kubernetes, costs are your own infrastructure. A managed cloud with a free tier and paid plans exists if you would rather not run it.
Categories in this layer
Where this sits in the stack
L3 below · Infrastructure — The clouds, orchestration and plumbing that turn silicon into something you can rent by the hour.
L5 above · Applications — The software people actually open — the layer where AI stops being infrastructure and becomes a product.