Promptfoo
Top pickPrompt regressions fail the build. Declarative evals that live in CI.
94
sovereigntyPromptfoo defines evaluations as YAML: your prompts, your test cases, your assertions, versioned in the repository next to the code they test. It runs from the CLI or in CI, compares outputs across models and prompt variants side by side, and fails the build when a change regresses. It also includes red-teaming for prompt injection and jailbreak testing. MIT licensed with no seat limits, which matters because everyone who edits a prompt should be running it.
OPEN SOURCEMITSELF-HOSTLOCAL-FIRST
LicenseMIT
PricingFree and MIT, unlimited seats. An enterprise tier exists for larger organisations.
Open sourceYes
Self-hostableYes
Local-first dataYes
What it does well
- +Evals live in your repo and run in CI — a regression blocks the merge
- +Side-by-side model and prompt comparison out of the box
- +Includes red-teaming for injection and jailbreak testing
- +MIT, no seat limits, nothing leaves your infrastructure by default
Where it falls short
- −YAML configuration gets long on large test suites
- −Reporting UI is lighter than a hosted platform's
- −Trace history is yours to store and manage
Promptfoo as an alternative to
Where Promptfoo shows up in our comparisons, and how it ranked.
Promptfoo head-to-head
Straight comparisons against the tools people weigh it against.