WhisperX
Accurate word timestamps and speaker labels — Whisper's two weak spots, fixed.
90
sovereigntyWhisperX wraps Whisper with forced phoneme alignment to produce genuinely accurate word-level timestamps, and adds speaker diarization so output is attributed by speaker. Those are precisely the two things plain Whisper does poorly and the two things a managed API is usually bought for. If your product needs subtitles that land on the word, or meeting transcripts that say who spoke, this closes the gap. BSD-2-Clause, though note the diarization component it uses carries its own model terms worth checking.
OPEN SOURCEBSD-2-ClauseSELF-HOSTLOCAL-FIRST
LicenseBSD-2-Clause
PricingFree. Diarization models may require accepting separate terms.
Open sourceYes
Self-hostableYes
Local-first dataYes
What it does well
- +Word-level timestamps accurate enough for subtitles
- +Speaker diarization included in the pipeline
- +Batched inference makes it fast on long recordings
- +BSD-2-Clause
Where it falls short
- −Diarization models have their own licence terms to review
- −More moving parts than faster-whisper alone
- −Heavier GPU memory requirement with diarization enabled
WhisperX as an alternative to
Where WhisperX shows up in our comparisons, and how it ranked.
WhisperX head-to-head
Straight comparisons against the tools people weigh it against.