How Accurate Is OpenAI Whisper in 2026? (Real-World WER by Condition and Language)

TL;DR — the headline numbers

One number can’t describe Whisper’s accuracy honestly. Here are the ranges by audio condition for English:

What “accurate” means — Word Error Rate explained

Word Error Rate (WER) is the standard accuracy metric for speech-to-text. It measures the percentage of words the transcript gets wrong, counting substitutions, insertions, and deletions equally.

• 5% WER = 5 wrong words per 100 = reads cleanly; light proofreading
• 10% WER = 1 in 10 wrong = readable but edit-required before publication
• 25% WER = 1 in 4 wrong = unreliable for direct quoting; you need to re-listen

The metric’s blind spot:WER weights every error equally. One missed phone number in a voicemail is functionally worse than 10 missed filler words, but WER doesn’t see that. Always spot-check proper nouns, numbers, and named entities — they’re the highest-impact error sites regardless of which ASR you use.

Whisper model sizes — accuracy/speed tradeoff

OpenAI publishes Whisper in six sizes. Production deployments mostly use large-v3 or large-v3-turbo; smaller sizes are for edge deployment or hardware-constrained scenarios.

Most production systems use large-v3-turbo — the speed gain (8×) justifies the small accuracy cost (~1% WER) in almost every batch-transcription scenario.

WER by audio condition (the table nobody else publishes)

Whisper’s headline benchmark numbers are honest but misleading — they reflect the LibriSpeech corpus (clean-read audiobook English). Real audio looks very different. Here’s the breakdown by realistic condition, with the reason each one degrades accuracy:

The takeaway:if your audio is studio quality, expect Whisper’s headline numbers. If it’s anything else, plan for 2-5× higher error rates. Phone numbers, named entities, and technical jargon are mis-heard at higher rates than common words regardless of condition. For video-source-specific WER expectations (marketing videos, screen recordings, meeting recordings, field video, music-over-speech), Video to Text has a companion breakdown by video type. The music-over-speech row above is especially relevant for short-form social media (TikTok, Instagram Reels, YouTube Shorts), where a background music track is nearly universal. The other axis that changes WER predictably is audio format: uncompressed WAV vs low-bitrate MP3 produces measurably different results on identical source recordings — WAV to Texthas the WER-by-format comparison table. Also worth reading if you’re generating subtitle files: Whisper’s word-level timestamps are accurate to ~200ms but its segment boundaries are heuristic — see Video to SRT for how that interacts with professional captioning standards.

WER by language

Whisper supports 99 languages, but accuracy varies sharply. The OpenAI Whisper paper publishes WER on the Fleurs benchmark by language; the grouped summary:

Top tier (under 5% WER on Fleurs)

English, Spanish, French, German, Italian, Portuguese, Japanese, Korean, Russian, Chinese (Mandarin). These languages have abundant training data and Whisper performs near-English levels on clean audio.

Mid tier (5-15% WER)

Arabic, Hindi, Turkish, Dutch, Polish, Vietnamese, Indonesian, Hebrew, Greek, Czech, Swedish, Danish, Finnish, Norwegian, Romanian, Bulgarian, Ukrainian, Thai, Catalan. Production-usable with editing.

Low tier (15-30%+ WER)

Lower-resource languages — many African languages, several Southeast Asian dialects, less-represented Indian languages, Welsh, Basque. Whisper covers them in name but results are often unusable without significant cleanup or domain-specific fine-tuning.

The multilingual leadership claim

On most multilingual benchmarks (Fleurs, Common Voice), Whisper outperforms Deepgram Nova-3 and AssemblyAI Universal-2 — both of those are English-first models that added multilingual support later. If your use case is multilingual, Whisper is typically the right starting point.

Whisper variants — which to actually use

“Whisper” is several things. The original OpenAI release is one implementation; the community has built faster, smaller, and more feature-rich variants on top of the same models. All use the same or derived OpenAI weights — word accuracy differences are near-zero; speed, features, and hardware requirements differ substantially.

60-second decision tree:Production Python server → faster-whisper. Podcasts / multi-speaker meetings needing speaker labels → WhisperX. CPU-only, mobile, or offline desktop app → whisper.cpp. GPU batch processing → insanely-fast-whisper. English-only batch with lower cost → whisper-large-v3-turbo. Apple Silicon Mac → MLX Whisper.

faster-whisper

What it is:A reimplementation of Whisper using CTranslate2 as the inference backend. Same OpenAI weights, ~4× faster on GPU, ~2× faster on CPU, and significantly lower memory footprint. Maintained by SYSTRAN.

When to use: Any production Python service transcribing files or streams. This is the default choice for self-hosted transcription — better than base Whisper in every measurable way (except license — both are MIT so a wash there).

Install: pip install faster-whisper

Gotchas:First run downloads the CT2 model (~3 GB for large-v3). GPU inference needs matching CUDA/cuDNN versions or you’ll silently fall back to CPU. Use compute_type="float16" on GPU for the speed gains.

WhisperX

What it is:faster-whisper + Wav2Vec2 forced alignment (word-level timestamps accurate to ~100ms) + optional pyannote.audio speaker diarization. The complete “professional podcast” pipeline in one package. Maintained by Max Bain (m-bain).

When to use:Podcasts, multi-speaker meetings, any content where you need to say “Speaker A said X at 03:12” instead of just a wall of text. Also the right choice if you need reliable per-word timestamps for SRT/VTT export.

Install: pip install whisperx— but requires a HuggingFace token for the pyannote model (free, register at huggingface.co).

Gotchas:Diarization needs the HF token step — easy to miss. First run downloads Wav2Vec2 + pyannote weights (~2 GB extra). Batched processing (the source of the 7× speedup) needs enough GPU memory to hold the batch.

whisper.cpp

What it is:Georgi Gerganov’s pure C++ port using ggml tensor library. Runs Whisper without Python, without CUDA, with aggressive quantization (Q4/Q5/Q8) that shrinks models to run on phones and Raspberry Pi. The backbone of most offline mobile / desktop Whisper apps (MacWhisper, Whisperfile, Superwhisper, Handy).

When to use:Mobile app (iOS, Android), offline desktop app, embedded/edge device, or any system without Python. Also excellent for a personal “drop a file, get a transcript” utility that doesn’t need server infra.

Install: git clone github.com/ggerganov/whisper.cpp && make — or brew install for Mac.

Gotchas:Quantization at Q4 shaves 1–3% WER; stick with Q5 or Q8 if accuracy matters. CPU-only performance varies wildly by CPU generation — a modern Apple Silicon or AMD Ryzen 9 runs medium-v3 near real-time; an older Intel might struggle.

insanely-fast-whisper

What it is:A wrapper around HuggingFace Transformers that adds Flash Attention 2 batching for ~8× speedup on GPU. Maintained by Vaibhav Srivastav (Vaibhavs10). The go-to for bulk GPU batch jobs.

When to use:You have a folder of hundreds or thousands of audio files, a modern NVIDIA GPU (A100/H100 preferred; RTX 3090/4090 workable), and want them all transcribed as fast as possible. Not for single-file interactive use — the batching overhead makes single files no faster than faster-whisper.

Install: pipx install insanely-fast-whisper

Gotchas:Requires Flash Attention 2 (needs CUDA 11.8+, PyTorch with FA2 support). Batch size tuning is manual — too high and you OOM; too low and you lose the speedup.

whisper-large-v3-turbo

What it is:OpenAI-official pruned decoder of large-v3. Decoder layer count reduced from 32 to 4, which drops inference time ~8× on the decode step (the bottleneck for long audio) with minimal English accuracy loss. Released October 2024.

When to use:English-heavy batch transcription where you want the OpenAI-official model (some enterprise procurement processes require this). Also a good default for the OpenAI Whisper API — noticeably faster and cheaper per minute than large-v3.

Install: Available in HuggingFace Transformers as openai/whisper-large-v3-turbo, and in faster-whisper as a model name.

Gotchas:Non-English languages take a 2–5% WER penalty vs large-v3 — more pronounced on less-resourced languages. For multilingual content, stick with large-v3 or WhisperX.

MLX Whisper (Apple Silicon)

What it is:Native Apple Silicon implementation using MLX (Apple’s ML framework optimized for M-series chips). ~3× faster than reference Whisper on the same Mac.

When to use:You’re on Apple Silicon (M1/M2/M3/M4) and want maximum performance. Also the right choice for macOS apps that don’t want a CUDA dependency.

Gotchas: Apple-only. Requires macOS 13.5+ and Xcode command-line tools.

OpenAI Whisper API vs self-hosted

OpenAI’s hosted Whisper API is the simplest integration — a REST call, no infrastructure. Pricing is $0.006 per minute of audio (verified July 15, 2026 at openai.com/api/pricing). For high volume (10,000+ minutes/month), self-hosting faster-whisper on a modest GPU is cheaper per minute. For low volume or when the accounting of a fixed per-minute cost matters more than the price per minute, the API wins.

Which Whisper model the API runsis not always transparent — OpenAI has historically shipped large-v2 and large-v3 at different times. As of July 15, 2026, the API defaults to whisper-1 which is a large-v2-based model. large-v3 and turbo are available via the newer gpt-4o-transcribe and gpt-4o-mini-transcribe endpoints.

Whisper vs commercial alternatives

Honest comparison across the dominant ASR options in 2026. WER numbers are from each vendor’s published benchmarks plus the Open ASR Leaderboard.

The honest verdict

• Multilingual or self-host needed → Whisper. Nothing else covers 99 languages with an open-source license.
• Real-time streaming + telephony → Deepgram. Built for sub-300ms latency and phone-audio codecs.
• English meeting audio with strong speaker labels → AssemblyAI. Their diarization consistently outperforms WhisperX on multi-speaker real-world meetings.
• Already on Google Cloud → Chirp-2, despite higher cost — integration savings often dominate.

When Whisper fails — the failure modes

Hallucination during silence

Whisper has a documented and well-reproduced failure mode: during long silences or very low-volume audio, it invents text— often a repeated phrase like “Thank you for watching,” “Subtitles by the Amara.org community,” or a foreign-language sentence. The cause is the model’s training on YouTube-style content where these phrases follow silent sections in subtitle tracks.

A 2024 Stanford study documented hallucinations in 1.4% of Whisper transcripts of clinical audio, with some inventing entire fabricated medical content — a serious concern for healthcare use.

Mitigation:run Voice Activity Detection (VAD) preprocessing to skip silent segments, or use faster-whisper’s no_speech_threshold + vad_filter parameters to flag and drop low-confidence segments. WhisperX bundles VAD by default.

Music + speech overlap

When music plays under speech, Whisper often produces lyrics-as-transcript or skips the segment entirely. The model wasn’t trained to separate sources.

Mitigation: source separation preprocessing (Spleeter, demucs) to isolate vocals before transcription.

Multi-speaker overlap (no built-in diarization)

Whisper produces a single text stream regardless of how many people are talking. When two speakers overlap, output garbles or drops one speaker entirely.

Mitigation: pair Whisper with pyannote-audio for speaker diarization, or use WhisperX which bundles both.

Code-switching (mid-sentence language switches)

Speakers who switch languages mid-sentence (common in bilingual conversations) confuse Whisper’s language detection. Output often picks one language and mis-transcribes the other.

Mitigation: chunk audio at language boundaries if possible, or use specialized code-switching ASR (research models exist but not commercial yet).

Numbers and named entities

Phone numbers, addresses, drug names, proper nouns — the highest-error categories across all ASR, not just Whisper. Always spot-check these before relying on the transcript.

Mitigation: domain-specific post-processing (e.g., regex validation for phone numbers; lookup tables for known proper nouns) catches the most common errors.

Cost per minute

Pricing captured June 2026. Verify on each vendor’s current page before committing.

At what volume does self-host beat the API? Roughly 1,000-3,000 hours/month, depending on your engineering capacity. Below that, the OpenAI API is cheaper than running your own GPU when you factor in DevOps time.

When to use Whisper, when not to

Use Whisper when

• You need multilingual transcription across more than ~40 languages
• You need self-hosting (privacy, compliance, no cloud upload)
• You’re doing batch processing (latency doesn’t matter)
• Budget-sensitive and you have GPU capacity
• You’re building a research or academic project where open-source is required
• You’re an edge / mobile deployment (whisper.cpp on phones)

Don’t use Whisper when

• You need real-time streaming with sub-300ms latency → Deepgram wins
• You’re transcribing telephony (8 kHz codec) at scale → Deepgram has telephony-tuned models
• You need built-in diarizationwithout the engineering work → AssemblyAI’s diarization is stronger than WhisperX out of the box
• You need a HIPAA-compliant managed service → AssemblyAI Enterprise or Deepgram Enterprise with active BAA; or self-host Whisper on institutional hardware
• You don’t have the engineering capacity to manage GPU inference and the OpenAI API cost isn’t justified by your volume → use a managed alternative

How we use Whisper at DeluxeScribe

DeluxeScribe uses Whisper-family models in production. The specific stack: WhisperX for diarization + word-level timestamps, faster-whisper backend for speed, custom VAD preprocessing to mitigate the silence-hallucination problem, and post-processing for proper nouns and numbers.

We picked Whisper for two reasons that drove the decision:

• 99-language coverage— none of the commercial alternatives match this. For a multilingual transcription product it’s essentially the only viable starting point.
• Self-hosting on our own infrastructure — keeps cost per minute predictable as we scale and avoids per-call API charges

Where DeluxeScribe adds value on top of Whisper: API ergonomics, in-browser editor, six export formats (TXT, DOCX, PDF, SRT, VTT, JSON), speaker label cleanup, and tuned preprocessing for problematic conditions (phone audio, background noise). The transcription quality is what Whisper delivers; the surrounding product is what you pay for.