Research
Benchmarking speech-to-text accuracy in 2026
Part one ended with a promise. No public benchmark scores all the major engines and the open models on the same audio with the same scorer, so we said we would do it ourselves. This is that benchmark.
We sent an identical test set through each engine, scored every transcript with one open scorer, and put a 95% confidence interval on each number. No single engine wins: the best one depends on the audio. The engine tied for the lowest error on clean read speech collapses to the worst of the engines we finished on hard conference call audio. The engine that wins that hard audio is only mid-pack on clean.
How we tested
We used two English test sets that sit at opposite ends of difficulty.
- LibriSpeech test-clean: 2,620 utterances, 324.2 minutes of clean read audiobook speech. The easy end, where good engines cluster below 3% error.
- Earnings-22: 2,741 utterances, 325.7 minutes of real earnings call audio, built by rev.com as an accent benchmark, with a range of accents and financial vocabulary. The hard end.
Every engine got the same audio files. We scored each transcript against the reference with the OpenAI Whisper text normalizer (lowercasing, normalizing numbers to one written form, stripping punctuation) so that "$100" and "one hundred dollars" are not counted as errors, then computed corpus word error rate. This is the same convention the Open ASR Leaderboard uses: normalize first, then score. The scoring is aligned, but our dataset slice, model versions, and decoding are our own, so read our numbers against each other, not against the leaderboard's cells.
Word error rate (WER) is the share of reference words the engine got wrong, counting substitutions, deletions, and insertions. Lower is better. On both sets the engines at the top land within tenths of a percent of each other, so a single engine's confidence interval is too blunt to separate two of them. To ask whether two engines really differ, we use a paired bootstrap. Because every engine transcribed the same clips, we can resample the clip set 1,000 times (fixed seed) and measure the gap between the two engines on each resample. That produces a range for the gap: if the whole range stays on one side of zero, the difference is real; if the range includes zero, the two are within noise of each other and we call them tied. The interval shown for each engine on its own is there for reference, but every tie claim in the text comes from this paired test.
Each row is the WER difference between two engines measured on the same clips, first minus second, so a point left of zero means the first engine had the lower error. The bar is the 95% interval from 1,000 resamples. When it clears zero the gap is real; when it crosses zero, as OpenAI and AssemblyAI do on clean speech, the pair is tied.
One rule matters for reading the tables. We score each engine only on the clips it actually returned, and report its coverage. A clip an engine refused is a coverage gap, not a wrong transcript, so it does not count against the score. This keeps one engine's inability to accept a file from masquerading as an accuracy problem, and it keeps the caveat visible: a number at 84% coverage is not measured on the same clips as a number at 100%.
One limit is worth stating before the numbers. Both test sets are public, and public benchmarks leak into training data. A 2025 study of speech-model contamination found about two-thirds of the LibriSpeech test sentences, and a third of Common Voice, already sitting in the Pile, a corpus commonly used to train models. OpenAI's own Whisper paper noted back in 2022 that some commercial systems may have trained on these public sets, so their scores "may not be accurately reflecting the relative robustness of the systems." We cannot see what any engine here trained on, so we cannot rule out that some have already seen these exact clips, which would let them score well for the wrong reason: memory of the recording, not accuracy on audio they have never heard. That risk is largest for LibriSpeech, the oldest and most used set, but Earnings-22 is public too. So treat a very low clean speech score as a best case, and read these numbers as a comparison between engines on this audio, not a promise about yours.
Clean speech: everyone is good, and the top is a tie
On LibriSpeech test-clean the field is tight, tight enough that the paired test separates engines the eye would call the same. OpenAI and AssemblyAI are tied for the lowest error: their WER differs by 0.08 points and the paired interval spans zero. Everything below them, starting with Fish 0.18 points behind AssemblyAI, is behind by a margin the paired test calls real, even though every gap on this table is a fraction of a point. Two engines we added after part one land in this pack: Inworld's STT-1 at 1.71%, tied by the paired test with Parakeet v2 and Fish just below the top two, and Cartesia's Ink-Whisper at 2.01%, tied with Groq and ElevenLabs a step lower. The whole top of the table is good clean speech transcription; the interesting movement is on the next one.
| Engine | WER % | 95% CI | Coverage | Type |
|---|---|---|---|---|
 OpenAI gpt-4o-transcribe | 1.47 | 1.33-1.63 | 100% | Cloud |
 AssemblyAI Universal-3.5 Pro | 1.55 | 1.40-1.69 | 100% | Cloud |
 NVIDIA Parakeet TDT 0.6B v2 | 1.69 | 1.54-1.84 | 100% | Open weights |
 Inworld STT-1 | 1.71 | 1.57-1.87 | 99.9% | Cloud |
 Fish Audio (transcribe-1) | 1.73 | 1.52-1.96 | 100% | Cloud |
 xAI Grok (grok-stt) | 1.82 | 1.66-1.99 | 100% | Cloud |
 Apple SpeechAnalyzer | 1.82 | 1.66-1.99 | 100% | On-device (macOS) |
 Gladia Solaria | 1.85 | 1.70-2.01 | 100% | Cloud |
 Lemonfox (Whisper large-v3) | 1.93 | 1.76-2.10 | 100% | Cloud |
 Groq (Whisper large-v3-turbo) | 1.98 | 1.80-2.14 | 100% | Cloud |
 ElevenLabs Scribe | 1.99 | 1.81-2.18 | 100% | Cloud |
 Cartesia Ink-Whisper | 2.01 | 1.83-2.21 | 100% | Cloud |
| NVIDIA Parakeet TDT 0.6B v3 | 2.15 | 1.92-2.40 | 100% | Open weights |
 Azure AI Speech | 2.19 | 2.02-2.37 | 100% | Cloud |
 Speechmatics Enhanced | 2.22 | 2.05-2.39 | 100% | Cloud |
 Google Chirp | 2.32 | 2.14-2.49 | 100% | Cloud |
 Gemini 2.5 Flash | 2.49 | 2.28-2.68 | 100% | Cloud (LLM) |
 Deepgram Nova-3 | 2.50 | 2.30-2.70 | 100% | Cloud |
 Soniox | 2.78 | 2.57-2.98 | 100% | Cloud |
 Rev AI (Reverb) | 4.44 | 4.21-4.69 | 98.5% | Cloud |
Scored 2026-07-16 on LibriSpeech test-clean. On-device models are scored on the same clips with the same scorer, from our part-one runs. Amazon Transcribe is absent here: we ran it on the SPGISpeech set only, because its 15-second per-job minimum makes these short clips too expensive to justify.
Coral marks the two engines tied for the lowest error by the paired test. Whiskers are 95% confidence intervals. Amazon is omitted because we ran it on the SPGISpeech set only.
Two things worth noting. OpenAI's gpt-4o-transcribe is tied with AssemblyAI for the lowest number here, which sets up the next section, because it does not stay near the top. And the two hosted Whisper builds, Lemonfox (large-v3) at 1.93% and Groq (large-v3-turbo) at 1.98%, land within five hundredths of each other, two Whisper variants served by different providers. Lemonfox exposes no model choice, so the large-v3 label is its own description of the service, not something we selected.
Rev is the one engine off the clean speech pack, at 4.44%, which is 1.66 points above the next highest finished engine (Soniox at 2.78%). Its 98.5% coverage is not the cause, but it is a caveat: Rev rejects any clip shorter than about two seconds, so it skipped 38 of the 2,620 clips and we scored it on the rest, a slightly different set than the 100%-coverage engines. The 4.44% is real error on the clips it accepted.
Hard audio: the ranking comes apart
Send the same engines at earnings call audio and the order barely resembles the clean speech table.
| Engine | WER % | Norm % | 95% CI | Coverage | Type |
|---|---|---|---|---|---|
| xAI Grok (grok-stt) | 9.53 | 9.17 | 9.09-9.96 | 100% | Cloud |
| Fish Audio (transcribe-1) | 10.27 | 9.59 | 9.86-10.71 | 100% | Cloud |
| Inworld STT-1 | 10.29 | 9.71 | 9.86-10.74 | 100% | Cloud |
| AssemblyAI Universal-3.5 Pro | 10.43 | 9.37 | 9.98-10.91 | 100% | Cloud |
| Azure AI Speech | 11.17 | 10.11 | 10.72-11.65 | 99.9% | Cloud |
| NVIDIA Parakeet TDT 0.6B v3 | 11.20 | 10.84 | 10.71-11.74 | 100% | Open weights |
| Gemini 2.5 Flash | 11.21 | 10.89 | 10.69-11.74 | 100% | Cloud (LLM) |
| NVIDIA Parakeet TDT 0.6B v2 | 11.23 | 10.53 | 10.73-11.73 | 100% | Open weights |
| Cartesia Ink-Whisper | 11.59 | 10.55 | 11.12-12.09 | 100% | Cloud |
| Soniox | 11.61 | 10.81 | 11.11-12.11 | 100% | Cloud |
| Gladia Solaria | 11.65 | 10.60 | 11.07-12.43 | 100% | Cloud |
| Lemonfox (Whisper large-v3) | 11.67 | 10.64 | 11.20-12.17 | 100% | Cloud |
| Groq (Whisper large-v3-turbo) | 11.90 | 10.87 | 11.43-12.40 | 100% | Cloud |
| Apple SpeechAnalyzer | 12.03 | 11.53 | 11.52-12.56 | 100% | On-device (macOS) |
| Speechmatics Enhanced | 12.28 | 11.66 | 11.84-12.73 | 100% | Cloud |
| Whisper large-v3 | 12.35 | 11.34 | 11.87-12.86 | 100% | Open weights |
| Google Chirp | 12.67 | 11.77 | 12.19-13.19 | 99.9% | Cloud |
| Rev AI (Reverb) | 13.01 | 12.71 | 12.33-13.73 | 84.3% | Cloud |
| Whisper small | 14.92 | 13.94 | 14.27-15.57 | 100% | Open weights |
| ElevenLabs Scribe | 15.44 | 14.79 | 14.68-16.23 | 100% | Cloud |
| Deepgram Nova-3 | 16.13 | 15.20 | 15.48-16.81 | 100% | Cloud |
| OpenAI gpt-4o-transcribe | 18.55 | 17.53 | 17.62-19.48 | 100% | Cloud |
Scored 2026-07-16 on Earnings-22 (Groq finished later, on 2026-07-17). WER % is the verbatim score against references that keep every filler and false start. Norm % is a second scoring that drops residual fillers and collapses immediate repeated words (like "in in") on both the reference and the transcript, so engines that quietly tidy speech are not charged for it. Its provisional Groq score, held out until the run finished, was 10.85% at 24.6% coverage; the full run came in at 11.90%, a reminder that a partial score over the first clips in file order can mislead. Amazon Transcribe is absent: we ran it on the SPGISpeech set only, for the cost reason noted with the clean table.
Coral marks the engine with the lowest error by the paired test: xAI Grok, clearly ahead of Fish, Inworld, and AssemblyAI, which tie for second. Whiskers are 95% confidence intervals. OpenAI, tied for the top of the clean speech chart above, is last here. Finished engines only.
Look at what moved. OpenAI's gpt-4o-transcribe went from the lowest error on clean speech (1.47%) to the worst of the finished engines here (18.55%). Deepgram went from 2.50% to 16.13%, ElevenLabs from 1.99% to 15.44%. Every engine gets worse on harder audio, but these three give up between 13.4 and 17.1 points of WER, while no other finished engine gives up more than 10.4. That is exactly the failure a clean speech demo hides.
One thing that failure is not: a scoring artifact. Earnings-22 references keep every filler and false start, and a few engines quietly clean those up, so a verbatim error rate could be charging them for being tidy. To check, we scored a second time with disfluencies normalized away on both sides, dropping residual fillers and collapsing immediate repeated words, shown as "Norm %" in the table. Every engine drops between 0.3 and 1.1 points, and the middle reshuffles a little, AssemblyAI (which tidies the most) rising from fourth to second. But the shape holds: Grok still leads at 9.17%, and OpenAI is still last at 17.53%, with Deepgram at 15.20% and ElevenLabs at 14.79%. The collapse on hard audio is real recognition error, not a penalty for scoring against verbatim references.
No engine wins both sets. On hard audio the lowest error belongs to xAI's Grok, at 9.53%, and the paired test puts it clearly ahead of the field: 0.74 points below Fish, 0.76 below Inworld, and 0.90 below AssemblyAI, all three intervals excluding zero. Those three trail Grok but tie each other for second, Fish, Inworld, and AssemblyAI landing within 0.16 points. Yet Grok is only mid-pack on clean speech, at 1.82%, tied with Fish and Apple and a third of a point behind OpenAI. So the winner of the hard set is unremarkable on the easy one, and the winners of the easy set, OpenAI and AssemblyAI, do not win the hard one. A few engines come closest to steady. AssemblyAI is tied for the top on clean and tied for second on hard; Fish and Inworld sit near the top of both. If you only measured on clean audio you would not have predicted any of it: that OpenAI drops to last, that Grok tops the hard set from the middle of the clean one, or that the clean leaders never lead on hard.
Both columns share one WER axis. Teal engines stay near the top on hard audio (Grok has the lowest hard error and the smallest drop of any engine); coral engines collapse (OpenAI, Deepgram, and ElevenLabs each give up more than 13 points); gray lines are the rest. Twenty engines that finished both datasets. Hover a line for its exact numbers.
A third set the models were unlikely to train on
Both sets so far are public, so the caveat from the top of this piece still stands: a very low clean speech score could be memory of the recording rather than skill. So we ran a third set picked for one property, that it is hard to scrape. SPGISpeech is 5,000 hours of earnings call audio transcribed by S&P Global and released by Kensho for non-commercial research only, behind a registration agreement that forbids passing the data on. That gating makes it far less likely to sit in the corpora scraped from the open web that swept up LibriSpeech. We scored a 1,200-clip sample, 3.05 hours, with the same normalizer, coverage rule, and paired test as the other two. It is earnings call audio like Earnings-22, yet the scores land much closer to the clean set, a reminder that financial audio is not one level of difficulty.
The order changes a third time. AssemblyAI has the lowest error, and the paired test puts its lead beyond doubt: 0.39 points clear of the next engine, with the interval excluding zero.
| Engine | WER % | 95% CI | Coverage | Type |
|---|---|---|---|---|
| AssemblyAI Universal-3.5 Pro | 1.87 | 1.67-2.07 | 100% | Cloud |
| NVIDIA Parakeet TDT 0.6B v2 | 2.04 | 1.84-2.22 | 100% | Open weights |
| Google Chirp | 2.26 | 2.07-2.46 | 100% | Cloud |
| Azure AI Speech | 2.39 | 2.19-2.61 | 99.7% | Cloud |
| Fish Audio (transcribe-1) | 2.61 | 2.39-2.84 | 100% | Cloud |
| Deepgram Nova-3 | 2.94 | 2.52-3.47 | 100% | Cloud |
| Speechmatics Enhanced | 3.00 | 2.77-3.25 | 100% | Cloud |
| Lemonfox (Whisper large-v3) | 3.02 | 2.76-3.29 | 100% | Cloud |
| Whisper large-v3 | 3.03 | 2.77-3.31 | 100% | Open weights |
| Soniox | 3.10 | 2.84-3.35 | 100% | Cloud |
| Groq (Whisper large-v3-turbo) | 3.12 | 2.83-3.43 | 100% | Cloud |
| ElevenLabs Scribe | 3.17 | 2.90-3.43 | 100% | Cloud |
| Gemini 2.5 Flash | 3.62 | 3.33-3.89 | 100% | Cloud (LLM) |
 Amazon Transcribe | 3.73 | 3.45-4.00 | 100% | Cloud |
| xAI Grok (grok-stt) | 3.80 | 3.47-4.17 | 100% | Cloud |
| NVIDIA Parakeet TDT 0.6B v3 | 3.84 | 3.55-4.12 | 100% | Open weights |
| Whisper small | 4.10 | 3.52-4.88 | 100% | Open weights |
| Apple SpeechAnalyzer | 4.34 | 4.05-4.65 | 100% | On-device (macOS) |
| OpenAI gpt-4o-transcribe | 9.49 | 8.31-10.65 | 100% | Cloud |
Scored 2026-07-17 on a 1,200-clip sample of SPGISpeech (3.05 hours), with the same normalizer and coverage rule as the other sets (Groq finished later, on 2026-07-18). The on-device models (Parakeet, Whisper, Apple) were run on this set too.
Coral marks AssemblyAI, the clear leader by the paired test. Parakeet v2, a free open model, is second. OpenAI's gpt-4o-transcribe, tied for the lowest error on clean speech, is last here by more than five points. Whiskers are 95% confidence intervals.
This is the check the contamination caveat needed. The engines that were strong on clean speech, and did not fall apart on hard audio, stay strong on a set they almost certainly never trained on. AssemblyAI reads 1.55% on clean LibriSpeech and 1.87% here. The free open model holds too: NVIDIA's Parakeet v2 is second at 2.04%, ahead of every cloud engine but AssemblyAI, on audio no one can train on without signing for it. Google Chirp and Azure, tied just behind by the paired test, sit within half a point of their clean scores. If those clean numbers were memorized recordings, they would not survive audio the models have not seen, and they do. So the clean speech rankings are mostly real skill, not leakage.
With one loud exception. OpenAI's gpt-4o-transcribe, tied for the lowest error on clean speech at 1.47%, collapses to 9.49% here, last by more than five points and the same break it showed on Earnings-22. Its clean speech win does not travel. The winner of the hard set does not repeat either: xAI's Grok, which led Earnings-22, is mid-pack on SPGISpeech at 3.80%, tied with Amazon and Gemini. Three sets, three orders, and the only engine near the top of all three is AssemblyAI.
One limit on this set. The audio is public earnings calls, so we cannot claim zero exposure for any engine, only that this transcribed and aligned corpus, held behind a research license that bans redistribution, is far less likely to be in a training scrape than a public domain set.
The on-device models are already in the fight
We ran the on-device models on both sets, the open-weight Parakeet and Whisper plus Apple's built-in SpeechAnalyzer, the same ones from part one that run on your own laptop. They are not a sideshow. On clean speech NVIDIA's Parakeet TDT 0.6B v2 scored 1.69%, 0.21 points behind OpenAI. The paired test says that gap is real, but 0.21 points is a fifth of a percent: a free model on hardware you already own, within touching distance of the best paid API on clean audio. On hard audio the gap widens. Parakeet v3 scored 11.20% and Apple's SpeechAnalyzer 12.03%, against Grok's 9.53% at the top, so Parakeet v3 is 1.67 points off the lead. Still close for a free model running on your own machine, but no longer the near tie it manages on clean.
So the gap between the best cloud API and a free model running locally is about a fifth of a point on clean audio and about 1.7 points on hard audio, both real by the paired test. On clean it is negligible; on hard it is real but still under two points. Whether that is worth a per-hour API bill is the tradeoff part one laid out in its cost section, now with an accuracy number attached.
What it costs
Accuracy only means something next to price. The rates below are US dollars per hour of audio from each provider's published pricing on July 15, 2026. The engines that were in part one carry the same figures; the engines added here come from Fish Audio's pricing, plus Gemini's, Lemonfox's, xAI's, Cartesia's, and Inworld's pricing pages. Cartesia publishes no dollar rate for its speech-to-text, so we derived one from its credit price, noted under the table. The APIs do not report a per-call charge, so these are estimates, computed from the audio duration each engine processed times its published rate.
| Engine | Cost per audio-hour | Clean WER | Hard WER |
|---|---|---|---|
| Groq (Whisper turbo) | $0.04 | 1.98% | 11.90% |
| Gemini 2.5 Flash | $0.066 (token-billed) | 2.49% | 11.21% |
| Cartesia Ink-Whisper | $0.09* | 2.01% | 11.59% |
| Soniox | $0.10 | 2.78% | 11.61% |
| xAI Grok (grok-stt) | $0.10 | 1.82% | 9.53% |
| Inworld STT-1 | $0.15 | 1.71% | 10.29% |
| Lemonfox | $0.167 | 1.93% | 11.67% |
| Azure AI Speech | $0.18 | 2.19% | 11.17% |
| AssemblyAI | $0.21 | 1.55% | 10.43% |
| ElevenLabs Scribe | $0.22 | 1.99% | 15.44% |
| Speechmatics Enhanced | $0.30 | 2.22% | 12.28% |
| Fish Audio | $0.36 | 1.73% | 10.27% |
| OpenAI gpt-4o-transcribe | $0.36 | 1.47% | 18.55% |
| Rev AI | $0.20* | 4.44% | 13.01% |
| Deepgram Nova-3 | $0.46 | 2.50% | 16.13% |
| Gladia (Solaria) | $0.61 | 1.85% | 11.65% |
| Google Chirp | $0.96 | 2.32% | 12.67% |
* Rev's published rate is $0.20/hr, but its 15-second per-job minimum on our short clips raised the modeled cost to $0.35-0.40/hr; assumed, not invoiced. Gemini is billed on tokens, so its per-hour figure is a conversion. Cartesia publishes no dollar STT rate; the $0.09 is derived from its Pro credit price (1 credit per 2 seconds of batch audio at $5 per 100,000 credits). Inworld's $0.15 is its On-Demand rate, which drops to $0.10 on paid tiers.
A few things stand out once accuracy and price sit in the same row. The engine that wins hard audio is also one of the cheapest: xAI's Grok leads Earnings-22 at 9.53% for $0.10 an hour. The three engines tied behind it for second cost $0.36 (Fish), $0.21 (AssemblyAI), and $0.15 (Inworld), so Inworld is the cheapest way into that second-place group on hard audio. On clean speech the value pick is different: Groq's hosted Whisper is the cheapest API we tested at $0.04 an hour and posts 1.98%, with Cartesia's Ink-Whisper just behind at $0.09 and 2.01%. Google Chirp is the most expensive at $0.96 an hour without leading either table, and OpenAI's gpt-4o-transcribe posts the lowest clean error but the worst hard error we finished, 9 points behind Grok.
The quirks of calling these APIs
The docs give you an endpoint and a price. They leave out the small things that each cost an afternoon. Part one covered the rate limits we ran into; these are the integration quirks that changed our transcripts or our bill, in case you are wiring these up yourself.
- Rev AI returns a plain-text transcript that prepends a speaker and timestamp label, "Speaker 0 00:00:00", to every turn. Scored as words, that label alone read as a 28.9% error rate until we switched to Rev's JSON transcript and joined the word elements. Rev also rejects clips shorter than about two seconds and bills a 15-second minimum per job.
- Amazon Transcribe has no synchronous pre-recorded endpoint. Batch jobs read from S3 one file at a time, which is impractical for thousands of clips, so we sent each clip through the streaming API instead. It also bills a 15-second minimum per request, so clips shorter than that, which is most of ours, each bill as 15 seconds and roughly double the audio-hours charged.
- Azure AI Speech puts the region in the hostname, so a key only works against the region its resource was created in. The fast-transcription endpoint returned 429s heavily at six concurrent requests, and we had to drop to two. Its free F0 tier allows one request at a time and caps at five audio-hours a month, less than a single one of our test sets.
- xAI Grok needs the audio to be the last field in the multipart form, because its API is order-sensitive and HTTP clients send text fields first. We also disable output formatting to get the words as spoken rather than rewritten into forms like "$100". And a 403 reading "used all available credits or reached its monthly spending limit" can fire with credits sitting on the account, when the monthly spending limit is still set to zero.
- Soniox caps the account at 1,000 stored files and 2,000 transcriptions. A long run fills that quota and uploads start failing with connection resets and 429s that look like rate limiting but are not. The fix is to delete each file and transcription right after reading its result, with a retry, since the deletes use the same throttled endpoint.
- Gemini is a language model, not a dedicated recognizer. We prompt it for a verbatim transcript and set its thinking budget to zero so it does not spend reasoning tokens, and it still cleans up disfluencies, which is why we label it as LLM transcription instead of comparing it directly.
- Lemonfox wants the language as a full word, "english", where every other API takes the ISO code "en".
- Deepgram needs smart formatting turned off, or it rewrites numbers and dates into forms the reference transcript does not use, and the scorer counts the difference as errors.
Speed
We measured per-call latency on its own, because the accuracy runs use many workers at once and that distorts timing. Each engine transcribed the same 40 short clips one at a time, and we recorded the wall time from sending the audio to holding the transcript. Two kinds of engine sit in this table and they are not comparable. A synchronous engine returns the transcript in the same response. An asynchronous engine takes the upload, queues a job, and makes you poll for the result, so its time includes queue turnaround and our poll interval, not just recognition.
| Engine | Median | p90 | Type |
|---|---|---|---|
| Cartesia Ink-Whisper | 0.36 | 0.57 | Sync |
| Deepgram Nova-3 | 0.51 | 0.96 | Sync |
| Fish Audio | 0.56 | 0.85 | Sync |
| xAI Grok (grok-stt) | 0.57 | 0.78 | Sync |
| OpenAI gpt-4o-transcribe | 1.00 | 1.65 | Sync |
| ElevenLabs Scribe | 1.07 | 1.59 | Sync |
| Inworld STT-1 | 1.34 | 2.71 | Sync |
| Lemonfox | 1.43 | 1.91 | Sync |
| Google Chirp | 1.53 | 2.01 | Sync |
| Azure AI Speech | 1.56 | 1.80 | Sync |
| Gemini 2.5 Flash | 1.73 | 2.07 | Sync |
| AssemblyAI | 2.44 | 2.91 | Async |
| Amazon Transcribe | 2.45 | 4.23 | Sync |
| Soniox | 4.27 | 4.64 | Async |
| Speechmatics Enhanced | 4.82 | 5.19 | Async |
| Gladia (Solaria) | 6.30 | 7.14 | Async |
| Rev AI | 9.10 | 10.95 | Async |
Measured 2026-07-17 from one location, 40 clips per engine sent one at a time; your latency will vary with network, region, and load. Async engines (AssemblyAI, Soniox, Speechmatics, Gladia, Rev) submit a job and poll, so their times include queue turnaround and are not comparable to a synchronous request. On-device models are not here; their speed is a real-time factor from the runs, a different measurement.
The fast synchronous engines return in about half a second or less: Cartesia's Ink-Whisper is quickest at 0.36s, then Deepgram at 0.51s, Fish at 0.56s, and xAI's Grok at 0.57s median. That still adds to Grok's case, since it also had the lowest error on hard audio and one of the lowest prices. The other synchronous engines land between one and 2.5 seconds. The asynchronous engines are slower by construction, from AssemblyAI at 2.4s to Rev at 9.1s, because you are timing a job queue, not a single request. For a product that shows words as the person is still speaking, none of these batch numbers is the metric that matters anyway; that is a streaming connection, which we did not measure here.
What is missing, and what we will not claim
The honest state of this benchmark at the time of this draft:
- Coverage. Rev is scored on 98.5% of LibriSpeech and 84.3% of Earnings-22 because it rejects clips under about two seconds. Its numbers are real on the clips it accepted, but they are not measured on the identical set as the 100% engines. Inworld is at 99.9% of LibriSpeech, two clips short after a repeated server error on them.
- Amazon on the main sets. Amazon Transcribe was run on the SPGISpeech set only, because its 15-second per-job minimum roughly doubles the billed cost on the short LibriSpeech and Earnings-22 clips, so it has no clean or hard number here.
- Cartesia and Inworld on two sets. We added Cartesia Ink-Whisper and Inworld STT-1 after the first runs and scored them on LibriSpeech and Earnings-22, not the later SPGISpeech sample, so they carry a clean and a hard number but no third-set contamination check.
- Resemble is not here. We started a Resemble run, but its free credits covered only half of one set before the account hit a payment wall, and finishing both sets would have cost several dollars per audio-hour, far above any other engine we tested. We stopped it rather than pay for a single provider's numbers, so Resemble has no score.
- Cost is modeled, not invoiced. We computed it from audio duration and published rates, not from a bill.
- Three sets, not yours. LibriSpeech, Earnings-22, and SPGISpeech cover clean read speech and two kinds of financial audio. Your audio, whether it is medical, multi-speaker, or accented in a way none of the three covers, can reorder these tables again.
That last point is the whole reason we built this. A single benchmark, even an honest one, is still not your benchmark. The method here is reproducible (the benchmark code is available on request). Point it at your own audio before you commit to an engine, because as the three rankings above show, the ranking you inherit from someone else's test set may not survive contact with yours.