Transcription Accuracy

We measure transcription accuracy on public test sets with human reference transcripts, so anyone can check the numbers. The audio goes through the same pipeline that serves POST /v1/transcripts: the same chunking, voice detection, model and decoding settings.

Results (Brazilian Portuguese)

Test set	Speech	What was transcribed	Word error rate	Accuracy
FLEURS pt_br, test split	read	all 919 utterances, 3.2 hours	4.10%	95.9%
CORAA MuPe, test split	spontaneous life-story interviews	3,000 randomly sampled segments from 30 interviews, 3.2 hours	17.87%	82.1%

Measured on October 7, 2026. Accuracy is 100% minus the word error rate.

What the numbers mean

Word error rate (WER) counts, for every 100 words of the human reference, the words the transcript got wrong: substituted, missing or added.
Read speech (FLEURS) is the clean case: one person reading a sentence. It is the most used public benchmark for Portuguese, so it compares across providers that publish it.
Spontaneous speech (MuPe) is people telling their life stories, with hesitations, repetitions and regional accents. It is the harder case and closer to classes, interviews and meetings. For reference, the paper that published MuPe reports 15.99% WER for its best model, trained on that same dataset, and 19.9% to 38.4% for earlier Brazilian Portuguese models (Leal et al., COLING 2025).
Most of the MuPe error is hesitation. The reference transcribes every "né", "eh" and repeated word; our transcript leaves most of them out, and each one counts as an error. Without hesitations the WER is 17.31%; on segments of 5 seconds or more it is 14.6%.

How we measure

Data. The FLEURS pt_br test split in full, and a fixed random sample of 3,000 segments (seed 8482) of the MuPe test split, scored segment by segment as in the dataset paper. Three MuPe segments listed in the dataset's index are missing from its published audio and are left out. Both datasets allow commercial use (CC-BY 4.0 and CC-BY-SA 4.0).
Transcription. Each file goes through the production pipeline: decode to 16 kHz mono, split into chunks of about 45 seconds at silences, skip chunks with no voice (Silero VAD), transcribe with Whisper large-v3-turbo with beam search 5, collapse decoder loops, merge the words. Language set to pt, no glossary.
Normalization. Applied to the reference and the transcript alike: lowercase, punctuation removed, numbers written out in Portuguese, hyphens turned into spaces. Non-speech notes in the MuPe reference, such as "(risos)", are removed.
Scoring. Corpus-level WER with jiwer: total errors divided by total reference words.

Known limits

Music and singing. Over sung or musical passages the model can still invent or repeat text. Chunks with no voice are skipped, but a song with vocals is not.
Repeated takes. The four worst FLEURS items are recordings where the speaker read the sentence twice. The transcript has both readings, which counts as added words. They stay in the number; without them the WER would be 3.65%.
Other languages. About 99 languages are supported, but these numbers are for Brazilian Portuguese only.
Other providers. Accuracy published on private test sets cannot be compared with these numbers. To compare two services fairly, run both on the same audio with the same normalizer.

Reproduce it

Everything above is public: the datasets, the sample seed, the normalization rules and the scoring library. Send the same audio to the API with language: "pt", normalize both texts the same way and compute the WER. A full run of both sets is 3,919 files and 6.4 hours of audio; since each file is billed per started minute, it costs about $1.31.