4 papers · 1 filter
Beyond Single Ground Truth: Reference Monism as Epistemic Injustice in ASR Evaluation
Anna Seo Gyeong Choi, Maria Teleki, James Caverlee +3
Automatic speech recognition (ASR) evaluation compares system output to ground truth transcripts, with Word Error Rate (WER) quantifying the distance between them. But ground truth…
Reverb: Open-Source ASR and Diarization from Rev
Nishchal Bhandari, Danny Chen, Miguel Ãngel del RÃo Fernández +10
Today, we are open-sourcing our core speech recognition and diarization models for non-commercial use. We are releasing both a full production pipeline for developers as well as pa…
Style-agnostic evaluation of ASR using multiple reference transcripts
Quinten McNamara, Miguel Ãngel del RÃo Fernández, Nishchal Bhandari +4
Word error rate (WER) as a metric has a variety of limitations that have plagued the field of speech recognition. Evaluation datasets suffer from varying style, formality, and inhe…
Quantification of stylistic differences in human- and ASR-produced transcripts of African American English
Annika Heuser, Tyler Kendall, Miguel del Rio +4
Common measures of accuracy used to assess the performance of automatic speech recognition (ASR) systems, as well as human transcribers, conflate multiple sources of error. Stylist…