4 papers · 1 filter
TRACE: Training-aware Robustness and Challenge Evaluation for Text-Vision Embedding Models
Pume Tuchinda, Tinnakit Udsa, Peerat Limkonchotiwat
Existing benchmarks for text-vision embedding models report aggregate scores that obscure whether performance comes from general capabilities, task-specific adaptation, or robustne…
Responsible Benchmarking of Fairness for Automatic Speech Recognition
Felix Herron, Ange Richard, François Portet +2
Many studies have shown automatic speech processing (ASR) systems have unequal performance across speakergroups (SG's). However, the manner in which such studies arrive at this con…
Identifying and typifying demographic unfairness in phoneme-level embeddings of self-supervised speech recognition models
Felix Herron, Solange Rossato, Alexandre Allauzen +1
Modern automatic speech recognition (ASR) systems have been observed to function better for certain speaker groups (SGs) than others, despite recent gains in overall performance. O…
Where Do Self-Supervised Speech Models Become Unfair?
Felix Herron, Maja Hjuler, Solange Rossato +2
Speech encoder models are known to model members of some speaker groups (SGs) better than others. However, there has been little work in establishing why this occurs on a technolog…