1 citations · 2 across the 8 of their papers we have counts for
9 papers
Trace Evolution Bench: Measuring Whether Agents Improve from Their Own Environment Traces
Fangfei Li, Chenyang Zhao, Long Wang +2
Self-evolving agents are usually scored by endpoint Gain: held-out success after an update minus success before. That difference can credit a frozen rescore as evolution. We introd…
Generative vs. Encoder Large Language Models for ASR Evaluation: A Comparative Study
Thibault Bañeras-Roux, Shashi Kumar, Driss Khalil +6
Automatic Speech Recognition (ASR) is typically evaluated using Word Error Rate (WER), which poorly reflects semantic similarity. While embedding-based metrics correlate better wit…
Evaluation of Automatic Speech Recognition Using Generative Large Language Models
Thibault Bañeras-Roux, Shashi Kumar, Driss Khalil +6
Automatic Speech Recognition (ASR) is traditionally evaluated using Word Error Rate (WER), a metric that is insensitive to meaning. Embedding-based semantic metrics are better corr…
A Comprehensive Analysis of Tokenization and Self-Supervised Learning in End-to-End Automatic Speech Recognition applied on French Language
Thibault Bañeras-Roux, Mickael Rouvier, Jane Wottawa +1
The performance of end-to-end automatic speech recognition (ASR) systems enables their increasing integration into numerous applications. While there are various benefits to such s…
A Paradigm for Interpreting Metrics and Identifying Critical Errors in Automatic Speech Recognition
Thibault Bañeras-Roux, Mickael Rouvier, Jane Wottawa +1
The most commonly used metrics for evaluating automatic speech transcriptions, namely Word Error Rate (WER) and Character Error Rate (CER), have been heavily criticized for their p…
HATS: An Open data set Integrating Human Perception Applied to the Evaluation of Automatic Speech Recognition Metrics
Thibault Bañeras Roux, Jane Wottawa, Mickael Rouvier +2
Conventionally, Automatic Speech Recognition (ASR) systems are evaluated on their ability to correctly recognize each word contained in a speech signal. In this context, the word e…
Qualitative Evaluation of Language Model Rescoring in Automatic Speech Recognition
Thibault Bañeras-Roux, Mickaël Rouvier, Jane Wottawa +1
Evaluating automatic speech recognition (ASR) systems is a classical but difficult and still open problem, which often boils down to focusing only on the word error rate (WER). How…
Closing the Speech-Text Gap with Limited Audio for Effective Domain Adaptation in LLM-Based ASR
Thibault Bañeras-Roux, Sergio Burdisso, Esaú Villatoro-Tello +9
Conventional end-to-end automatic speech recognition (ASR) systems rely on paired speech-text data for domain adaptation. Recent LLM-based ASR architectures connect a speech encode…
Abstract Compression of Conversational Audio Context for LLM-based ASR
Shashi Kumar, Esaú Villatoro-Tello, Sergio Burdisso +7
Standard LLM-based speech recognition systems process utterances in isolation, limiting their ability to leverage conversational context. Here we study whether multimodal context f…