5 papers
Trace Evolution Bench: Measuring Whether Agents Improve from Their Own Environment Traces
Fangfei Li, Chenyang Zhao, Long Wang +2
Self-evolving agents are usually scored by endpoint Gain: held-out success after an update minus success before. That difference can credit a frozen rescore as evolution. We introd…
Generative vs. Encoder Large Language Models for ASR Evaluation: A Comparative Study
Thibault Bañeras-Roux, Shashi Kumar, Driss Khalil +6
Automatic Speech Recognition (ASR) is typically evaluated using Word Error Rate (WER), which poorly reflects semantic similarity. While embedding-based metrics correlate better wit…
Evaluation of Automatic Speech Recognition Using Generative Large Language Models
Thibault Bañeras-Roux, Shashi Kumar, Driss Khalil +6
Automatic Speech Recognition (ASR) is traditionally evaluated using Word Error Rate (WER), a metric that is insensitive to meaning. Embedding-based semantic metrics are better corr…
Closing the Speech-Text Gap with Limited Audio for Effective Domain Adaptation in LLM-Based ASR
Thibault Bañeras-Roux, Sergio Burdisso, Esaú Villatoro-Tello +9
Conventional end-to-end automatic speech recognition (ASR) systems rely on paired speech-text data for domain adaptation. Recent LLM-based ASR architectures connect a speech encode…
CL-UZH submission to the NIST SRE 2024 Speaker Recognition Evaluation
Aref Farhadipour, Shiran Liu, Masoumeh Chapariniya +4
The CL-UZH team submitted one system each for the fixed and open conditions of the NIST SRE 2024 challenge. For the closed-set condition, results for the audio-only trials were ach…