221 citations · 296 across the 28 of their papers we have counts for
14 papers · 1 filter
DialFork: Going Beyond Factuality to Evaluate Procedural Hallucination in Conversational Agents
Sergio Burdisso, Esaú Villatoro-Tello, Dairazalia Sánchez-Cortés +4
Large language models (LLMs) deployed as conversational agents are increasingly expected to follow procedural logic. While factual hallucination has received extensive attention, a…
Steering Speech-Language Models: Training-Free Task Specialization via Contrastive Activation Addition
Séverin Baroudi, Yanis Labrak, Pierfrancesco Melucci +5
Activation steering has proven effective for controlling the behavior of Large Language Models (LLMs) at inference time, but its application to SpeechLLMs remains new, and training…
From Generative to Discriminative: Lightweight Adaptation of SpeechLLMs for Multimodal Emotion Recognition
Hasindri Watawana, Sergio Burdisso, Esaú Villatoro-Tello +4
SpeechLLMs can recognise emotion by generating a label token, but this approach relies on the LLM's existing output representations. We compare this generative approach with a disc…
Generative vs. Encoder Large Language Models for ASR Evaluation: A Comparative Study
Thibault Bañeras-Roux, Shashi Kumar, Driss Khalil +6
Automatic Speech Recognition (ASR) is typically evaluated using Word Error Rate (WER), which poorly reflects semantic similarity. While embedding-based metrics correlate better wit…
When Synthetic Speech Is All You Have: Better Call GRPO
Shashi Kumar, Yanis Labrak, Hasindri Watawana +5
LLM-based ASR adapted to regulated domains such as banking is bottlenecked by privacy: real speech is costly and legally constrained to collect, making synthetic text-to-speech (TT…
How to Leverage Synthetic Speech for LLM-Based ASR Systems?
Yanis Labrak, Dairazalia Sanchez-Cortes, Sergio Burdisso +9
In regulated domains such as banking and healthcare, where privacy constraints make real speech costly to collect and retain, synthetic speech from modern text-to-speech (TTS) is a…
Evaluation of Automatic Speech Recognition Using Generative Large Language Models
Thibault Bañeras-Roux, Shashi Kumar, Driss Khalil +6
Automatic Speech Recognition (ASR) is traditionally evaluated using Word Error Rate (WER), a metric that is insensitive to meaning. Embedding-based semantic metrics are better corr…
Closing the Speech-Text Gap with Limited Audio for Effective Domain Adaptation in LLM-Based ASR
Thibault Bañeras-Roux, Sergio Burdisso, Esaú Villatoro-Tello +9
Conventional end-to-end automatic speech recognition (ASR) systems rely on paired speech-text data for domain adaptation. Recent LLM-based ASR architectures connect a speech encode…
Generating Synthetic Doctor-Patient Conversations for Long-form Audio Summarization
Yanis Labrak, David Grünert, Séverin Baroudi +11
Long-context audio reasoning is underserved in both training data and evaluation. Existing benchmarks target short-context tasks, and the open-ended generation tasks most relevant…
Abstract Compression of Conversational Audio Context for LLM-based ASR
Shashi Kumar, Esaú Villatoro-Tello, Sergio Burdisso +7
Standard LLM-based speech recognition systems process utterances in isolation, limiting their ability to leverage conversational context. Here we study whether multimodal context f…
When Consistency Becomes Bias: Interviewer Effects in Semi-Structured Clinical Interviews
Hasindri Watawana, Sergio Burdisso, Diego A. Moreno-Galván +4
Automatic depression detection from doctor-patient conversations has gained momentum thanks to the availability of public corpora and advances in language modeling. However, interp…
DASHA: Diarization-Aware Speech-LLM ASR for Healthcare in Code-Switched Indic Doctor-Patient Conversations
Séverin Baroudi, Yanis Labrak, Shashi Kumar +7
Extracting patients' medical conditions from recorded primary-care conversations requires knowing who said what, which is difficult when speech is far-field, spontaneous, overlappi…
Reducing Prompt Sensitivity in LLM-based Speech Recognition Through Learnable Projection
Sergio Burdisso, Esaú Villatoro-Tello, Shashi Kumar +7
LLM-based automatic speech recognition (ASR), a well-established approach, connects speech foundation models to large language models (LLMs) through a speech-to-LLM projector, yiel…
Text-only adaptation in LLM-based ASR through text denoising
Andrés Carofilis, Sergio Burdisso, Esaú Villatoro-Tello +8
Adapting large language model (LLM)-based automatic speech recognition (ASR) systems to new domains using text-only data is a significant yet underexplored challenge. Standard fine…