16 papers
SpeechLLM Meets Federated Learning for End-to-End ASR: English and Italian Case Studies
Mohamed Nabih Ali, Daniele Falavigna, Alessio Brutti
Federated learning (FL) enables privacy-preserving training of automatic speech recognition (ASR) systems across distributed data sources, yet its application to large-scale speech…
MEUSLI: a Multilingual Projector for LLM-based ASR and Beyond
Lorenzo Concina, Seraphina Fong, Marco Matassoni +1
Lightweight projectors are an established way to connect pre-trained speech encoders with large language models (LLMs), mapping acoustic features into token-level embeddings for ta…
LuxSQA: Ask Me in Luxembourgish with TTS-Augmented Spoken Question Answering
Nina Hosseini-Kivanani, Marco Matassoni, Alessio Brutti
Spoken Question Answering (SQA) remains largely focused on high-resource languages and carefully recorded speech, limiting the reach of speech-LLM methods in low-resource settings.…
EgoAdapt: Enhancing Robustness in Egocentric Interactive Speaker Detection Under Missing Modalities
Xinyuan Qian, Xinjia Zhu, Alessio Brutti +1
TTM (Talking to Me) task is a pivotal component in understanding human social interactions, aiming to determine who is engaged in conversation with the camera-wearer. Traditional m…
Distillation-based Layer Dropping (DLD): Effective End-to-end Framework for Dynamic Speech Networks
Abdul Hannan, Daniele Falavigna, Shah Nawaz +3
Edge devices operate in constrained and varying resource settings, requiring dynamic architectures that can adapt to limitations of the available resources. To meet such demands, l…
MLMA: Towards Multilingual ASR With Mamba-based Architectures
Mohamed Nabih Ali, Daniele Falavigna, Alessio Brutti
Multilingual automatic speech recognition (ASR) remains a challenging task, especially when balancing performance across high- and low-resource languages. Recent advances in sequen…