3 citations · 5 across the 11 of their papers we have counts for
13 papers
Listen-to-Reason: Listen with Experts, Retrieve over a Graph, Reason with LLMs
Pooneh Mousavi, Mirco Ravanelli, Cem Subakan
Large audio-language models (LALMs) fuse an audio encoder into a large language model (LLM) through multi-stage training. This coupling means that a new domain or a stronger LLM re…
AnchorPrompt: Self-Distilled Soft Prompts for Robust Audio-Language Models
Pooneh Mousavi, Amir Ivry, Mirco Ravanelli +1
Large audio-language models (LALMs) are sensitive to input perturbations, such as noise, waveform corruption, and adversarial injections. We propose AnchorPrompt, an efficient adap…
Enhancing Audio Reasoning via Semantic Summary Prediction
Francesco Bonzi, Pooneh Mousavi, Cem Subakan +1
Large Audio Language Models (LALMs) perform well on complex question answering but often show a reasoning gap, where explicit Chain-of-Thought (CoT) reduces accuracy compared to di…
Listen First, Then Answer: Timestamp-Grounded Speech Reasoning
Jihoon Jeong, Pooneh Mousavi, Mirco Ravanelli +1
Large audio-language models (LALMs) can generate reasoning chains for their predictions, but it remains unclear whether these reasoning chains remain grounded in the input audio. I…
Investigating Faithfulness in Large Audio Language Models
Pooneh Mousavi, Lovenya Jain, Mirco Ravanelli +1
Large Audio Language Models (LALMs) integrate audio encoders with pretrained Large Language Models to perform complex multimodal reasoning tasks. While these models can generate Ch…
Discrete Audio Tokens: More Than a Survey!
Pooneh Mousavi, Gallil Maimon, Adel Moumen +18
Discrete audio tokens are compact representations that aim to preserve perceptual quality, phonetic content, and speaker characteristics while enabling efficient storage and infere…
LiSTEN: Learning Soft Token Embeddings for Neural Audio LLMs
Pooneh Mousavi, Shubham Gupta, Cem Subakan +1
Foundation models based on large language models (LLMs) have shown great success in handling various tasks and modalities. However, adapting these models for general-purpose audio-…
ALAS: An Automatic Latent Alignment Score for Audio Language Models
Pooneh Mousavi, Yingzhi Wang, Mirco Ravanelli +1
Large Language Models (LLMs) are extended into Speech-LLMs, and the quality of the audio--text alignment they learn affects most downstream Spoken Language Understanding (SLU) beha…
What Are They Doing? Joint Audio-Speech Co-Reasoning
Yingzhi Wang, Pooneh Mousavi, Artem Ploujnikov +1
In audio and speech processing, tasks usually focus on either the audio or speech modality, even when both sounds and human speech are present in the same audio clip. Recent Audito…
Open-Source Conversational AI with SpeechBrain 1.0
Mirco Ravanelli, Titouan Parcollet, Adel Moumen +30
SpeechBrain is an open-source Conversational AI toolkit based on PyTorch, focused particularly on speech processing tasks such as speech recognition, speech enhancement, speaker re…
How Should We Extract Discrete Audio Tokens from Self-Supervised Models?
Pooneh Mousavi, Jarod Duret, Salah Zaiem +4
Discrete audio tokens have recently gained attention for their potential to bridge the gap between audio and language processing. Ideal audio tokens must preserve content, paraling…
DASB - Discrete Audio and Speech Benchmark
Pooneh Mousavi, Jarod Duret, Darius Petermann +5
Discrete audio tokens have recently gained considerable attention for their potential to bridge audio and language processing, enabling multimodal language models that can both gen…
CL-MASR: A Continual Learning Benchmark for Multilingual ASR
Luca Della Libera, Pooneh Mousavi, Salah Zaiem +2
Modern multilingual automatic speech recognition (ASR) systems like Whisper have made it possible to transcribe audio in multiple languages with a single model. However, current st…