514 citations · 1.4k across the 79 of their papers we have counts for
98 papers
Listen-to-Reason: Listen with Experts, Retrieve over a Graph, Reason with LLMs
Pooneh Mousavi, Mirco Ravanelli, Cem Subakan
Large audio-language models (LALMs) fuse an audio encoder into a large language model (LLM) through multi-stage training. This coupling means that a new domain or a stronger LLM re…
Steering Speech-Language Models: Training-Free Task Specialization via Contrastive Activation Addition
Séverin Baroudi, Yanis Labrak, Pierfrancesco Melucci +5
Activation steering has proven effective for controlling the behavior of Large Language Models (LLMs) at inference time, but its application to SpeechLLMs remains new, and training…
AnchorPrompt: Self-Distilled Soft Prompts for Robust Audio-Language Models
Pooneh Mousavi, Amir Ivry, Mirco Ravanelli +1
Large audio-language models (LALMs) are sensitive to input perturbations, such as noise, waveform corruption, and adversarial injections. We propose AnchorPrompt, an efficient adap…
Random Recursive Models
Jama Hussein Mohamud, Mirco Ravanelli
Recursive models create computational depth through parameter reuse, offering a parameter-efficient alternative to increasing model size. However, most recursive models repeatedly…
GLaS-JEPA: Gaussian-Regularized Speech SSL without Engineered Prediction Targets
Gaspard Botté, Séverin Baroudi, Samir Sadok +5
Speech self-supervised learning aims to learn general-purpose representations for downstream speech tasks. However, current approaches rely on complex, carefully designed predictio…
Time-Incremental Continued Pretraining of LLMs: Knowledge Updates Without Catastrophic Forgetting
Fırat Öncel, Salman Hussain Ali, Mirco Ravanelli +2
Large language models (LLMs) drift out of date the moment their pretraining ends, yet retraining from scratch is prohibitively expensive. Continued pretraining (CPT) is the natural…
ZipCodec: Ultra-Low-Frame-Rate Streaming Speech Coding
Luca Della Libera, Cem Subakan, Mirco Ravanelli
Neural audio codecs are a fundamental component of modern speech generation systems. While recent codecs achieve increasingly low bitrates, reducing frame rate remains challenging,…
Enhancing Audio Reasoning via Semantic Summary Prediction
Francesco Bonzi, Pooneh Mousavi, Cem Subakan +1
Large Audio Language Models (LALMs) perform well on complex question answering but often show a reasoning gap, where explicit Chain-of-Thought (CoT) reduces accuracy compared to di…
Controllable and Content-Based Recommendations
Fırat Öncel, Jihoon Jeong, Emiliano Penaloza +3
Traditional recommendation systems rely on latent (dense) representations, making them difficult to interpret and control. We propose the Controllable and Content-Based Recommendat…
HybridCodec: Modeling Discrete and Continuous Representations for Efficient Speech Language Models
Artem Ploujnikov, Francesco Verdini, Samir Sadok +1
Discrete audio representations have become increasingly popular for building multimodal text-audio systems and integrating audio capabilities into Large Language Models (LLMs). How…
MambAdapter: Lightweight Mamba-Based Adapters for Parameter-Efficient Transfer Learning in Speech and Audio
Salman Hussain Ali, Umberto Cappellazzo, Mirco Ravanelli
Fine-tuning Transformer-based foundation models has become the dominant strategy for domain adaptation in audio and speech processing. To reduce the computational and memory costs…
Adaptive Order Policies for Masked Diffusion
Jama Hussein Mohamud, Mohsin Hasan, Mirco Ravanelli +1
Masked diffusion models have seen great success in capturing data distributions over discrete sequences in domains such as text and proteins. These models generate data by iterativ…
Exploring Token-Space Manipulation in Latent Audio Tokenizers
Francesco Paissan, Luca Della Libera, Mirco Ravanelli +1
Neural audio codecs provide compact discrete representations for speech generation and manipulation. However, most codecs organize tokens as frame-level sequences, making it diffic…
Self-Routing: Parameter-Free Expert Routing from Hidden States
Jama Hussein Mohamud, Drew Wagner, Mirco Ravanelli
Mixture-of-Experts (MoE) layers increase model capacity by activating only a small subset of experts per token, and typically rely on a learned router to map hidden states to exper…
Listen First, Then Answer: Timestamp-Grounded Speech Reasoning
Jihoon Jeong, Pooneh Mousavi, Mirco Ravanelli +1
Large audio-language models (LALMs) can generate reasoning chains for their predictions, but it remains unclear whether these reasoning chains remain grounded in the input audio. I…
LL-SDR: Low-Latency Speech enhancement through Discrete Representations
Jingyi Li, Luca Della Libera, Mirco Ravanelli +2
Many speech enhancement (SE) methods rely on continuous representations. Recently, discrete audio tokens have been explored to enable autoregressive generation for SE. However, it…
WavSLM: Single-Stream Speech Language Modeling via WavLM Distillation
Luca Della Libera, Cem Subakan, Mirco Ravanelli
Large language models show that simple autoregressive training can yield scalable and coherent generation, but extending this paradigm to speech remains challenging due to the enta…
Beyond Fixed Frames: Dynamic Character-Aligned Speech Tokenization
Luca Della Libera, Cem Subakan, Mirco Ravanelli
Neural audio codecs are at the core of modern conversational speech technologies, converting continuous speech into sequences of discrete tokens that can be processed by LLMs. Howe…
Toward Faithful Explanations in Acoustic Anomaly Detection
Maab Elrashid, Anthony Deschênes, Cem Subakan +3
Interpretability is essential for user trust in real-world anomaly detection applications. However, deep learning models, despite their strong performance, often lack transparency.…
Comparison of Speech Tasks in Human Expert and Machine Detection of Parkinson's Disease
Peter Plantinga, Roozbeh Sattari, Karine Marcotte +8
The speech of people with Parkinson's Disease (PD) has been shown to hold important clues about the presence and progression of the disease. We investigate the factors based on whi…
Virtual Consistency for Audio Editing
Matthieu Cervera, Francesco Paissan, Mirco Ravanelli +1
Free-form, text-based audio editing remains a persistent challenge, despite progress in inversion-based neural methods. Current approaches rely on slow inversion procedures, limiti…
FocalCodec-Stream: Streaming Low-Bitrate Speech Coding via Causal Distillation
Luca Della Libera, Cem Subakan, Mirco Ravanelli
Neural audio codecs are a fundamental component of modern generative audio pipelines. Although recent codecs achieve strong low-bitrate reconstruction and provide powerful represen…
Investigating Faithfulness in Large Audio Language Models
Pooneh Mousavi, Lovenya Jain, Mirco Ravanelli +1
Large Audio Language Models (LALMs) integrate audio encoders with pretrained Large Language Models to perform complex multimodal reasoning tasks. While these models can generate Ch…
Audio Prototypical Network For Controllable Music Recommendation
Fırat Öncel, Emiliano Penaloza, Haolun Wu +4
Traditional recommendation systems represent user preferences in dense representations obtained through black-box encoder models. While these models often provide strong recommenda…