514 citations · 550 across the 50 of their papers we have counts for
14 papers · 1 filter
HybridCodec: Modeling Discrete and Continuous Representations for Efficient Speech Language Models
Artem Ploujnikov, Francesco Verdini, Samir Sadok +1
Discrete audio representations have become increasingly popular for building multimodal text-audio systems and integrating audio capabilities into Large Language Models (LLMs). How…
Adaptive Order Policies for Masked Diffusion
Jama Hussein Mohamud, Mohsin Hasan, Mirco Ravanelli +1
Masked diffusion models have seen great success in capturing data distributions over discrete sequences in domains such as text and proteins. These models generate data by iterativ…
WavSLM: Single-Stream Speech Language Modeling via WavLM Distillation
Luca Della Libera, Cem Subakan, Mirco Ravanelli
Large language models show that simple autoregressive training can yield scalable and coherent generation, but extending this paradigm to speech remains challenging due to the enta…
Beyond Fixed Frames: Dynamic Character-Aligned Speech Tokenization
Luca Della Libera, Cem Subakan, Mirco Ravanelli
Neural audio codecs are at the core of modern conversational speech technologies, converting continuous speech into sequences of discrete tokens that can be processed by LLMs. Howe…
Investigating Faithfulness in Large Audio Language Models
Pooneh Mousavi, Lovenya Jain, Mirco Ravanelli +1
Large Audio Language Models (LALMs) integrate audio encoders with pretrained Large Language Models to perform complex multimodal reasoning tasks. While these models can generate Ch…
FocalCodec: Low-Bitrate Speech Coding via Focal Modulation Networks
Luca Della Libera, Francesco Paissan, Cem Subakan +1
Large language models have revolutionized natural language processing through self-supervised pretraining on massive datasets. Inspired by this success, researchers have explored a…