19 papers
Exploring Representation Autoencoders for Text-to-Audio Generation
Bernardo V. Miranda, Hugo Malard, Gaël Richard +2
Representation autoencoders (RAE), created by pairing a frozen pretrained self-supervised learning (SSL) encoder with a trained decoder, have recently demonstrated state-of-the-art…
The Internet Archive Music Dataset
Paraskevas Stamatiadis, Bernardo V Miranda, Clémentine Berger +3
We introduce the Internet Archive Music Dataset (IAMD), a large-scale collection of captioned music segments derived from the Internet Archive. To the best of our knowledge, IAMD c…
Neural Multichannel Distant Speaker Diarization with Heavy-tailed Source Separation Model
Sicheng Mao, Baihan Li, Mathieu Fontaine +2
Distant speaker diarization remains challenging due to difficult acoustic environments, varying numbers of speakers and overlapping speech. Model-driven methods are proposed to exp…
Neural Multichannel Distant Speaker Diarization and Source Separation with Beta Speaker Activity Prior
Sicheng Mao, Mathieu Fontaine, Anthony Larcher +1
Distant speaker diarization remains challenging due to adverse acoustic conditions, varying numbers of speakers and overlapping speech. While data-driven approaches have shown stro…
Taylor-SWFT: fast discrete Statistical Wave Field Theory using Taylor expansion for late reverberation Work under review
Marius Rodrigues, Louis Lalay, Roland Badeau +2
Dynamic room acoustic simulation aims to render the acoustic effects of an environment in real time while accounting for potentially moving sources and receivers. In this context,…
Contrastive Knowledge Distillation for Embedding Refinement in Personalized Speech Enhancement
Thomas Serre, Mathieu Fontaine, Éric Benhaim +1
Personalized speech enhancement (PSE) has shown convincing results when it comes to extracting a known target voice among interfering ones. The corresponding systems usually incorp…
O-EENC-SD: Efficient Online End-to-End Neural Clustering for Speaker Diarization
Elio Gruttadauria, Mathieu Fontaine, Jonathan Le Roux +1
We introduce O-EENC-SD: an end-to-end online speaker diarization system based on EEND-EDA, featuring a novel RNN-based stitching mechanism for online prediction. In particular, we…
GLA-Grad++: An Improved Griffin-Lim Guided Diffusion Model for Speech Synthesis
Teysir Baoueb, Xiaoyu Bie, Mathieu Fontaine +1
Recent advances in diffusion models have positioned them as powerful generative frameworks for speech synthesis, demonstrating substantial improvements in audio quality and stabili…
The silence of the weights: a structural pruning strategy for attention-based audio signal architectures with second order metrics
Andrea Diecidue, Carlo Alberto Barbano, Piero Fraternali +2
Transformer-based models have become the state of the art across multiple domains, from natural language processing to machine listening, thanks to the attention mechanisms. Howeve…
U-DREAM: Unsupervised Dereverberation guided by a Reverberation Model
Louis Bahrman, Marius Rodrigues, Mathieu Fontaine +1
This paper explores the outcome of training state-of-the-art dereverberation models with supervision settings ranging from weakly-supervised to virtually unsupervised, relying sole…
Multiple Choice Learning of Low-Rank Adapters for Language Modeling
Victor Letzelter, Hugo Malard, Mathieu Fontaine +4
We propose LoRA-MCL, a training scheme that extends next-token prediction in language models with a method designed to decode diverse, plausible sentence continuations at inference…
A Hybrid Model for Weakly-Supervised Speech Dereverberation
Louis Bahrman, Mathieu Fontaine, Gael Richard
This paper introduces a new training strategy to improve speech dereverberation systems using minimal acoustic information and reverberant (wet) speech. Most existing algorithms re…
Speech dereverberation constrained on room impulse response characteristics
Louis Bahrman, Mathieu Fontaine, Jonathan Le Roux +1
Single-channel speech dereverberation aims at extracting a dry speech signal from a recording affected by the acoustic reflections in a room. However, most current deep learning-ba…
Winner-takes-all learners are geometry-aware conditional density estimators
Victor Letzelter, David Perera, Cédric Rommel +4
Winner-takes-all training is a simple learning paradigm, which handles ambiguous tasks by predicting a set of plausible hypotheses. Recently, a connection was established between W…
A lightweight dual-stage framework for personalized speech enhancement based on DeepFilterNet2
Thomas Serre, Mathieu Fontaine, Éric Benhaim +2
Isolating the desired speaker's voice amidst multiplespeakers in a noisy acoustic context is a challenging task. Per-sonalized speech enhancement (PSE) endeavours to achievethis by…
GLA-Grad: A Griffin-Lim Extended Waveform Generation Diffusion Model
Haocheng Liu, Teysir Baoueb, Mathieu Fontaine +2
Diffusion models are receiving a growing interest for a variety of signal generation tasks such as speech or music synthesis. WaveGrad, for example, is a successful diffusion model…
SpecDiff-GAN: A Spectrally-Shaped Noise Diffusion GAN for Speech and Music Synthesis
Teysir Baoueb, Haocheng Liu, Mathieu Fontaine +2
Generative adversarial network (GAN) models can synthesize highquality audio signals while ensuring fast sample generation. However, they are difficult to train and are prone to se…
Online speaker diarization of meetings guided by speech separation
Elio Gruttadauria, Mathieu Fontaine, Slim Essid
Overlapped speech is notoriously problematic for speaker diarization systems. Consequently, the use of speech separation has recently been proposed to improve their performance. Al…
Resilient Multiple Choice Learning: A learned scoring scheme with application to audio scene analysis
Victor Letzelter, Mathieu Fontaine, Mickaël Chen +3
We introduce Resilient Multiple Choice Learning (rMCL), an extension of the MCL approach for conditional distribution estimation in regression settings where multiple targets may b…