257 citations · 441 across the 24 of their papers we have counts for
27 papers
Large Language Continuous Diffusion Models
Zhihan Yang, Wei Guo, Jean-Marie Lemercier +14
Despite the success of discrete diffusion language models (dLMs) for fast parallel decoding, their non-smooth, high-dimensional space hinders trajectory steering for reasoning and…
EffVOC: Low-Delay Efficient Speech Waveform Reconstruction from Spectral Representations Without Phase
Renzheng Shi, Simon Welker, Timo Gerkmann +1
The Griffin-Lim algorithm has been a seminal contribution for phase reconstruction from amplitude spectrograms, however, requiring (infinitely) high algorithmic delay. Its low-dela…
Real-Time Streamable Generative Speech Restoration with Flow Matching
Simon Welker, Bunlong Lay, Maris Hillemann +2
Diffusion-based generative models have greatly impacted the speech processing field in recent years, exhibiting high speech naturalness and spawning a new research direction. Their…
Diffusion Buffer for Online Generative Speech Enhancement
Bunlong Lay, Rostislav Makarov, Simon Welker +2
Online Speech Enhancement was mainly reserved for predictive models. A key advantage of these models is that for an incoming signal frame from a stream of data, the model is called…
Real-Time Streaming Mel Vocoding with Generative Flow Matching
Simon Welker, Tal Peer, Timo Gerkmann
The task of Mel vocoding, i.e., the inversion of a Mel magnitude spectrogram to an audio waveform, is still a key component in many text-to-speech (TTS) systems today. Based on gen…
Position-Blind Ptychography: Viability of image reconstruction via data-driven variational inference
Simon Welker, Lorenz Kuger, Tim Roith +4
In this work, we present and investigate the novel blind inverse problem of position-blind ptychography, i.e., ptychographic phase retrieval without any knowledge of scan positions…
FlowDec: A flow-based full-band general audio codec with high perceptual quality
Simon Welker, Matthew Le, Ricky T. Q. Chen +4
We propose FlowDec, a neural full-band audio codec for general audio sampled at 48 kHz that combines non-adversarial codec training with a stochastic postfilter based on a novel co…
Non-intrusive Speech Quality Assessment with Diffusion Models Trained on Clean Speech
Danilo de Oliveira, Julius Richter, Jean-Marie Lemercier +2
Diffusion models have found great success in generating high quality, natural samples of speech, but their potential for density estimation for speech has so far remained largely u…
Unsupervised Blind Joint Dereverberation and Room Acoustics Estimation with Diffusion Models
Jean-Marie Lemercier, Eloi Moliner, Simon Welker +2
This paper presents an unsupervised method for single-channel blind dereverberation and room impulse response (RIR) estimation, called BUDDy. The algorithm is rooted in Bayesian po…
EARS: An Anechoic Fullband Speech Dataset Benchmarked for Speech Enhancement and Dereverberation
Julius Richter, Yi-Chiao Wu, Steven Krenn +5
We release the EARS (Expressive Anechoic Recordings of Speech) dataset, a high-quality speech dataset comprising 107 speakers from diverse backgrounds, totaling in 100 hours of cle…
The PESQetarian: On the Relevance of Goodhart's Law for Speech Enhancement
Danilo de Oliveira, Simon Welker, Julius Richter +1
To obtain improved speech enhancement models, researchers often focus on increasing performance according to specific instrumental metrics. However, when the same metric is used in…
BUDDy: Single-Channel Blind Unsupervised Dereverberation with Diffusion Models
Eloi Moliner, Jean-Marie Lemercier, Simon Welker +2
In this paper, we present an unsupervised single-channel method for joint blind dereverberation and room impulse response estimation, based on posterior sampling with diffusion mod…
Diffusion Models for Audio Restoration
Jean-Marie Lemercier, Julius Richter, Simon Welker +3
With the development of audio playback devices and fast data transmission, the demand for high sound quality is rising for both entertainment and communications. In this quest for…
Live Iterative Ptychography with projection-based algorithms
Simon Welker, Tal Peer, Henry N. Chapman +1
In this work, we demonstrate that the ptychographic phase problem can be solved in a live fashion during scanning, while data is still being collected. We propose a generally appli…
A Flexible Online Framework for Projection-Based STFT Phase Retrieval
Tal Peer, Simon Welker, Johannes Kolhoff +1
Several recent contributions in the field of iterative STFT phase retrieval have demonstrated that the performance of the classical Griffin-Lim method can be considerably improved…
EMOCONV-DIFF: Diffusion-based Speech Emotion Conversion for Non-parallel and In-the-wild Data
Navin Raj Prabhu, Bunlong Lay, Simon Welker +2
Speech emotion conversion is the task of converting the expressed emotion of a spoken utterance to a target emotion while preserving the lexical content and speaker identity. While…
Diffusion Posterior Sampling for Informed Single-Channel Dereverberation
Jean-Marie Lemercier, Simon Welker, Timo Gerkmann
We present in this paper an informed single-channel dereverberation method based on conditional generation with diffusion models. With knowledge of the room impulse response, the a…
Speech Signal Improvement Using Causal Generative Diffusion Models
Julius Richter, Simon Welker, Jean-Marie Lemercier +3
In this paper, we present a causal speech signal improvement system that is designed to handle different types of distortions. The method is based on a generative diffusion model w…
Reducing the Prior Mismatch of Stochastic Differential Equations for Diffusion-based Speech Enhancement
Bunlong Lay, Simon Welker, Julius Richter +1
Recently, score-based generative models have been successfully employed for the task of speech enhancement. A stochastic differential equation is used to model the iterative forwar…
StoRM: A Diffusion-based Stochastic Regeneration Model for Speech Enhancement and Dereverberation
Jean-Marie Lemercier, Julius Richter, Simon Welker +1
Diffusion models have shown a great ability at bridging the performance gap between predictive and generative approaches for speech enhancement. We have shown that they may even ou…
DriftRec: Adapting diffusion models to blind JPEG restoration
Simon Welker, Henry N. Chapman, Timo Gerkmann
In this work, we utilize the high-fidelity generation abilities of diffusion models to solve blind JPEG restoration at high compression levels. We propose an elegant modification o…
DiffPhase: Generative Diffusion-based STFT Phase Retrieval
Tal Peer, Simon Welker, Timo Gerkmann
Diffusion probabilistic models have been recently used in a variety of tasks, including speech enhancement and synthesis. As a generative approach, diffusion models have been shown…
Analysing Diffusion-based Generative Approaches versus Discriminative Approaches for Speech Restoration
Jean-Marie Lemercier, Julius Richter, Simon Welker +1
Diffusion-based generative models have had a high impact on the computer vision and speech processing communities these past years. Besides data generation tasks, they have also be…
Speech Enhancement and Dereverberation with Diffusion-based Generative Models
Julius Richter, Simon Welker, Jean-Marie Lemercier +2
In this work, we build upon our previous publication and use diffusion-based generative models for speech enhancement. We present a detailed overview of the diffusion process that…