1 citations · 1 across the 24 of their papers we have counts for
32 papers
Perceptual Quality Loss or Loss of Perceptual Quality?
Danilo de Oliveira, Tal Peer, Maurício do V. M. da Costa +1
Contemporary deep speech enhancement (SE) models are often trained with specific auxiliary terms in the loss function as a way to improve their performance in terms of perceptual m…
Livin' on a Prior: Likelihood Score Approximation for Inverse Problems
Rostislav Makarov, Tal Peer, Danilo de Oliveira +1
Generative models have found great success as data-driven methods of solving inverse problems. Two popular approaches work either by combining a pretrained generative prior with a…
Directivity-Conditioned Low-Latency Neural Filtering for Speech Enhancement in Hearing Aids
Lennart Uphaus, André Merboldt, Markus Hofbauer +1
Latest advances in neural directional filtering show exceptional results in adapting the direction and shape of directivity patterns during the inference phase. However, in the exi…
Test-time adaptation for speech enhancement with an autoregressive speech prior
Sofiene Kammoun, Simon Leglaive, Xavier Alameda-Pineda +1
Test-time adaptation (TTA) offers a promising direction for improving speech enhancement models under mismatched acoustic conditions, without requiring access to labeled target dat…
EffVOC: Low-Delay Efficient Speech Waveform Reconstruction from Spectral Representations Without Phase
Renzheng Shi, Simon Welker, Timo Gerkmann +1
The Griffin-Lim algorithm has been a seminal contribution for phase reconstruction from amplitude spectrograms, however, requiring (infinitely) high algorithmic delay. Its low-dela…
Repurposing a Speech Classifier for Guided Diffusion-Based Speech Generation
Rostislav Makarov, Timo Gerkmann
Classifier guidance is a way to control diffusion generation by using a noise-conditioned classifier to steer the sampling process toward a target class. One drawback of classifier…
Your U-Net Dereverberation Model is Secretly an RIR Encoder
Sina Khanagha, Timo Gerkmann
In this work, we analyze the ability of NCSN++ U-Net based audio dereverberation models to capture global room characteristics in their intermediate representations. Through an emp…
Too Good to Be True: A Study on Modern Automatic Speech Recognition for the Evaluation of Speech Enhancement
Danilo de Oliveira, Tal Peer, Timo Gerkmann
Speech enhancement (SE) systems are typically evaluated using a variety of instrumental metrics. The use of automatic speech recognition (ASR) systems to evaluate SE performance is…
A Fast Solver for Interpolating Stochastic Differential Equation Diffusion Models for Speech Restoration
Bunlong Lay, Timo Gerkmann
Diffusion Probabilistic Models (DPMs) are a well-established class of diffusion models for unconditional image generation, while SGMSE+ is a well-established conditional diffusion…
Bone-conduction Guided Multimodal Speech Enhancement with Conditional Diffusion Models
Sina Khanagha, Bunlong Lay, Timo Gerkmann
Single-channel speech enhancement models face significant performance degradation in extremely noisy environments. While prior work has shown that complementary bone-conducted spee…
Real-Time Streamable Generative Speech Restoration with Flow Matching
Simon Welker, Bunlong Lay, Maris Hillemann +2
Diffusion-based generative models have greatly impacted the speech processing field in recent years, exhibiting high speech naturalness and spawning a new research direction. Their…
Are These Even Words? Quantifying the Gibberishness of Generative Speech Models
Danilo de Oliveira, Tal Peer, Jonas Rochdi +1
Significant research efforts are currently being dedicated to non-intrusive quality and intelligibility assessment, especially given how it enables curation of large scale datasets…
Diffusion Buffer for Online Generative Speech Enhancement
Bunlong Lay, Rostislav Makarov, Simon Welker +2
Online Speech Enhancement was mainly reserved for predictive models. A key advantage of these models is that for an incoming signal frame from a stream of data, the model is called…
Real-Time Streaming Mel Vocoding with Generative Flow Matching
Simon Welker, Tal Peer, Timo Gerkmann
The task of Mel vocoding, i.e., the inversion of a Mel magnitude spectrogram to an audio waveform, is still a key component in many text-to-speech (TTS) systems today. Based on gen…
Position-Blind Ptychography: Viability of image reconstruction via data-driven variational inference
Simon Welker, Lorenz Kuger, Tim Roith +4
In this work, we present and investigate the novel blind inverse problem of position-blind ptychography, i.e., ptychographic phase retrieval without any knowledge of scan positions…
Are Modern Speech Enhancement Systems Vulnerable to Adversarial Attacks?
Rostislav Makarov, Lea Schönherr, Timo Gerkmann
Machine learning approaches for speech enhancement are becoming increasingly expressive, enabling ever more powerful modifications of input signals. In this paper, we demonstrate t…
Enhancing In-the-Wild Speech Emotion Conversion with Resynthesis-based Duration Modeling
Navin Raj Prabhu, Danilo de Oliveira, Nale Lehmann-Willenbrock +1
Speech Emotion Conversion aims to modify the emotion expressed in input speech while preserving lexical content and speaker identity. Recently, generative modeling approaches have…
Diffusion Buffer: Online Diffusion-based Speech Enhancement with Sub-Second Latency
Bunlong Lay, Rostislav Makarov, Timo Gerkmann
Diffusion models are a class of generative models that have been recently used for speech enhancement with remarkable success but are computationally expensive at inference time. T…
ReverbFX: A Dataset of Room Impulse Responses Derived from Reverb Effect Plugins for Singing Voice Dereverberation
Julius Richter, Till Svajda, Timo Gerkmann
We present ReverbFX, a new room impulse response (RIR) dataset designed for singing voice dereverberation research. Unlike existing datasets based on real recorded RIRs, ReverbFX f…
LipDiffuser: Lip-to-Speech Generation with Conditional Diffusion Models
Julius Richter, Danilo de Oliveira, Tal Peer +1
We present LipDiffuser, a conditional diffusion model for lip-to-speech generation synthesizing natural and intelligible speech directly from silent video recordings. Our approach…
Do We Need EMA for Diffusion-Based Speech Enhancement? Toward a Magnitude-Preserving Network Architecture
Julius Richter, Danilo de Oliveira, Timo Gerkmann
We study diffusion-based speech enhancement using a Schrodinger bridge formulation and extend the EDM2 framework to this setting. We employ time-dependent preconditioning of networ…
FlowDec: A flow-based full-band general audio codec with high perceptual quality
Simon Welker, Matthew Le, Ricky T. Q. Chen +4
We propose FlowDec, a neural full-band audio codec for general audio sampled at 48 kHz that combines non-adversarial codec training with a stochastic postfilter based on a novel co…
HRTF Estimation using a Score-based Prior
Etienne Thuillier, Jean-Marie Lemercier, Eloi Moliner +2
We present a head-related transfer function (HRTF) estimation method which relies on a data-driven prior given by a score-based diffusion model. The HRTF is estimated in reverberan…
Non-intrusive Speech Quality Assessment with Diffusion Models Trained on Clean Speech
Danilo de Oliveira, Julius Richter, Jean-Marie Lemercier +2
Diffusion models have found great success in generating high quality, natural samples of speech, but their potential for density estimation for speech has so far remained largely u…