255 citations · 1.2k across the 38 of their papers we have counts for
27 papers · 1 filter
Recomposer: Event-roll-guided generative audio editing
Daniel P. W. Ellis, Eduardo Fonseca, Ron J. Weiss +7
Editing complex real-world sound scenes is difficult because individual sound sources overlap in time. Generative models can fill-in missing or corrupted details based on their str…
Towards Sub-millisecond Latency Real-Time Speech Enhancement Models on Hearables
Artem Dementyev, Chandan K. A. Reddy, Scott Wisdom +3
Low latency models are critical for real-time speech enhancement applications, such as hearing aids and hearables. However, the sub-millisecond latency space for resource-constrain…
Objective and subjective evaluation of speech enhancement methods in the UDASE task of the 7th CHiME challenge
Simon Leglaive, Matthieu Fraticelli, Hend ElGhazaly +7
Supervised models for speech enhancement are trained using artificially generated mixtures of clean speech and noise signals. However, the synthetic training conditions may not acc…
TokenSplit: Using Discrete Speech Representations for Direct, Refined, and Transcript-Conditioned Speech Separation and Recognition
Hakan Erdogan, Scott Wisdom, Xuankai Chang +4
We present TokenSplit, a speech separation model that acts on discrete token sequences. The model is trained on multiple tasks simultaneously: separate and transcribe each speech s…
The CHiME-7 UDASE task: Unsupervised domain adaptation for conversational speech enhancement
Simon Leglaive, Léonie Borne, Efthymios Tzinis +6
Supervised speech enhancement models are trained using artificially generated mixtures of clean speech and noise signals, which may not match real-world recording conditions at tes…
AudioSlots: A slot-centric generative model for audio separation
Pradyumna Reddy, Scott Wisdom, Klaus Greff +2
In a range of recent works, object-centric architectures have been shown to be suitable for unsupervised scene decomposition in the vision domain. Inspired by these methods we pres…
Unsupervised Multi-channel Separation and Adaptation
Cong Han, Kevin Wilson, Scott Wisdom +1
A key challenge in machine learning is to generalize from training data to an application domain of interest. This work generalizes the recently-proposed mixture invariant training…
AudioScopeV2: Audio-Visual Attention Architectures for Calibrated Open-Domain On-Screen Sound Separation
Efthymios Tzinis, Scott Wisdom, Tal Remez +1
We introduce AudioScopeV2, a state-of-the-art universal audio-visual on-screen sound separation system which is capable of learning to separate sounds and associate them with on-sc…
Distance-Based Sound Separation
Katharine Patterson, Kevin Wilson, Scott Wisdom +1
We propose the novel task of distance-based sound separation, where sounds are separated based only on their distance from a single microphone. In the context of assisted listening…
Adapting Speech Separation to Real-World Meetings Using Mixture Invariant Training
Aswin Sivaraman, Scott Wisdom, Hakan Erdogan +1
The recently-proposed mixture invariant training (MixIT) is an unsupervised method for training single-channel sound separation models in the sense that it does not require ground-…
Improving On-Screen Sound Separation for Open-Domain Videos with Audio-Visual Self-Attention
Efthymios Tzinis, Scott Wisdom, Tal Remez +1
We introduce a state-of-the-art audio-visual on-screen sound separation system which is capable of learning to separate sounds and associate them with on-screen objects by looking…
End-to-End Diarization for Variable Number of Speakers with Local-Global Networks and Discriminative Speaker Embeddings
Soumi Maiti, Hakan Erdogan, Kevin Wilson +3
We present an end-to-end deep network model that performs meeting diarization from single-channel audio recordings. End-to-end diarization models have the advantage of handling spe…
Self-Supervised Learning from Automatically Separated Sound Scenes
Eduardo Fonseca, Aren Jansen, Daniel P. W. Ellis +7
Real-world sound scenes consist of time-varying collections of sound sources, each generating characteristic sound events that are mixed together in audio recordings. The associati…
What's All the FUSS About Free Universal Sound Separation Data?
Scott Wisdom, Hakan Erdogan, Daniel Ellis +6
We introduce the Free Universal Sound Separation (FUSS) dataset, a new corpus for experiments in separating mixtures of an unknown number of sounds from an open domain of sound typ…
Sound Event Detection and Separation: a Benchmark on Desed Synthetic Soundscapes
Nicolas Turpault, Romain Serizel, Scott Wisdom +5
We propose a benchmark of state-of-the-art sound event detection systems (SED). We designed synthetic evaluation sets to focus on specific sound event detection challenges. We anal…
Into the Wild with AudioScope: Unsupervised Audio-Visual Separation of On-Screen Sounds
Efthymios Tzinis, Scott Wisdom, Aren Jansen +4
Recent progress in deep learning has enabled many advances in sound separation and visual scene understanding. However, extracting sound sources which are apparent in natural video…
Improving Sound Event Detection In Domestic Environments Using Sound Separation
Nicolas Turpault, Scott Wisdom, Hakan Erdogan +5
Performing sound event detection on real-world recordings often implies dealing with overlapping target sound events and non-target sounds, also referred to as interference or nois…
Improving Universal Sound Separation Using Sound Classification
Efthymios Tzinis, Scott Wisdom, John R. Hershey +2
Deep learning approaches have recently achieved impressive performance on both audio source separation and sound classification. Most audio source separation approaches focus only…
Sequential Multi-Frame Neural Beamforming for Speech Separation and Enhancement
Zhong-Qiu Wang, Hakan Erdogan, Scott Wisdom +5
This work introduces sequential neural beamforming, which alternates between neural network based spectral separation and beamforming based spatial separation. Our neural networks…
Universal Sound Separation
Ilya Kavalerov, Scott Wisdom, Hakan Erdogan +4
Recent deep learning approaches have achieved impressive performance on speech enhancement and separation tasks. However, these approaches have not been investigated for separating…
Differentiable Consistency Constraints for Improved Deep Speech Enhancement
Scott Wisdom, John R. Hershey, Kevin Wilson +4
In recent years, deep networks have led to dramatic improvements in speech enhancement by framing it as a data-driven pattern recognition problem. In many modern enhancement system…
Exploring Tradeoffs in Models for Low-latency Speech Enhancement
Kevin Wilson, Michael Chinen, Jeremy Thorpe +5
We explore a variety of neural networks configurations for one- and two-channel spectrogram-mask-based speech enhancement. Our best model improves on previous state-of-the-art perf…
SDR - half-baked or well done?
Jonathan Le Roux, Scott Wisdom, Hakan Erdogan +1
In speech enhancement and source separation, signal-to-noise ratio is a ubiquitous objective measure of denoising/separation quality. A decade ago, the BSS_eval toolkit was develop…
Phasebook and Friends: Leveraging Discrete Representations for Source Separation
Jonathan Le Roux, Gordon Wichern, Shinji Watanabe +2
Deep learning based speech enhancement and source separation systems have recently reached unprecedented levels of quality, to the point that performance is reaching a new ceiling.…