91 citations · 141 across the 16 of their papers we have counts for
17 papers
Recomposer: Event-roll-guided generative audio editing
Daniel P. W. Ellis, Eduardo Fonseca, Ron J. Weiss +7
Editing complex real-world sound scenes is difficult because individual sound sources overlap in time. Generative models can fill-in missing or corrupted details based on their str…
Live Music Models
Lyria Team, Antoine Caillon, Brian McWilliams +33
We introduce a new class of generative models for music called live music models that produce a continuous stream of music in real-time with synchronized user control. We release M…
SoundSculpt: Direction and Semantics Driven Ambisonic Target Sound Extraction
Tuochao Chen, D Shin, Hakan Erdogan +1
This paper introduces SoundSculpt, a neural network designed to extract target sound fields from ambisonic recordings. SoundSculpt employs an ambisonic-in-ambisonic-out architectur…
Binaural Angular Separation Network
Yang Yang, George Sung, Shao-Fu Shih +3
We propose a neural network model that can separate target speech sources from interfering sources at different angular regions using two microphones. The model is trained with sim…
TokenSplit: Using Discrete Speech Representations for Direct, Refined, and Transcript-Conditioned Speech Separation and Recognition
Hakan Erdogan, Scott Wisdom, Xuankai Chang +4
We present TokenSplit, a speech separation model that acts on discrete token sequences. The model is trained on multiple tasks simultaneously: separate and transcribe each speech s…
Guided Speech Enhancement Network
Yang Yang, Shao-Fu Shih, Hakan Erdogan +5
High quality speech capture has been widely studied for both voice communication and human computer interface reasons. To improve the capture performance, we can often find multi-m…
CycleGAN-Based Unpaired Speech Dereverberation
Hannah Muckenhirn, Aleksandr Safin, Hakan Erdogan +4
Typically, neural network-based speech dereverberation models are trained on paired data, composed of a dry utterance and its corresponding reverberant utterance. The main limitati…
Adapting Speech Separation to Real-World Meetings Using Mixture Invariant Training
Aswin Sivaraman, Scott Wisdom, Hakan Erdogan +1
The recently-proposed mixture invariant training (MixIT) is an unsupervised method for training single-channel sound separation models in the sense that it does not require ground-…
DF-Conformer: Integrated architecture of Conv-TasNet and Conformer using linear complexity self-attention for speech enhancement
Yuma Koizumi, Shigeki Karita, Scott Wisdom +4
Single-channel speech enhancement (SE) is an important task in speech processing. A widely used framework combines an analysis/synthesis filterbank with a mask prediction network,…
Sparse, Efficient, and Semantic Mixture Invariant Training: Taming In-the-Wild Unsupervised Sound Separation
Scott Wisdom, Aren Jansen, Ron J. Weiss +2
Supervised neural network training has led to significant progress on single-channel sound separation. This approach relies on ground truth isolated sources, which precludes scalin…
End-to-End Diarization for Variable Number of Speakers with Local-Global Networks and Discriminative Speaker Embeddings
Soumi Maiti, Hakan Erdogan, Kevin Wilson +3
We present an end-to-end deep network model that performs meeting diarization from single-channel audio recordings. End-to-end diarization models have the advantage of handling spe…
Continuous Speech Separation Using Speaker Inventory for Long Multi-talker Recording
Cong Han, Yi Luo, Chenda Li +8
Leveraging additional speaker information to facilitate speech separation has received increasing attention in recent years. Recent research includes extracting target speech by us…
Integration of speech separation, diarization, and recognition for multi-speaker meetings: System description, comparison, and analysis
Desh Raj, Pavel Denisov, Zhuo Chen +11
Multi-speaker speech recognition of unsegmented recordings has diverse applications such as meeting transcription and automatic subtitle generation. With technical advances in syst…
What's All the FUSS About Free Universal Sound Separation Data?
Scott Wisdom, Hakan Erdogan, Daniel Ellis +6
We introduce the Free Universal Sound Separation (FUSS) dataset, a new corpus for experiments in separating mixtures of an unknown number of sounds from an open domain of sound typ…
Sound Event Detection and Separation: a Benchmark on Desed Synthetic Soundscapes
Nicolas Turpault, Romain Serizel, Scott Wisdom +5
We propose a benchmark of state-of-the-art sound event detection systems (SED). We designed synthetic evaluation sets to focus on specific sound event detection challenges. We anal…
Improving Sound Event Detection In Domestic Environments Using Sound Separation
Nicolas Turpault, Scott Wisdom, Hakan Erdogan +5
Performing sound event detection on real-world recordings often implies dealing with overlapping target sound events and non-target sounds, also referred to as interference or nois…
Unsupervised Sound Separation Using Mixture Invariant Training
Scott Wisdom, Efthymios Tzinis, Hakan Erdogan +3
In recent years, rapid progress has been made on the problem of single-channel sound separation using supervised training of deep neural networks. In such supervised approaches, a…