104 citations · 218 across the 21 of their papers we have counts for
26 papers
Evaluating Inference-time Algorithms for Semantic Sound Scene Segmentation
Sripathi Sridhar, Gordon Wichern, Yoshiki Masuyama +2
Spatial semantic segmentation of sound scenes (S5) involves the detection and extraction of target sound events in audio files. However, in typical detection-extraction pipelines,…
Transcript-Supervised Post-Training of Generative Speech Enhancement on Real Recordings via Reinforce Adjoint Matching
Julius Richter, Christoph Boeddeker, Yoshiki Masuyama +4
We adapt Reinforce Adjoint Matching (RAM), a reward-based post-training method, to generative speech enhancement (SE). Starting from a pretrained SE model, RAM tilts the model's co…
Downstream-Task-Aware Unified Source Separation
Yoshiki Mitsui, Ryo Aihara, Tatsuhiko Saito +5
Task-aware unified source separation (TUSS) enables a single model to handle diverse separation tasks by conditioning on input prompts. However, conventional TUSS does not account…
Anomalous Sound Detection Meets Noise-Aware Self-Supervised Learning
Takuya Fujimura, Gordon Wichern, Yoshiki Masuyama +5
In this paper, we introduce noise-aware self-supervised learning (NA-SSL) models for noise-aware anomalous sound detection (NA-ASD). NA-ASD is an ASD task with two-channel audio re…
NABEATs: Noise-Aware Audio Representation Learning
Takuya Fujimura, Yoshiki Masuyama, Gordon Wichern +3
We propose the concept of noise-aware audio self-supervised learning (SSL), whose goal is to encode audio mixtures while suppressing undesired noise, and present Noise-Aware BEATs…
Technical Report for MERL's Real-TSE Challenge Submission
Dominik Klement, Yoshiki Masuyama, Christoph Boeddeker +4
Target speech extraction (TSE) has largely been dominated by neural network-based approaches trained and evaluated on synthetic fully overlapped data. The Real-TSE Challenge aims t…
Predictive-Generative Drift Decomposition for Speech Enhancement and Separation
Julius Richter, Yoshiki Masuyama, Christoph Boeddeker +3
We propose a plug-and-play framework for speech enhancement and separation that augments predictive methods with a generative speech prior. Our approach, termed Stochastic Interpol…
Local Density-Based Anomaly Score Normalization for Domain Generalization
Kevin Wilkinghoff, Haici Yang, Janek Ebbers +3
State-of-the-art anomalous sound detection (ASD) systems in domain-shifted conditions rely on projecting audio signals into an embedding space and using distance-based outlier dete…
FlexIO: Flexible Single- and Multi-Channel Speech Separation and Enhancement
Yoshiki Masuyama, Kohei Saijo, Francesco Paissan +6
Speech separation and enhancement (SSE) has advanced remarkably and achieved promising results in controlled settings, such as a fixed number of speakers and a fixed array configur…
Exploring Disentangled Neural Speech Codecs from Self-Supervised Representations
Ryo Aihara, Yoshiki Masuyama, Gordon Wichern +2
Neural audio codecs (NACs), which use neural networks to generate compact audio representations, have garnered interest for their applicability to many downstream tasks -- especial…
TF-Locoformer: Transformer with Local Modeling by Convolution for Speech Separation and Enhancement
Kohei Saijo, Gordon Wichern, François G. Germain +2
Time-frequency (TF) domain dual-path models achieve high-fidelity speech separation. While some previous state-of-the-art (SoTA) models rely on RNNs, this reliance means they lack…
Enhanced Reverberation as Supervision for Unsupervised Speech Separation
Kohei Saijo, Gordon Wichern, François G. Germain +2
Reverberation as supervision (RAS) is a framework that allows for training monaural speech separation models from multi-channel mixtures in an unsupervised manner. In RAS, models a…
Speech dereverberation constrained on room impulse response characteristics
Louis Bahrman, Mathieu Fontaine, Jonathan Le Roux +1
Single-channel speech dereverberation aims at extracting a dry speech signal from a recording affected by the acoustic reflections in a room. However, most current deep learning-ba…
Sound Event Bounding Boxes
Janek Ebbers, Francois G. Germain, Gordon Wichern +1
Sound event detection is the task of recognizing sounds and determining their extent (onset/offset times) within an audio clip. Existing systems commonly predict sound presence con…
Why does music source separation benefit from cacophony?
Chang-Bin Jeon, Gordon Wichern, François G. Germain +1
In music source separation, a standard training data augmentation procedure is to create new training samples by randomly combining instrument stems from different songs. These ran…
NIIRF: Neural IIR Filter Field for HRTF Upsampling and Personalization
Yoshiki Masuyama, Gordon Wichern, François G. Germain +4
Head-related transfer functions (HRTFs) are important for immersive audio, and their spatial interpolation has been studied to upsample finite measurements. Recently, neural fields…
GLA-Grad: A Griffin-Lim Extended Waveform Generation Diffusion Model
Haocheng Liu, Teysir Baoueb, Mathieu Fontaine +2
Diffusion models are receiving a growing interest for a variety of signal generation tasks such as speech or music synthesis. WaveGrad, for example, is a successful diffusion model…
SpecDiff-GAN: A Spectrally-Shaped Noise Diffusion GAN for Speech and Music Synthesis
Teysir Baoueb, Haocheng Liu, Mathieu Fontaine +2
Generative adversarial network (GAN) models can synthesize highquality audio signals while ensuring fast sample generation. However, they are difficult to train and are prone to se…
NeuroHeed+: Improving Neuro-steered Speaker Extraction with Joint Auditory Attention Detection
Zexu Pan, Gordon Wichern, Francois G. Germain +2
Neuro-steered speaker extraction aims to extract the listener's brain-attended speech signal from a multi-talker speech signal, in which the attention is derived from the cortical…
Scenario-Aware Audio-Visual TF-GridNet for Target Speech Extraction
Zexu Pan, Gordon Wichern, Yoshiki Masuyama +4
Target speech extraction aims to extract, based on a given conditioning cue, a target speech signal that is corrupted by interfering sources, such as noise or competing speakers. B…
TS-SEP: Joint Diarization and Separation Conditioned on Estimated Speaker Embeddings
Christoph Boeddeker, Aswin Shanmugam Subramanian, Gordon Wichern +2
Since diarization and source separation of meeting data are closely related tasks, we here propose an approach to perform the two objectives jointly. It builds upon the target-spea…
Hyperbolic Audio Source Separation
Darius Petermann, Gordon Wichern, Aswin Subramanian +1
We introduce a framework for audio source separation using embeddings on a hyperbolic manifold that compactly represent the hierarchical relationship between sound sources and time…
Late Audio-Visual Fusion for In-The-Wild Speaker Diarization
Zexu Pan, Gordon Wichern, François G. Germain +2
Speaker diarization is well studied for constrained audios but little explored for challenging in-the-wild videos, which have more speakers, shorter utterances, and inconsistent on…
(2.5+1)D Spatio-Temporal Scene Graphs for Video Question Answering
Anoop Cherian, Chiori Hori, Tim K. Marks +1
Spatio-temporal scene-graph approaches to video-based reasoning tasks, such as video question-answering (QA), typically construct such graphs for every video frame. These approache…