17 citations · 65 across the 38 of their papers we have counts for
41 papers
Velocity Potential Neural Field for Efficient Ambisonics Impulse Response Modeling
Yoshiki Masuyama, Francois G. Germain, Gordon Wichern +2
First-order Ambisonics (FOA) is a standard spatial audio format based on spherical harmonic decomposition. Its zeroth- and first-order components capture the sound pressure and par…
Robot Confirmation Generation and Action Planning Using Long-context Q-Former Integrated with Multimodal LLM
Chiori Hori, Yoshiki Masuyama, Siddarth Jain +4
Human-robot collaboration towards a shared goal requires robots to understand human action and interaction with the surrounding environment. This paper focuses on human-robot inter…
SUNAC: Source-aware Unified Neural Audio Codec
Ryo Aihara, Yoshiki Masuyama, Francesco Paissan +3
Neural audio codecs (NACs) provide compact representations that can be leveraged in many downstream applications, in particular large language models. Yet most NACs encode mixtures…
Physics-Informed Direction-Aware Neural Acoustic Fields
Yoshiki Masuyama, François G. Germain, Gordon Wichern +2
This paper presents a physics-informed neural network (PINN) for modeling first-order Ambisonic (FOA) room impulse responses (RIRs). PINNs have demonstrated promising performance i…
Factorized RVQ-GAN For Disentangled Speech Tokenization
Sameer Khurana, Dominik Klement, Antoine Laurent +13
We propose Hierarchical Audio Codec (HAC), a unified neural speech codec that factorizes its bottleneck into three linguistic levels-acoustic, phonetic, and lexical-within a single…
HASRD: Hierarchical Acoustic and Semantic Representation Disentanglement
Amir Hussein, Sameer Khurana, Gordon Wichern +2
Effective speech representations for spoken language models must balance semantic relevance with acoustic fidelity for high-quality reconstruction. However, existing approaches str…
Direction-Aware Neural Acoustic Fields for Few-Shot Interpolation of Ambisonic Impulse Responses
Christopher Ick, Gordon Wichern, Yoshiki Masuyama +2
The characteristics of a sound field are intrinsically linked to the geometric and spatial properties of the environment surrounding a sound source and a listener. The physics of s…
Data Augmentation Using Neural Acoustic Fields With Retrieval-Augmented Pre-training
Christopher Ick, Gordon Wichern, Yoshiki Masuyama +2
This report details MERL's system for room impulse response (RIR) estimation submitted to the Generative Data Augmentation Workshop at ICASSP 2025 for Augmenting RIR Data (Task 1)…
Handling Domain Shifts for Anomalous Sound Detection: A Review of DCASE-Related Work
Kevin Wilkinghoff, Takuya Fujimura, Keisuke Imoto +3
When detecting anomalous sounds in complex environments, one of the main difficulties is that trained models must be sensitive to subtle differences in monitored target signals, wh…
Retrieval-Augmented Neural Field for HRTF Upsampling and Personalization
Yoshiki Masuyama, Gordon Wichern, François G. Germain +2
Head-related transfer functions (HRTFs) with dense spatial grids are desired for immersive binaural audio generation, but their recording is time-consuming. Although HRTF spatial u…
Leveraging Audio-Only Data for Text-Queried Target Sound Extraction
Kohei Saijo, Janek Ebbers, François G. Germain +3
The goal of text-queried target sound extraction (TSE) is to extract from a mixture a sound source specified with a natural-language caption. While it is preferable to have access…
Disentangled Acoustic Fields For Multimodal Physical Scene Understanding
Jie Yin, Andrew Luo, Yilun Du +4
We study the problem of multimodal physical scene understanding, where an embodied agent needs to find fallen objects by inferring object properties, direction, and distance of an…
SMITIN: Self-Monitored Inference-Time INtervention for Generative Music Transformers
Junghyun Koo, Gordon Wichern, Francois G. Germain +2
We introduce Self-Monitored Inference-Time INtervention (SMITIN), an approach for controlling an autoregressive generative music transformer using classifier probes. These simple l…
Generation or Replication: Auscultating Audio Latent Diffusion Models
Dimitrios Bralios, Gordon Wichern, François G. Germain +4
The introduction of audio latent diffusion models possessing the ability to generate realistic sound clips on demand from a text description has the potential to revolutionize how…
Improving Audio Captioning Models with Fine-grained Audio Features, Text Embedding Supervision, and LLM Mix-up Augmentation
Shih-Lun Wu, Xuankai Chang, Gordon Wichern +4
Automated audio captioning (AAC) aims to generate informative descriptions for various sounds from nature and/or human activities. In recent years, AAC has quickly attracted resear…
The Sound Demixing Challenge 2023 $\unicode{x2013}$ Cinematic Demixing Track
Stefan Uhlich, Giorgio Fabbro, Masato Hirano +14
This paper summarizes the cinematic demixing (CDX) track of the Sound Demixing Challenge 2023 (SDX'23). We provide a comprehensive summary of the challenge setup, detailing the str…
Style-transfer based Speech and Audio-visual Scene Understanding for Robot Action Sequence Acquisition from Videos
Chiori Hori, Puyuan Peng, David Harwath +7
To realize human-robot collaboration, robots need to execute actions for new tasks according to human instructions given finite prior knowledge. Human experts can share their knowl…
Pac-HuBERT: Self-Supervised Music Source Separation via Primitive Auditory Clustering and Hidden-Unit BERT
Ke Chen, Gordon Wichern, François G. Germain +1
In spite of the progress in music source separation research, the small amount of publicly-available clean source data remains a constant limiting factor for performance. Thus, rec…
Tackling the Cocktail Fork Problem for Separation and Transcription of Real-World Soundtracks
Darius Petermann, Gordon Wichern, Aswin Shanmugam Subramanian +2
Emulating the human ability to solve the cocktail party problem, i.e., focus on a source of interest in a complex acoustic scene, is a long standing goal of audio source separation…
Reverberation as Supervision for Speech Separation
Rohith Aralikatti, Christoph Boeddeker, Gordon Wichern +2
This paper proposes reverberation as supervision (RAS), a novel unsupervised loss function for single-channel reverberant speech separation. Prior methods for unsupervised separati…
Optimal Condition Training for Target Source Separation
Efthymios Tzinis, Gordon Wichern, Paris Smaragdis +1
Recent research has shown remarkable performance in leveraging multiple extraneous conditional and non-mutually exclusive semantic concepts for sound source separation, allowing th…
Latent Iterative Refinement for Modular Source Separation
Dimitrios Bralios, Efthymios Tzinis, Gordon Wichern +2
Traditional source separation approaches train deep neural network models end-to-end with all the data available at once by minimizing the empirical risk on the whole training set.…
Cold Diffusion for Speech Enhancement
Hao Yen, François G. Germain, Gordon Wichern +1
Diffusion models have recently shown promising results for difficult enhancement tasks such as the conditional and unconditional restoration of natural images and audio signals. In…
SANE-TTS: Stable And Natural End-to-End Multilingual Text-to-Speech
Hyunjae Cho, Wonbin Jung, Junhyeok Lee +1
In this paper, we present SANE-TTS, a stable and natural end-to-end multilingual TTS model. By the difficulty of obtaining multilingual corpus for given speaker, training multiling…