63 citations · 364 across the 60 of their papers we have counts for
61 papers
VIRTUE: Visual-Interactive Text-Image Universal Embedder
Wei-Yao Wang, Kazuya Tateishi, Qiyu Wu +2
Multimodal representation learning models have demonstrated successful operation across complex tasks, and the integration of vision-language models (VLMs) has further enabled embe…
Schrödinger Bridge Consistency Trajectory Models for Speech Enhancement
Shuichiro Nishigori, Koichi Saito, Naoki Murata +3
Speech enhancement (SE) utilizing diffusion models is a promising technology that improves speech quality in noisy speech data. Furthermore, the Schrödinger bridge (SB) has recentl…
LOCKEY: A Novel Approach to Model Authentication and Deepfake Tracking
Mayank Kumar Singh, Naoya Takahashi, Wei-Hsiang Liao +1
This paper presents a novel approach to deter unauthorized deepfakes and enable user tracking in generative models, even when the user has full access to the model parameters, by i…
GRAFX: An Open-Source Library for Audio Processing Graphs in PyTorch
Sungho Lee, Marco Martínez-Ramírez, Wei-Hsiang Liao +4
We present GRAFX, an open-source library designed for handling audio processing graphs in PyTorch. Along with various library functionalities, we describe technical details on the…
SilentCipher: Deep Audio Watermarking
Mayank Kumar Singh, Naoya Takahashi, Weihsiang Liao +1
In the realm of audio watermarking, it is challenging to simultaneously encode imperceptible messages while enhancing the message capacity and robustness. Although recent advanceme…
VRDMG: Vocal Restoration via Diffusion Posterior Sampling with Multiple Guidance
Carlos Hernandez-Olivan, Koichi Saito, Naoki Murata +4
Restoring degraded music signals is essential to enhance audio quality for downstream music manipulation. Recent diffusion-based music restoration methods have demonstrated impress…
Timbre-Trap: A Low-Resource Framework for Instrument-Agnostic Music Transcription
Frank Cwitkowitz, Kin Wai Cheuk, Woosung Choi +4
In recent years, research on music transcription has focused mainly on architecture design and instrument-specific data acquisition. With the lack of availability of diverse datase…
Zero- and Few-shot Sound Event Localization and Detection
Kazuki Shimada, Kengo Uchida, Yuichiro Koyama +4
Sound event localization and detection (SELD) systems estimate direction-of-arrival (DOA) and temporal activation for sets of target classes. Neural network (NN)-based SELD systems…
BigVSAN: Enhancing GAN-based Neural Vocoders with Slicing Adversarial Network
Takashi Shibuya, Yuhta Takida, Yuki Mitsufuji
Generative adversarial network (GAN)-based vocoders have been intensively studied because they can synthesize high-fidelity audio waveforms faster than real-time. However, it has b…
The Sound Demixing Challenge 2023 $\unicode{x2013}$ Cinematic Demixing Track
Stefan Uhlich, Giorgio Fabbro, Masato Hirano +14
This paper summarizes the cinematic demixing (CDX) track of the Sound Demixing Challenge 2023 (SDX'23). We provide a comprehensive summary of the challenge setup, detailing the str…
The Sound Demixing Challenge 2023 $\unicode{x2013}$ Music Demixing Track
Giorgio Fabbro, Stefan Uhlich, Chieh-Hsin Lai +24
This paper summarizes the music demixing (MDX) track of the Sound Demixing Challenge (SDX'23). We provide a summary of the challenge setup and introduce the task of robust music so…
Automatic Piano Transcription with Hierarchical Frequency-Time Transformer
Keisuke Toyama, Taketo Akama, Yukara Ikemiya +3
Taking long-term spectral and temporal dependencies into account is essential for automatic piano transcription. This is especially helpful when determining the precise onset and o…
STARSS23: An Audio-Visual Dataset of Spatial Recordings of Real Scenes with Spatiotemporal Annotations of Sound Events
Kazuki Shimada, Archontis Politis, Parthasaarathy Sudarsanam +9
While direction of arrival (DOA) of sound events is generally estimated from multichannel audio data recorded in a microphone array, sound events usually derive from visually perce…
On the Equivalence of Consistency-Type Models: Consistency Models, Consistent Diffusion Models, and Fokker-Planck Regularization
Chieh-Hsin Lai, Yuhta Takida, Toshimitsu Uesaka +3
The emergence of various notions of ``consistency'' in diffusion models has garnered considerable attention and helped achieve improved sample quality, likelihood estimation, and a…
PeaCoK: Persona Commonsense Knowledge for Consistent and Engaging Narratives
Silin Gao, Beatriz Borges, Soyoung Oh +5
Sustaining coherent and engaging narratives requires dialogue or storytelling agents to understand how the personas of speakers or listeners ground the narrative. Specifically, the…
Diffusion-Based Speech Enhancement with Joint Generative and Predictive Decoders
Hao Shi, Kazuki Shimada, Masato Hirano +6
Diffusion-based generative speech enhancement (SE) has recently received attention, but reverse diffusion remains time-consuming. One solution is to initialize the reverse diffusio…
The Whole Is Greater than the Sum of Its Parts: Improving Music Source Separation by Bridging Network
Ryosuke Sawata, Naoya Takahashi, Stefan Uhlich +2
This paper presents the crossing scheme (X-scheme) for improving the performance of deep neural network (DNN)-based music source separation (MSS) with almost no increasing calculat…
Extending Audio Masked Autoencoders Toward Audio Restoration
Zhi Zhong, Hao Shi, Masato Hirano +5
Audio classification and restoration are among major downstream tasks in audio signal processing. However, restoration derives less of a benefit from pretrained models compared to…
Diffusion-based Signal Refiner for Speech Enhancement and Separation
Masato Hirano, Ryosuke Sawata, Naoki Murata +2
Although recent speech processing technologies have achieved significant improvements in objective metrics, there still remains a gap in human perceptual quality. This paper propos…
Cross-modal Face- and Voice-style Transfer
Naoya Takahashi, Mayank K. Singh, Yuki Mitsufuji
Image-to-image translation and voice conversion enable the generation of a new facial image and voice while maintaining some of the semantics such as a pose in an image and linguis…
An Attention-based Approach to Hierarchical Multi-label Music Instrument Classification
Zhi Zhong, Masato Hirano, Kazuki Shimada +3
Although music is typically multi-label, many works have studied hierarchical music tagging with simplified settings such as single-label data. Moreover, there lacks a framework to…
SAN: Inducing Metrizability of GAN with Discriminative Normalized Linear Layer
Yuhta Takida, Masaaki Imaizumi, Takashi Shibuya +4
Generative adversarial networks (GANs) learn a target probability distribution by optimizing a generator and a discriminator with minimax objectives. This paper addresses the quest…
GibbsDDRM: A Partially Collapsed Gibbs Sampler for Solving Blind Inverse Problems with Denoising Diffusion Restoration
Naoki Murata, Koichi Saito, Chieh-Hsin Lai +4
Pre-trained diffusion models have been successfully used as priors in a variety of linear inverse problems, where the goal is to reconstruct a signal from noisy linear measurements…
CLIPSep: Learning Text-queried Sound Separation with Noisy Unlabeled Videos
Hao-Wen Dong, Naoya Takahashi, Yuki Mitsufuji +2
Recent years have seen progress beyond domain-specific sound separation for speech or music towards universal sound separation for arbitrary sounds. Prior work on universal sound s…