52 citations · 95 across the 12 of their papers we have counts for
13 papers
Towards Perception-Informed Latent HRTF Representations
You Zhang, Andrew Francl, Ruohan Gao +3
Personalized head-related transfer functions (HRTFs) are essential for ensuring a realistic auditory experience over headphones, because they take into account individual anatomica…
Investigating an Overfitting and Degeneration Phenomenon in Self-Supervised Multi-Pitch Estimation
Frank Cwitkowitz, Zhiyao Duan
Multi-Pitch Estimation (MPE) continues to be a sought after capability of Music Information Retrieval (MIR) systems, and is critical for many applications and downstream tasks invo…
PartialEdit: Identifying Partial Deepfakes in the Era of Neural Speech Editing
You Zhang, Baotong Tian, Lin Zhang +1
Neural speech editing enables seamless partial edits to speech utterances, allowing modifications to selected content while preserving the rest of the audio unchanged. This useful…
HARP 2.0: Expanding Hosted, Asynchronous, Remote Processing for Deep Learning in the DAW
Christodoulos Benetatos, Frank Cwitkowitz, Nathan Pruyne +4
HARP 2.0 brings deep learning models to digital audio workstation (DAW) software through hosted, asynchronous, remote processing, allowing users to route audio from a plug-in inter…
Audio Visual Segmentation Through Text Embeddings
Kyungbok Lee, You Zhang, Zhiyao Duan
The goal of Audio-Visual Segmentation (AVS) is to localize and segment the sounding source objects from video frames. Research on AVS suffers from data scarcity due to the high cos…
SVDD 2024: The Inaugural Singing Voice Deepfake Detection Challenge
You Zhang, Yongyi Zang, Jiatong Shi +3
With the advancements in singing voice generation and the growing presence of AI singers on media platforms, the inaugural Singing Voice Deepfake Detection (SVDD) Challenge aims to…
CtrSVDD: A Benchmark Dataset and Baseline Analysis for Controlled Singing Voice Deepfake Detection
Yongyi Zang, Jiatong Shi, You Zhang +8
Recent singing voice synthesis and conversion advancements necessitate robust singing voice deepfake detection (SVDD) models. Current SVDD datasets face challenges due to limited c…
A Multi-Stream Fusion Approach with One-Class Learning for Audio-Visual Deepfake Detection
Kyungbok Lee, You Zhang, Zhiyao Duan
This paper addresses the challenge of developing a robust audio-visual deepfake detection model. In practical use cases, new generation algorithms are continually emerging, and the…
SVDD Challenge 2024: A Singing Voice Deepfake Detection Challenge Evaluation Plan
You Zhang, Yongyi Zang, Jiatong Shi +5
The rapid advancement of AI-generated singing voices, which now closely mimic natural human singing and align seamlessly with musical scores, has led to heightened concerns for art…
Toward Fully Self-Supervised Multi-Pitch Estimation
Frank Cwitkowitz, Zhiyao Duan
Multi-pitch estimation is a decades-long research problem involving the detection of pitch activity associated with concurrent musical events within multi-instrument mixtures. Supe…
Learning Arousal-Valence Representation from Categorical Emotion Labels of Speech
Enting Zhou, You Zhang, Zhiyao Duan
Dimensional representations of speech emotions such as the arousal-valence (AV) representation provide a continuous and fine-grained description and control than their categorical…
SynthTab: Leveraging Synthesized Data for Guitar Tablature Transcription
Yongyi Zang, Yi Zhong, Frank Cwitkowitz +1
Guitar tablature is a form of music notation widely used among guitarists. It captures not only the musical content of a piece, but also its implementation and ornamentation on the…
SingFake: Singing Voice Deepfake Detection
Yongyi Zang, You Zhang, Mojtaba Heydari +1
The rise of singing voice synthesis presents critical challenges to artists and industry stakeholders over unauthorized voice usage. Unlike synthesized speech, synthesized singing…