211 citations · 535 across the 34 of their papers we have counts for
35 papers
MuseTimbre: Zero-Shot Timbre Transfer by Controlling a Frozen Music Generator
Yuan-Chiao Cheng, Zhiyao Duan
Instrument timbre transfer re-voices a performance using the timbre of another instrument. Extracting the target timbre from an audio reference capture more nuances than inferring…
Mitigating Cross-Database Differences for Learning Unified HRTF Representation
Yutong Wen, You Zhang, Zhiyao Duan
Individualized head-related transfer functions (HRTFs) are crucial for accurate sound positioning in virtual auditory displays. As the acoustic measurement of HRTFs is resource-int…
SingNet: A Real-time Singing Voice Beat and Downbeat Tracking System
Mojtaba Heydari, Ju-Chiang Wang, Zhiyao Duan
Singing voice beat and downbeat tracking posses several applications in automatic music production, analysis and manipulation. Among them, some require real-time processing, such a…
Phase perturbation improves channel robustness for speech spoofing countermeasures
Yongyi Zang, You Zhang, Zhiyao Duan
In this paper, we aim to address the problem of channel robustness in speech countermeasure (CM) systems, which are used to distinguish synthetic speech from human natural speech.…
Transcription free filler word detection with Neural semi-CRFs
Ge Zhu, Yujia Yan, Juan-Pablo Caceres +1
Non-linguistic filler words, such as "uh" or "um", are prevalent in spontaneous speech and serve as indicators for expressing hesitation or uncertainty. Previous works for detectin…
SAMO: Speaker Attractor Multi-Center One-Class Learning for Voice Anti-Spoofing
Siwen Ding, You Zhang, Zhiyao Duan
Voice anti-spoofing systems are crucial auxiliaries for automatic speaker verification (ASV) systems. A major challenge is caused by unseen attacks empowered by advanced speech syn…
HRTF Field: Unifying Measured HRTF Magnitude Representation with Neural Fields
You Zhang, Yuxiang Wang, Zhiyao Duan
Head-related transfer functions (HRTFs) are a set of functions describing the spatial filtering effect of the outer ear (i.e., torso, head, and pinnae) onto sound sources at differ…
ControlVC: Zero-Shot Voice Conversion with Time-Varying Controls on Pitch and Speed
Meiying Chen, Zhiyao Duan
Recent developments in neural speech synthesis and vocoding have sparked a renewed interest in voice conversion (VC). Beyond timbre transfer, achieving controllability on para-ling…
Singing Beat Tracking With Self-supervised Front-end and Linear Transformers
Mojtaba Heydari, Zhiyao Duan
Tracking beats of singing voices without the presence of musical accompaniment can find many applications in music production, automatic song arrangement, and social media interact…
Rethinking Audio-visual Synchronization for Active Speaker Detection
Abudukelimu Wuerkaixi, You Zhang, Zhiyao Duan +1
Active speaker detection (ASD) systems are important modules for analyzing multi-talker conversations. They aim to detect which speakers or none are talking in a visual scene at an…
Predicting Global HRTFs From Scanned Head Geometry Using Deep Learning and Compact Representations
Yuxiang Wang, You Zhang, Zhiyao Duan +1
In the growing field of virtual auditory display, personalized head-related transfer functions (HRTFs) play a vital role in establishing an accurate sound image for mixed and augme…
A Data-Driven Methodology for Considering Feasibility and Pairwise Likelihood in Deep Learning Based Guitar Tablature Transcription Systems
Frank Cwitkowitz, Jonathan Driedger, Zhiyao Duan
Guitar tablature transcription is an important but understudied problem within the field of music information retrieval. Traditional signal processing approaches offer only limited…
Music Source Separation with Generative Flow
Ge Zhu, Jordan Darefsky, Fei Jiang +2
Fully-supervised models for source separation are trained on parallel mixture-source data and are currently state-of-the-art. However, such parallel data is often difficult to obta…
A Probabilistic Fusion Framework for Spoofing Aware Speaker Verification
You Zhang, Ge Zhu, Zhiyao Duan
The performance of automatic speaker verification (ASV) systems could be degraded by voice spoofing attacks. Most existing works aimed to develop standalone spoofing countermeasure…
A Novel 1D State Space for Efficient Music Rhythmic Analysis
Mojtaba Heydari, Matthew McCallum, Andreas Ehmann +1
Inferring music time structures has a broad range of applications in music production, processing and analysis. Scholars have proposed various methods to analyze different aspects…
A study of the robustness of raw waveform based speaker embeddings under mismatched conditions
Ge Zhu, Frank Cwitkowitz, Zhiyao Duan
In this paper, we conduct a cross-dataset study on parametric and non-parametric raw-waveform based speaker embeddings through speaker verification experiments. In general, we obse…
UR Channel-Robust Synthetic Speech Detection System for ASVspoof 2021
Xinhui Chen, You Zhang, Ge Zhu +1
In this paper, we present UR-AIR system submission to the logical access (LA) and the speech deepfake (DF) tracks of the ASVspoof 2021 Challenge. The LA and DF tasks focus on synth…
BeatNet: CRNN and Particle Filtering for Online Joint Beat Downbeat and Meter Tracking
Mojtaba Heydari, Frank Cwitkowitz, Zhiyao Duan
The online estimation of rhythmic information, such as beat positions, downbeat positions, and meter, is critical for many real-time music applications. Musical rhythm comprises co…
Learning Sparse Analytic Filters for Piano Transcription
Frank Cwitkowitz, Mojtaba Heydari, Zhiyao Duan
In recent years, filterbank learning has become an increasingly popular strategy for various audio-related machine learning tasks. This is partly due to its ability to discover tas…
Audiovisual Singing Voice Separation
Bochen Li, Yuxuan Wang, Zhiyao Duan
Separating a song into vocal and accompaniment components is an active research topic, and recent years witnessed an increased performance from supervised training using deep learn…
An Empirical Study on Channel Effects for Synthetic Voice Spoofing Countermeasure Systems
You Zhang, Ge Zhu, Fei Jiang +1
Spoofing countermeasure (CM) systems are critical in speaker verification; they aim to discern spoofing attacks from bona fide speech trials. In practice, however, acoustic conditi…
Don't look back: an online beat tracking method using RNN and enhanced particle filtering
Mojtaba Heydari, Zhiyao Duan
Online beat tracking (OBT) has always been a challenging task. Due to the inaccessibility of future data and the need to make inference in real-time. We propose Do not Look back! (…
Themes Informed Audio-visual Correspondence Learning
Runze Su, Fei Tao, Xudong Liu +6
The applications of short-term user-generated video (UGV), such as Snapchat, and Youtube short-term videos, booms recently, raising lots of multimodal machine learning tasks. Among…
One-class Learning Towards Synthetic Voice Spoofing Detection
You Zhang, Fei Jiang, Zhiyao Duan
Human voices can be used to authenticate the identity of the speaker, but the automatic speaker verification (ASV) systems are vulnerable to voice spoofing attacks, such as imperso…