11 citations · 12 across the 8 of their papers we have counts for
9 papers
PerformSinger: Multimodal Singing Voice Synthesis Leveraging Synchronized Lip Cues from Singing Performance Videos
Ke Gu, Zhicong Wu, Peng Bai +5
Existing singing voice synthesis (SVS) models largely rely on fine-grained, phoneme-level durations, which limits their practical application. These methods overlook the complement…
Objective Evaluation of Prosody and Intelligibility in Speech Synthesis via Conditional Prediction of Discrete Tokens
Ismail Rasim Ulgen, Zongyang Du, Junchen Lu +2
Objective evaluation of synthesized speech is critical for advancing speech generation systems, yet existing metrics for intelligibility and prosody remain limited in scope and wea…
EmotionRankCLAP: Bridging Natural Language Speaking Styles and Ordinal Speech Emotion via Rank-N-Contrast
Shreeram Suresh Chandra, Lucas Goncalves, Junchen Lu +2
Current emotion-based contrastive language-audio pretraining (CLAP) methods typically learn by naïvely aligning audio samples with corresponding text prompts. Consequently, this ap…
Text-to-Speech for Unseen Speakers via Low-Complexity Discrete Unit-Based Frame Selection
Ismail Rasim Ulgen, Shreeram Suresh Chandra, Junchen Lu +1
Synthesizing the voices of unseen speakers remains a persisting challenge in multi-speaker text-to-speech (TTS). Existing methods model speaker characteristics through speaker cond…
Style Mixture of Experts for Expressive Text-To-Speech Synthesis
Ahad Jawaid, Shreeram Suresh Chandra, Junchen Lu +1
Recent advances in style transfer text-to-speech (TTS) have improved the expressiveness of synthesized speech. However, encoding stylistic information (e.g., timbre, emotion, and p…
Converting Anyone's Voice: End-to-End Expressive Voice Conversion with a Conditional Diffusion Model
Zongyang Du, Junchen Lu, Kun Zhou +2
Expressive voice conversion (VC) conducts speaker identity conversion for emotional speakers by jointly converting speaker identity and emotional style. Emotional style modeling fo…
High-Quality Automatic Voice Over with Accurate Alignment: Supervision through Self-Supervised Discrete Speech Units
Junchen Lu, Berrak Sisman, Mingyang Zhang +1
The goal of Automatic Voice Over (AVO) is to generate speech in sync with a silent video given its text script. Recent AVO frameworks built upon text-to-speech synthesis (TTS) have…
VisualTTS: TTS with Accurate Lip-Speech Synchronization for Automatic Voice Over
Junchen Lu, Berrak Sisman, Rui Liu +2
In this paper, we formulate a novel task to synthesize speech in sync with a silent pre-recorded video, denoted as automatic voice over (AVO). Unlike traditional speech synthesis,…
VAW-GAN for Singing Voice Conversion with Non-parallel Training Data
Junchen Lu, Kun Zhou, Berrak Sisman +1
Singing voice conversion aims to convert singer's voice from source to target without changing singing content. Parallel training data is typically required for the training of sin…