Zero-Shot Multi-Speaker Text-To-Speech with State-of-the-art Neural Speaker Embeddings

Abstract

While speaker adaptation for end-to-end speech synthesis using speaker embeddings can produce good speaker similarity for speakers seen during training, there remains a gap for zero-shot adaptation to unseen speakers. We investigate multi-speaker modeling for end-to-end text-to-speech synthesis and study the effects of different types of state-of-the-art neural speaker embeddings on speaker similarity for unseen speakers. Learnable dictionary encoding-based speaker embeddings with angular softmax loss can improve equal error rates over x-vectors in a speaker verification task; these embeddings also improve speaker similarity and naturalness for unseen speakers when used for zero-shot adaptation to new speakers in end-to-end speech synthesis.

arXiv commentAccepted to ICASSP 2020

References in corpus (8)

• WaveNet: A Generative Model for Raw Audio
• VoxCeleb2: Deep Speaker Recognition
• VoxCeleb: a large-scale speaker identification dataset
• Forward Attention in Sequence-to-sequence Acoustic Modelling for Speech Synthesis
• Spatial Pyramid Encoding with Convex Length Normalization for Text-Independent Speaker Verification
• Modeling Multi-speaker Latent Space to Improve Neural TTS: Quick Enrolling New Speaker and Enhancing Premium Voice
• Multi-Speaker End-to-End Speech Synthesis
• Margin Matters: Towards More Discriminative Deep Neural Network Embeddings for Speaker Recognition

Cited by in corpus (4)

• NAUTILUS: a Versatile Voice Cloning System
• Attentron: Few-Shot Text-to-Speech Utilizing Attention-Based Variable-Length Embedding
• Noise Tokens: Learning Neural Noise Templates for Environment-Aware Speech Enhancement
• Speech Enhancement with Zero-Shot Model Selection