Fitting New Speakers Based on a Short Untranscribed Sample

Abstract

Learning-based Text To Speech systems have the potential to generalize from one speaker to the next and thus require a relatively short sample of any new voice. However, this promise is currently largely unrealized. We present a method that is designed to capture a new speaker from a short untranscribed audio sample. This is done by employing an additional network that given an audio sample, places the speaker in the embedding space. This network is trained as part of the speech synthesis system using various consistency losses. Our results demonstrate a greatly improved performance on both the dataset speakers, and, more importantly, when fitting new voices, even from very short samples.

References in corpus (5)

• WaveNet: A Generative Model for Raw Audio
• Training Very Deep Networks
• Deep Voice: Real-time Neural Text-to-Speech
• Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions
• Tacotron: Towards End-to-End Speech Synthesis

Cited by in corpus (12)

• Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis
• Sample Efficient Adaptive Text-to-Speech
• Deep Encoder-Decoder Models for Unsupervised Learning of Controllable Speech Synthesis
• Hierarchical Generative Modeling for Controllable Speech Synthesis
• Modeling Multi-speaker Latent Space to Improve Neural TTS: Quick Enrolling New Speaker and Enhancing Premium Voice
• Review of end-to-end speech synthesis technology based on deep learning
• Bootstrapping non-parallel voice conversion from speaker-adaptive text-to-speech
• Hierarchical Sequence to Sequence Voice Conversion with Limited Data
• Speaking style adaptation in Text-To-Speech synthesis using Sequence-to-sequence models with attention
• Multimodal speech synthesis architecture for unsupervised speaker adaptation
• AI based Presentation Creator With Customized Audio Content Delivery
• Neural voice cloning with a few low-quality samples