Controllable Generation of Artificial Speaker Embeddings through Discovery of Principal Directions

Abstract

Customizing voice and speaking style in a speech synthesis system with intuitive and fine-grained controls is challenging, given that little data with appropriate labels is available. Furthermore, editing an existing human's voice also comes with ethical concerns. In this paper, we propose a method to generate artificial speaker embeddings that cannot be linked to a real human while offering intuitive and fine-grained control over the voice and speaking style of the embeddings, without requiring any labels for speaker or style. The artificial and controllable embeddings can be fed to a speech synthesis system, conditioned on embeddings of real humans during training, without sacrificing privacy during inference.

arXiv commentPublished at ISCA Interspeech 2023 https://www.isca-speech.org/archive/interspeech_2023/lux23_interspeech.html

References in corpus (11)

• ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification
• Barlow Twins: Self-Supervised Learning via Redundancy Reduction
• HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis
• Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis
• FastSpeech 2: Fast and High-Quality End-to-End Text to Speech
• SpeechBrain: A General-Purpose Speech Toolkit
• Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis
• MLS: A Large-Scale Multilingual Dataset for Speech Research
• Neural Voice Cloning with a Few Samples
• AISHELL-3: A Multi-speaker Mandarin TTS Corpus and the Baselines
• NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Quality