AISHELL-3: A Multi-speaker Mandarin TTS Corpus and the Baselines

Abstract

In this paper, we present AISHELL-3, a large-scale and high-fidelity multi-speaker Mandarin speech corpus which could be used to train multi-speaker Text-to-Speech (TTS) systems. The corpus contains roughly 85 hours of emotion-neutral recordings spoken by 218 native Chinese mandarin speakers. Their auxiliary attributes such as gender, age group and native accents are explicitly marked and provided in the corpus. Accordingly, transcripts in Chinese character-level and pinyin-level are provided along with the recordings. We present a baseline system that uses AISHELL-3 for multi-speaker Madarin speech synthesis. The multi-speaker speech synthesis system is an extension on Tacotron-2 where a speaker verification model and a corresponding loss regarding voice similarity are incorporated as the feedback constraint. We aim to use the presented corpus to build a robust synthesis model that is able to achieve zero-shot voice cloning. The system trained on this dataset also generalizes well on speakers that are never seen in the training process. Objective evaluation results from our experiments show that the proposed multi-speaker synthesis system achieves high voice similarity concerning both speaker embedding similarity and equal error rate measurement. The dataset, baseline system code and generated samples are available online.

References in corpus (2)

• MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis
• Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis

Cited by in corpus (25)

• A Survey on Neural Speech Synthesis
• KazakhTTS: An Open-Source Kazakh Text-to-Speech Synthesis Dataset
• Augmented Datasheets for Speech Datasets and Ethical Decision-Making
• VoiceFixer: Toward General Speech Restoration with Neural Vocoder
• Review of end-to-end speech synthesis technology based on deep learning
• SANE-TTS: Stable And Natural End-to-End Multilingual Text-to-Speech
• Heterogeneous Target Speech Separation
• Parameter-Efficient Learning for Text-to-Speech Accent Adaptation
• INTERSPEECH 2021 ConferencingSpeech Challenge: Towards Far-field Multi-Channel Speech Enhancement for Video Conferencing
• The Multi-speaker Multi-style Voice Cloning Challenge 2021
• End-to-End Mandarin Tone Classification with Short Term Context Information
• Building Bilingual and Code-Switched Voice Conversion with Limited Training Data Using Embedding Consistency Loss
• Uformer: A Unet based dilated complex & real dual-path conformer network for simultaneous speech enhancement and dereverberation
• Deep Spectro-temporal Artifacts for Detecting Synthesized Speech
• Automatic recognition of suprasegmentals in speech
• Multi-Task Learning in Utterance-Level and Segmental-Level Spoof Detection
• EditSpeech: A Text Based Speech Editing System Using Partial Inference and Bidirectional Fusion
• Controllable Generation of Artificial Speaker Embeddings through Discovery of Principal Directions
• AdaVocoder: Adaptive Vocoder for Custom Voice
• Improving Prosody for Unseen Texts in Speech Synthesis by Utilizing Linguistic Information and Noisy Data
• Half-Truth: A Partially Fake Audio Detection Dataset
• RefineGAN: Universally Generating Waveform Better than Ground Truth with Highly Accurate Pitch and Intensity Responses
• Wav-BERT: Cooperative Acoustic and Linguistic Representation Learning for Low-Resource Speech Recognition
• Multi-channel Speech Enhancement with 2-D Convolutional Time-frequency Domain Features and a Pre-trained Acoustic Model
• How Speech is Recognized to Be Emotional - A Study Based on Information Decomposition