fairseq S^2: A Scalable and Integrable Speech Synthesis Toolkit
arXiv:2109.06912
Abstract
This paper presents fairseq S^2, a fairseq extension for speech synthesis. We implement a number of autoregressive (AR) and non-AR text-to-speech models, and their multi-speaker variants. To enable training speech synthesis models with less curated data, a number of preprocessing tools are built and their importance is shown empirically. To facilitate faster iteration of development and analysis, a suite of automatic metrics is included. Apart from the features added specifically for this extension, fairseq S^2 also benefits from the scalability offered by fairseq and can be easily integrated with other state-of-the-art systems provided in this framework. The code, documentation, and pre-trained models are available at https://github.com/pytorch/fairseq/tree/master/examples/speech_synthesis.
arXiv commentAccepted to EMNLP 2021 Demo
References in corpus (18)
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Neural Discrete Representation Learning
- Mixed Precision Training
- Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis
- FastSpeech 2: Fast and High-Quality End-to-End Text to Speech
- Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis
- vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations
- Deep Voice 2: Multi-Speaker Neural Text-to-Speech
- A Survey on Neural Speech Synthesis
- Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning
- fairseq S2T: Fast Speech-to-Text Modeling with fairseq
- ESPnet: End-to-End Speech Processing Toolkit
- Sequence-to-Sequence Models Can Directly Translate Foreign Speech
- Hierarchical Generative Modeling for Controllable Speech Synthesis
- MultiSpeech: Multi-Speaker Text to Speech with Transformer
- Multilingual Speech Translation with Efficient Finetuning of Pretrained Models
- DiscreTalk: Text-to-Speech as a Machine Translation Problem
- Learning Hierarchical Discrete Linguistic Units from Visually-Grounded Speech