Tacotron: Towards End-to-End Speech Synthesis
arXiv:1703.10135
Abstract
A text-to-speech synthesis system typically consists of multiple stages, such as a text analysis frontend, an acoustic model and an audio synthesis module. Building these components often requires extensive domain expertise and may contain brittle design choices. In this paper, we present Tacotron, an end-to-end generative text-to-speech model that synthesizes speech directly from characters. Given <text, audio> pairs, the model can be trained completely from scratch with random initialization. We present several key techniques to make the sequence-to-sequence framework perform well for this challenging task. Tacotron achieves a 3.82 subjective 5-scale mean opinion score on US English, outperforming a production parametric system in terms of naturalness. In addition, since Tacotron generates speech at the frame level, it's substantially faster than sample-level autoregressive methods.
Submitted to Interspeech 2017. v2 changed paper title to be consistent with our conference submission (no content change other than typo fixes)
References in corpus (6)
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
- WaveNet: A Generative Model for Raw Audio
- Deep Voice: Real-time Neural Text-to-Speech
- SampleRNN: An Unconditional End-to-End Neural Audio Generation Model
- RNN Approaches to Text Normalization: A Challenge
Cited by in corpus (83)
- FastSpeech: Fast, Robust and Controllable Text to Speech
- Emotion Intensity and its Control for Emotional Voice Conversion
- Fast Spectrogram Inversion using Multi-head Convolutional Neural Networks
- MelNet: A Generative Model for Audio in the Frequency Domain
- Forward Attention in Sequence-to-sequence Acoustic Modelling for Speech Synthesis
- Sound field reconstruction in rooms: inpainting meets super-resolution
- Flowtron: an Autoregressive Flow-based Generative Network for Text-to-Speech Synthesis
- Behavioral Use Licensing for Responsible AI
- Semi-Supervised Neural Architecture Search
- Learning the Travelling Salesperson Problem Requires Rethinking Generalization
- Characterization of Analytic Wavelet Transforms and a New Phaseless Reconstruction Algorithm
- Audeo: Audio Generation for a Silent Performance Video
- Unsupervised End-to-End Learning of Discrete Linguistic Units for Voice Conversion
- Monotonic Chunkwise Attention
- MultiSpeech: Multi-Speaker Text to Speech with Transformer
- On the Resilience of Biometric Authentication Systems against Random Inputs
- Fine-grained Style Modeling, Transfer and Prediction in Text-to-Speech Synthesis via Phone-Level Content-Style Disentanglement
- FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset
- Towards Robust Neural Vocoding for Speech Generation: A Survey
- NU-GAN: High resolution neural upsampling with GAN
- Attacking Speaker Recognition With Deep Generative Models
- SAPIEN: Affective Virtual Agents Powered by Large Language Models
- One-shot Voice Conversion by Separating Speaker and Content Representations with Instance Normalization
- Preech: A System for Privacy-Preserving Speech Transcription
- Recent Advances and Trends in Multimodal Deep Learning: A Review
- TTS-Portuguese Corpus: a corpus for speech synthesis in Brazilian Portuguese
- Listening while Speaking: Speech Chain by Deep Learning
- Fitting New Speakers Based on a Short Untranscribed Sample
- Unified Mandarin TTS Front-end Based on Distilled BERT Model
- Semi-Supervised Generative Modeling for Controllable Speech Synthesis
- Noise Adaptive Speech Enhancement using Domain Adversarial Training
- Mel-spectrogram augmentation for sequence to sequence voice conversion
- Machine Speech Chain with One-shot Speaker Adaptation
- Prosody Transfer in Neural Text to Speech Using Global Pitch and Loudness Features
- DurIAN-SC: Duration Informed Attention Network based Singing Voice Conversion System
- SpecMix : A Mixed Sample Data Augmentation method for Training withTime-Frequency Domain Features
- Learning Singing From Speech
- Multi-target Voice Conversion without Parallel Data by Adversarially Learning Disentangled Audio Representations
- AdaSpeech 3: Adaptive Text to Speech for Spontaneous Style
- Diff-TTS: A Denoising Diffusion Model for Text-to-Speech
- Singing Voice Synthesis Using Deep Autoregressive Neural Networks for Acoustic Modeling
- Joint training framework for text-to-speech and voice conversion using multi-source Tacotron and WaveNet
- Variational Bi-LSTMs
- Effective parameter estimation methods for an ExcitNet model in generative text-to-speech systems
- Semi-supervised URL Segmentation with Recurrent Neural Networks Pre-trained on Knowledge Graph Entities
- Statistical Parametric Speech Synthesis Using Generative Adversarial Networks Under A Multi-task Learning Framework
- PortaSpeech: Portable and High-Quality Generative Text-to-Speech
- End-to-End Feedback Loss in Speech Chain Framework via Straight-Through Estimator
- A Variational Prosody Model for Mapping the Context-Sensitive Variation of Functional Prosodic Prototypes
- Hierarchical Sequence to Sequence Voice Conversion with Limited Data
- Transfer Learning from Monolingual ASR to Transcription-free Cross-lingual Voice Conversion
- Data Efficient Voice Cloning from Noisy Samples with Domain Adversarial Training
- DiDiSpeech: A Large Scale Mandarin Speech Corpus
- Attention Forcing for Sequence-to-sequence Model Training
- AttS2S-VC: Sequence-to-Sequence Voice Conversion with Attention and Context Preservation Mechanisms
- Speaking style adaptation in Text-To-Speech synthesis using Sequence-to-sequence models with attention
- Multi-Reference Neural TTS Stylization with Adversarial Cycle Consistency
- Embodied Self-supervised Learning by Coordinated Sampling and Training
- Speech-to-Singing Conversion in an Encoder-Decoder Framework
- A Framework for Synthetic Audio Conversations Generation using Large Language Models
- Non-iterative Filter Bank Phase (Re)Construction
- The Multi-speaker Multi-style Voice Cloning Challenge 2021
- Building Multi lingual TTS using Cross Lingual Voice Conversion
- FBWave: Efficient and Scalable Neural Vocoders for Streaming Text-To-Speech on the Edge
- Unsupervised Learning of Audio Perception for Robotics Applications: Learning to Project Data to T-SNE/UMAP space
- A unified sequence-to-sequence front-end model for Mandarin text-to-speech synthesis
- Rhythm-Flexible Voice Conversion without Parallel Data Using Cycle-GAN over Phoneme Posteriorgram Sequences
- Phoneme-based Distribution Regularization for Speech Enhancement
- Whispered and Lombard Neural Speech Synthesis
- Audiovisual Speech Synthesis using Tacotron2
- Multi-scale Alignment and Contextual History for Attention Mechanism in Sequence-to-sequence Model
- Emotional Voice Conversion using Multitask Learning with Text-to-speech
- Zero-Shot Text-to-Speech for Text-Based Insertion in Audio Narration
- Facetron: A Multi-speaker Face-to-Speech Model based on Cross-modal Latent Representations
- LSTM Acoustic Models Learn to Align and Pronounce with Graphemes
- Generalized Spoofing Detection Inspired from Audio Generation Artifacts
- Mathematical Vocoder Algorithm : Modified Spectral Inversion for Efficient Neural Speech Synthesis
- AI based Presentation Creator With Customized Audio Content Delivery
- AdaSpeech 2: Adaptive Text to Speech with Untranscribed Data
- GSEP: A robust vocal and accompaniment separation system using gated CBHG module and loudness normalization
- Learning neural trans-dimensional random field language models with noise-contrastive estimation
- Large-scale Speaker Retrieval on Random Speaker Variability Subspace
- Adversarially Trained Autoencoders for Parallel-Data-Free Voice Conversion