Forward Attention in Sequence-to-sequence Acoustic Modelling for Speech Synthesis

Abstract

This paper proposes a forward attention method for the sequenceto- sequence acoustic modeling of speech synthesis. This method is motivated by the nature of the monotonic alignment from phone sequences to acoustic sequences. Only the alignment paths that satisfy the monotonic condition are taken into consideration at each decoder timestep. The modified attention probabilities at each timestep are computed recursively using a forward algorithm. A transition agent for forward attention is further proposed, which helps the attention mechanism to make decisions whether to move forward or stay at each decoder timestep. Experimental results show that the proposed forward attention method achieves faster convergence speed and higher stability than the baseline attention method. Besides, the method of forward attention with transition agent can also help improve the naturalness of synthetic speech and control the speed of synthetic speech effectively.

arXiv comment5 pages, 3 figures, 2 tables. Published in IEEE International Conference on Acoustics, Speech and Signal Processing 2018 (ICASSP2018)

References in corpus (6)

• Neural Machine Translation by Jointly Learning to Align and Translate
• Sequence to Sequence Learning with Neural Networks
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Attention-Based Models for Speech Recognition
• Tacotron: Towards End-to-End Speech Synthesis

Cited by in corpus (24)

• Sequence-to-Sequence Acoustic Modeling for Voice Conversion
• Any-to-Many Voice Conversion with Location-Relative Sequence-to-Sequence Modeling
• Non-Parallel Sequence-to-Sequence Voice Conversion with Disentangled Linguistic and Speaker Representations
• Non-Attentive Tacotron: Robust and Controllable Neural TTS Synthesis Including Unsupervised Duration Modeling
• End-to-End Adversarial Text-to-Speech
• MultiSpeech: Multi-Speaker Text to Speech with Transformer
• Review of end-to-end speech synthesis technology based on deep learning
• JDI-T: Jointly trained Duration Informed Transformer for Text-To-Speech without Explicit Alignment
• Investigation of Japanese PnG BERT language model in text-to-speech synthesis for pitch accent language
• VARA-TTS: Non-Autoregressive Text-to-Speech Synthesis based on Very Deep VAE with Residual Attention
• ESPnet-TTS: Unified, Reproducible, and Integratable Open Source End-to-End Text-to-Speech Toolkit
• Zero-Shot Multi-Speaker Text-To-Speech with State-of-the-art Neural Speaker Embeddings
• Robust Sequence-to-Sequence Acoustic Modeling with Stepwise Monotonic Attention for Neural TTS
• A study of latent monotonic attention variants
• Initial investigation of an encoder-decoder end-to-end TTS framework using marginalization of monotonic hard latent alignments
• Investigation of enhanced Tacotron text-to-speech synthesis systems with self-attention for pitch accent language
• DSS: Synthesizing long Digital Ink using Data augmentation, Style encoding and Split generation
• Using IPA-Based Tacotron for Data Efficient Cross-Lingual Speaker Adaptation and Pronunciation Enhancement
• Triple M: A Practical Text-to-speech Synthesis System With Multi-guidance Attention And Multi-band Multi-time LPCNet
• Prosody Learning Mechanism for Speech Synthesis System Without Text Length Limit
• Rhythm-controllable Attention with High Robustness for Long Sentence Speech Synthesis
• FeatherTTS: Robust and Efficient attention based Neural TTS
• Nana-HDR: A Non-attentive Non-autoregressive Hybrid Model for TTS
• Enhancing audio quality for expressive Neural Text-to-Speech