ContextSpeech: Expressive and Efficient Text-to-Speech for Paragraph Reading
arXiv:2307.00782 · doi:10.21437/Interspeech.2023-122
Abstract
While state-of-the-art Text-to-Speech systems can generate natural speech of very high quality at sentence level, they still meet great challenges in speech generation for paragraph / long-form reading. Such deficiencies are due to i) ignorance of cross-sentence contextual information, and ii) high computation and memory cost for long-form synthesis. To address these issues, this work develops a lightweight yet effective TTS system, ContextSpeech. Specifically, we first design a memory-cached recurrence mechanism to incorporate global text and speech context into sentence encoding. Then we construct hierarchically-structured textual semantics to broaden the scope for global context enhancement. Additionally, we integrate linearized self-attention to improve model efficiency. Experiments show that ContextSpeech significantly improves the voice quality and prosody expressiveness in paragraph reading with competitive model efficiency. Audio samples are available at: https://contextspeech.github.io/demo/
arXiv comment5 pages, 4 figures, Proceedings of Interspeech 2023
References in corpus (11)
- WaveNet: A Generative Model for Raw Audio
- Linformer: Self-Attention with Linear Complexity
- HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis
- MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis
- FastSpeech: Fast, Robust and Controllable Text to Speech
- FastSpeech 2: Fast and High-Quality End-to-End Text to Speech
- Parallel WaveNet: Fast High-Fidelity Speech Synthesis
- Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Search
- NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Quality
- ContextSpeech: Expressive and Efficient Text-to-Speech for Paragraph Reading
- Conversational End-to-End TTS for Voice Agent