On-device neural speech synthesis

Abstract

Recent advances in text-to-speech (TTS) synthesis, such as Tacotron and WaveRNN, have made it possible to construct a fully neural network based TTS system, by coupling the two components together. Such a system is conceptually simple as it only takes grapheme or phoneme input, uses Mel-spectrogram as an intermediate feature, and directly generates speech samples. The system achieves quality equal or close to natural speech. However, the high computational cost of the system and issues with robustness have limited their usage in real-world speech synthesis applications and products. In this paper, we present key modeling improvements and optimization strategies that enable deploying these models, not only on GPU servers, but also on mobile devices. The proposed system can generate high-quality 24 kHz speech at 5x faster than real time on server and 3x faster than real time on mobile devices.

arXiv comment7 pages 2 figures, accepted to ASRU 2021

References in corpus (8)

• WaveNet: A Generative Model for Raw Audio
• Categorical Reparameterization with Gumbel-Softmax
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis
• SampleRNN: An Unconditional End-to-End Neural Audio Generation Model
• Online and Linear-Time Attention by Enforcing Monotonic Alignments
• Efficient Neural Audio Synthesis
• Parallel Tacotron: Non-Autoregressive and Controllable TTS