TalkNet: Fully-Convolutional Non-Autoregressive Speech Synthesis Model

Abstract

We propose TalkNet, a convolutional non-autoregressive neural model for speech synthesis. The model consists of two feed-forward convolutional networks. The first network predicts grapheme durations. An input text is expanded by repeating each symbol according to the predicted duration. The second network generates a mel-spectrogram from the expanded text. To train a grapheme duration predictor, we add the grapheme duration to the training dataset using a pre-trained Connectionist Temporal Classification (CTC)-based speech recognition model. The explicit duration prediction eliminates word skipping and repeating. Experiments on the LJSpeech dataset show that the speech quality nearly matches auto-regressive models. The model is very compact -- it has 10.8M parameters, almost 3x less than the present state-of-the-art text-to-speech models. The non-autoregressive architecture allows for fast training and inference.

References in corpus (3)

• Mixed Precision Training
• Deep Voice: Real-time Neural Text-to-Speech
• NeMo: a toolkit for building AI applications using Neural Modules

Cited by in corpus (6)

• A Survey on Neural Speech Synthesis
• Non-Attentive Tacotron: Robust and Controllable Neural TTS Synthesis Including Unsupervised Duration Modeling
• Parallel Tacotron: Non-Autoregressive and Controllable TTS
• Review of end-to-end speech synthesis technology based on deep learning
• Hi-Fi Multi-Speaker English TTS Dataset
• Sprachsynthese -- State-of-the-Art in englischer und deutscher Sprache