OverFlow: Putting flows on top of neural transducers for better TTS
arXiv:2211.06892 · doi:10.21437/Interspeech.2023-1996
Abstract
Neural HMMs are a type of neural transducer recently proposed for sequence-to-sequence modelling in text-to-speech. They combine the best features of classic statistical speech synthesis and modern neural TTS, requiring less data and fewer training updates, and are less prone to gibberish output caused by neural attention failures. In this paper, we combine neural HMM TTS with normalising flows for describing the highly non-Gaussian distribution of speech acoustics. The result is a powerful, fully probabilistic model of durations and acoustics that can be trained using exact maximum likelihood. Experiments show that a system based on our proposal needs fewer updates than comparable methods to produce accurate pronunciations and a subjective speech quality close to natural speech. Please see https://shivammehta25.github.io/OverFlow/ for audio examples and code.
5 pages, 2 figures. Accepted for publication at Interspeech 2023
References in corpus (6)
- Robust Speech Recognition via Large-Scale Weak Supervision
- Variational Diffusion Models
- Flowtron: an Autoregressive Flow-based Generative Network for Text-to-Speech Synthesis
- Prosody-controllable spontaneous TTS with neural HMMs
- Neural HMMs are all you need (for high-quality attention-free TTS)
- Normalizing Flow based Hidden Markov Models for Classification of Speech Phones with Explainability