Improving the expressiveness of neural vocoding with non-affine Normalizing Flows

Abstract

This paper proposes a general enhancement to the Normalizing Flows (NF) used in neural vocoding. As a case study, we improve expressive speech vocoding with a revamped Parallel Wavenet (PW). Specifically, we propose to extend the affine transformation of PW to the more expressive invertible non-affine function. The greater expressiveness of the improved PW leads to better-perceived signal quality and naturalness in the waveform reconstruction and text-to-speech (TTS) tasks. We evaluate the model across different speaking styles on a multi-speaker, multi-lingual dataset. In the waveform reconstruction task, the proposed model closes the naturalness and signal quality gap from the original PW to recordings by $10\%$, and from other state-of-the-art neural vocoding systems by more than $60\%$. We also demonstrate improvements in objective metrics on the evaluation test set with L2 Spectral Distance and Cross-Entropy reduced by $3\%$ and $6\unicode{x2030}$ comparing to the affine PW. Furthermore, we extend the probability density distillation procedure proposed by the original PW paper, so that it works with any non-affine invertible and differentiable function.

Accepted to Interspeech 2021, 5 pages,3 figures

References in corpus (15)

• Distilling the Knowledge in a Neural Network
• WaveNet: A Generative Model for Raw Audio
• Normalizing Flows: An Introduction and Review of Current Methods
• HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis
• MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis
• Normalizing Flows for Probabilistic Modeling and Inference
• Generalization and Equilibrium in Generative Adversarial Nets (GANs)
• Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design
• MelNet: A Generative Model for Audio in the Frequency Domain
• Sum-of-Squares Polynomial Flow
• FloWaveNet : A Generative Flow for Raw Audio
• Neural Spline Flows
• Generalization in Generative Adversarial Networks: A Novel Perspective from Privacy Protection
• Cubic-Spline Flows
• WaveNODE: A Continuous Normalizing Flow for Speech Synthesis