Fast Spectrogram Inversion using Multi-head Convolutional Neural Networks

Abstract

We propose the multi-head convolutional neural network (MCNN) architecture for waveform synthesis from spectrograms. Nonlinear interpolation in MCNN is employed with transposed convolution layers in parallel heads. MCNN achieves more than an order of magnitude higher compute intensity than commonly-used iterative algorithms like Griffin-Lim, yielding efficient utilization for modern multi-core processors, and very fast (more than 300x real-time) waveform synthesis. For training of MCNN, we use a large-scale speech recognition dataset and losses defined on waveforms that are related to perceptual audio quality. We demonstrate that MCNN constitutes a very promising approach for high-quality speech synthesis, without any iterative algorithms or autoregression in computations.

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• WaveNet: A Generative Model for Raw Audio
• A guide to convolution arithmetic for deep learning
• Deep Voice: Real-time Neural Text-to-Speech
• Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders
• Parallel WaveNet: Fast High-Fidelity Speech Synthesis
• Deep Voice 2: Multi-Speaker Neural Text-to-Speech
• Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions
• Neural Voice Cloning with a Few Samples
• Tacotron: Towards End-to-End Speech Synthesis
• Audio style transfer
• A Non-iterative Method for (Re)Construction of Phase from STFT Magnitude

Cited by in corpus (26)

• A Survey on Neural Speech Synthesis
• MelNet: A Generative Model for Audio in the Frequency Domain
• Jukebox: A Generative Model for Music
• DDSP: Differentiable Digital Signal Processing
• Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram
• Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
• Temporal Convolution for Real-time Keyword Spotting on Mobile Devices
• End-to-End Adversarial Text-to-Speech
• DiscreTalk: Text-to-Speech as a Machine Translation Problem
• Multi-band MelGAN: Faster Waveform Generation for High-Quality Text-to-Speech
• Probability density distillation with generative adversarial networks for high-quality parallel waveform generation
• Phonetic Posteriorgrams based Many-to-Many Singing Voice Conversion via Adversarial Training
• Learning the Proximity Operator in Unfolded ADMM for Phase Retrieval
• Towards achieving robust universal neural vocoding
• Assisted Sound Sample Generation with Musical Conditioning in Adversarial Auto-Encoders
• Parallel Synthesis for Autoregressive Speech Generation
• Catch-A-Waveform: Learning to Generate Audio from a Single Short Example
• Improved parallel WaveGAN vocoder with perceptually weighted spectrogram loss
• Universal Neural Vocoding with Parallel WaveNet
• Parallel waveform synthesis based on generative adversarial networks with voicing-aware conditional discriminators
• MelGlow: Efficient Waveform Generative Network Based on Location-Variable Convolution
• Hierarchical Timbre-Painting and Articulation Generation
• PPSpeech: Phrase based Parallel End-to-End TTS System
• Sines, Transient, Noise Neural Modeling of Piano Notes
• Generative Adversarial Network based Speaker Adaptation for High Fidelity WaveNet Vocoder
• Single Channel Speech Enhancement Using Temporal Convolutional Recurrent Neural Networks