Low Bit-Rate Speech Coding with VQ-VAE and a WaveNet Decoder
arXiv:1910.06464 · doi:10.1109/ICASSP.2019.8683277
Abstract
In order to efficiently transmit and store speech signals, speech codecs create a minimally redundant representation of the input signal which is then decoded at the receiver with the best possible perceptual quality. In this work we demonstrate that a neural network architecture based on VQ-VAE with a WaveNet decoder can be used to perform very low bit-rate speech coding with high reconstruction quality. A prosody-transparent and speaker-independent model trained on the LibriSpeech corpus coding audio at 1.6 kbps exhibits perceptual quality which is around halfway between the MELP codec at 2.4 kbps and AMR-WB codec at 23.05 kbps. In addition, when training on high-quality recorded speech with the test speaker included in the training set, a model coding speech at 1.6 kbps produces output of similar perceptual quality to that generated by AMR-WB at 23.05 kbps.
ICASSP 2019
References in corpus (10)
- Auto-Encoding Variational Bayes
- WaveNet: A Generative Model for Raw Audio
- Neural Discrete Representation Learning
- End-to-end Optimized Image Compression
- Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis
- Lossy Image Compression with Compressive Autoencoders
- Real-Time Adaptive Image Compression
- Sample Efficient Adaptive Text-to-Speech
- Speex: A Free Codec For Free Speech
- Composition of Deep and Spiking Neural Networks for Very Low Bit Rate Speech Coding
Cited by in corpus (21)
- Dynamical Variational Autoencoders: A Comprehensive Review
- AudioDec: An Open-source Streaming High-fidelity Neural Audio Codec
- SemantiCodec: An Ultra Low Bitrate Semantic Audio Codec for General Sound
- Psychoacoustic Calibration of Loss Functions for Efficient End-to-End Neural Audio Coding
- Scalable and Efficient Neural Speech Coding: A Hybrid Design
- Latent-Domain Predictive Neural Speech Coding
- Cascaded Cross-Module Residual Learning towards Lightweight End-to-End Speech Coding
- Neural Speech and Audio Coding: Modern AI Technology Meets Traditional Codecs
- Recurrent autoencoder with sequence-aware encoding
- Deep Residual Mixture Models
- Universal Speech Token Learning via Low-Bitrate Neural Codec and Pretrained Representations
- Personalized Neural Speech Codec
- Improved Prosody from Learned F0 Codebook Representations for VQ-VAE Speech Waveform Reconstruction
- Preference-based training framework for automatic speech quality assessment using deep neural network
- I'm Sorry for Your Loss: Spectrally-Based Audio Distances Are Bad at Pitch
- Perceptual Audio Coding: A 40-Year Historical Perspective
- Feedback Recurrent AutoEncoder
- Low Bit-Rate Wideband Speech Coding: A Deep Generative Model based Approach
- Make Some Noise: Towards LLM audio reasoning and generation using sound tokens
- Efficient And Scalable Neural Residual Waveform Coding With Collaborative Quantization
- Unified Signal Compression Using a GAN with Iterative Latent Representation Optimization