Bandwidth Extension on Raw Audio via Generative Adversarial Networks

Abstract

Neural network-based methods have recently demonstrated state-of-the-art results on image synthesis and super-resolution tasks, in particular by using variants of generative adversarial networks (GANs) with supervised feature losses. Nevertheless, previous feature loss formulations rely on the availability of large auxiliary classifier networks, and labeled datasets that enable such classifiers to be trained. Furthermore, there has been comparatively little work to explore the applicability of GAN-based methods to domains other than images and video. In this work we explore a GAN-based method for audio processing, and develop a convolutional neural network architecture to perform audio super-resolution. In addition to several new architectural building blocks for audio processing, a key component of our approach is the use of an autoencoder-based loss that enables training in the GAN framework, with feature losses derived from unlabeled data. We explore the impact of our architectural choices, and demonstrate significant improvements over previous works in terms of both objective and perceptual quality.

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Auto-Encoding Variational Bayes
• Conditional Generative Adversarial Nets
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• SoundNet: Learning Sound Representations from Unlabeled Video

Cited by in corpus (6)

• NU-Wave: A Diffusion Probabilistic Model for Neural Audio Upsampling
• NU-Wave 2: A General Neural Audio Upsampling Model for Various Sampling Rates
• On Filter Generalization for Music Bandwidth Extension Using Deep Neural Networks
• EBEN: Extreme bandwidth extension network applied to speech signals captured with noise-resilient body-conduction microphones
• Real-time Speech Frequency Bandwidth Extension
• An investigation of pre-upsampling generative modelling and Generative Adversarial Networks in audio super resolution