Deep Long Audio Inpainting

Abstract

Long (> 200 ms) audio inpainting, to recover a long missing part in an audio segment, could be widely applied to audio editing tasks and transmission loss recovery. It is a very challenging problem due to the high dimensional, complex and non-correlated audio features. While deep learning models have made tremendous progress in image and video inpainting, audio inpainting did not attract much attention. In this work, we take a pioneering step, exploring the possibility of adapting deep learning frameworks from various domains inclusive of audio synthesis and image inpainting for audio inpainting. Also, as the first to systematically analyze factors affecting audio inpainting performance, we explore how factors ranging from mask size, receptive field and audio representation could affect the performance. We also set up a benchmark for long audio inpainting. The code will be available on GitHub upon accepted.

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Neural Architecture Search with Reinforcement Learning
• Conditional Image Generation with PixelCNN Decoders
• Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition
• EdgeConnect: Generative Image Inpainting with Adversarial Edge Learning
• Efficiently Trainable Text-to-Speech System Based on Deep Convolutional Networks with Guided Attention
• Learnable Gated Temporal Shift Module for Deep Video Inpainting

Cited by in corpus (5)

• A Comprehensive Survey on Deep Music Generation: Multi-level Representations, Algorithms, Evaluations, and Future Directions
• GACELA -- A generative adversarial context encoder for long audio inpainting
• Audio inpainting with generative adversarial network
• Learning to Maximize Speech Quality Directly Using MOS Prediction for Neural Text-to-Speech
• On Deep Speech Packet Loss Concealment: A Mini-Survey