Audio Spectrogram Representations for Processing with Convolutional Neural Networks

Abstract

One of the decisions that arise when designing a neural network for any application is how the data should be represented in order to be presented to, and possibly generated by, a neural network. For audio, the choice is less obvious than it seems to be for visual images, and a variety of representations have been used for different applications including the raw digitized sample stream, hand-crafted features, machine discovered features, MFCCs and variants that include deltas, and a variety of spectral representations. This paper reviews some of these representations and issues that arise, focusing particularly on spectrograms for generating audio using neural networks for style transfer.

Proceedings of the First International Conference on Deep Learning and Music, Anchorage, US, May, 2017 (arXiv:1706.08675v1 [cs.NE])

References in corpus (2)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• SoundNet: Learning Sound Representations from Unlabeled Video

Cited by in corpus (13)

• An Ensemble of Convolutional Neural Networks for Audio Classification
• Environmental Sound Classification on the Edge: A Pipeline for Deep Acoustic Networks on Extremely Resource-Constrained Devices
• A Comprehensive Survey on Deep Music Generation: Multi-level Representations, Algorithms, Evaluations, and Future Directions
• Music Genre Classification using Machine Learning Techniques
• Pruning vs XNOR-Net: A Comprehensive Study of Deep Learning for Audio Classification on Edge-devices
• Listening to Sounds of Silence for Speech Denoising
• When are Deep Networks really better than Decision Forests at small sample sizes, and how?
• Time Domain Neural Audio Style Transfer
• A Bird Song Detector for improving bird identification through Deep Learning: a case study from Doñana
• Improving automated segmentation of radio shows with audio embeddings
• nnAudio: An on-the-fly GPU Audio to Spectrogram Conversion Toolbox Using 1D Convolution Neural Networks
• Audio Spectrogram Factorization for Classification of Telephony Signals below the Auditory Threshold
• Perceiving Music Quality with GANs