CLAR: Contrastive Learning of Auditory Representations
arXiv:2010.09542
Abstract
Learning rich visual representations using contrastive self-supervised learning has been extremely successful. However, it is still a major question whether we could use a similar approach to learn superior auditory representations. In this paper, we expand on prior work (SimCLR) to learn better auditory representations. We (1) introduce various data augmentations suitable for auditory data and evaluate their impact on predictive performance, (2) show that training with time-frequency audio features substantially improves the quality of the learned representations compared to raw signals, and (3) demonstrate that training with both supervised and contrastive losses simultaneously improves the learned representations compared to self-supervised pre-training followed by supervised fine-tuning. We illustrate that by combining all these methods and with substantially less labeled data, our framework (CLAR) achieves significant improvement on prediction performance compared to supervised approach. Moreover, compared to self-supervised approach, our framework converges faster with significantly better representations.
References in corpus (13)
- A Simple Framework for Contrastive Learning of Visual Representations
- SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
- Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
- SGDR: Stochastic Gradient Descent with Warm Restarts
- Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition
- Data-Efficient Image Recognition with Contrastive Predictive Coding
- An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
- Large Batch Training of Convolutional Networks
- Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders
- GANSynth: Adversarial Neural Audio Synthesis
- Learning from Between-class Examples for Deep Sound Recognition
- Spatiotemporal Contrastive Video Representation Learning
- Revisiting Self-Supervised Visual Representation Learning