Rethinking CNN Models for Audio Classification

Abstract

In this paper, we show that ImageNet-Pretrained standard deep CNN models can be used as strong baseline networks for audio classification. Even though there is a significant difference between audio Spectrogram and standard ImageNet image samples, transfer learning assumptions still hold firmly. To understand what enables the ImageNet pretrained models to learn useful audio representations, we systematically study how much of pretrained weights is useful for learning spectrograms. We show (1) that for a given standard model using pretrained weights is better than using randomly initialized weights (2) qualitative results of what the CNNs learn from the spectrograms by visualizing the gradients. Besides, we show that even though we use the pretrained model weights for initialization, there is variance in performance in various output runs of the same model. This variance in performance is due to the random initialization of linear classification layer and random mini-batch orderings in multiple runs. This brings significant diversity to build stronger ensemble models with an overall improvement in accuracy. An ensemble of ImageNet pretrained DenseNet achieves 92.89% validation accuracy on the ESC-50 dataset and 87.42% validation accuracy on the UrbanSound8K dataset which is the current state-of-the-art on both of these datasets.

arXiv comment8 pages, 3 figures

References in corpus (19)

• UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
• WaveNet: A Generative Model for Raw Audio
• How transferable are features in deep neural networks?
• Axiomatic Attribution for Deep Networks
• Tune: A Research Platform for Distributed Model Selection and Training
• TernausNet: U-Net with VGG11 Encoder Pre-Trained on ImageNet for Image Segmentation
• A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music
• SoundNet: Learning Sound Representations from Unlabeled Video
• Automatic tagging using deep convolutional neural networks
• Transfer learning for music classification and regression tasks
• Comparison of Time-Frequency Representations for Environmental Sound Classification using Convolutional Neural Networks
• Sample-level Deep Convolutional Neural Networks for Music Auto-tagging Using Raw Waveforms
• Learning Multiscale Features Directly From Waveforms
• CNN Architectures for Large-Scale Audio Classification
• Multi-stream Network With Temporal Attention For Environmental Sound Classification
• Attention based Convolutional Recurrent Neural Network for Environmental Sound Classification
• Convolutional Neural Network Achieves Human-level Accuracy in Music Genre Classification
• Environmental Sound Classification with Parallel Temporal-spectral Attention
• Multi-Temporal Resolution Convolutional Neural Networks for Acoustic Scene Classification

Cited by in corpus (14)

• PSLA: Improving Audio Tagging with Pretraining, Sampling, Labeling, and Aggregation
• AST: Audio Spectrogram Transformer
• Dual Bayesian ResNet: A Deep Learning Approach to Heart Murmur Detection
• SoundCLR: Contrastive Learning of Representations For Improved Environmental Sound Classification
• CCMusic: An Open and Diverse Database for Chinese Music Information Retrieval Research
• Complex-valued Federated Learning with Differential Privacy and MRI Applications
• Detection of Audio-Video Synchronization Errors Via Event Detection
• Example-Based Framework for Perceptually Guided Audio Texture Generation
• A Visual Domain Transfer Learning Approach for Heartbeat Sound Classification
• Detection and classification of vocal productions in large scale audio recordings
• NeuroView: Explainable Deep Network Decision Making
• Deep Learning for Enhanced Scratch Input
• HumBugDB: A Large-scale Acoustic Mosquito Dataset
• A Methodology for Exploring Deep Convolutional Features in Relation to Hand-Crafted Features with an Application to Music Audio Modeling