PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition

Abstract

Audio pattern recognition is an important research topic in the machine learning area, and includes several tasks such as audio tagging, acoustic scene classification, music classification, speech emotion classification and sound event detection. Recently, neural networks have been applied to tackle audio pattern recognition problems. However, previous systems are built on specific datasets with limited durations. Recently, in computer vision and natural language processing, systems pretrained on large-scale datasets have generalized well to several tasks. However, there is limited research on pretraining systems on large-scale datasets for audio pattern recognition. In this paper, we propose pretrained audio neural networks (PANNs) trained on the large-scale AudioSet dataset. These PANNs are transferred to other audio related tasks. We investigate the performance and computational complexity of PANNs modeled by a variety of convolutional neural networks. We propose an architecture called Wavegram-Logmel-CNN using both log-mel spectrogram and waveform as input feature. Our best PANN system achieves a state-of-the-art mean average precision (mAP) of 0.439 on AudioSet tagging, outperforming the best previous system of 0.392. We transfer PANNs to six audio pattern recognition tasks, and demonstrate state-of-the-art performance in several of those tasks. We have released the source code and pretrained models of PANNs: https://github.com/qiuqiangkong/audioset_tagging_cnn.

arXiv comment14 pages

References in corpus (9)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
• A multi-device dataset for urban acoustic scene classification
• General-purpose Tagging of Freesound Audio with AudioSet Labels: Task Description, Dataset, and Baseline
• Weakly Labelled AudioSet Tagging with Attention Neural Networks
• Integrating the Data Augmentation Scheme with Various Classifiers for Acoustic Scene Modeling
• musicnn: Pre-trained convolutional neural networks for music audio tagging
• Cross-task learning for audio tagging, sound event detection and spatial localization: DCASE 2019 baseline systems

Cited by in corpus (13)

• Self-Supervised MultiModal Versatile Networks
• Towards Learning a Universal Non-Semantic Representation of Speech
• Text-based depression detection on sparse data
• Addressing Missing Labels in Large-Scale Sound Event Recognition Using a Teacher-Student Framework With Loss Masking
• Acoustic Scene Classification with Spectrogram Processing Strategies
• Real-Time Idling Vehicles Detection using Combined Audio-Visual Deep Learning
• A General Network Architecture for Sound Event Localization and Detection Using Transfer Learning and Recurrent Neural Network
• Large-Scale MIDI-based Composer Classification
• CRNNs for Urban Sound Tagging with spatiotemporal context
• Distilling Audio-Visual Knowledge by Compositional Contrastive Learning
• TENET: A Time-reversal Enhancement Network for Noise-robust ASR
• Towards Learning Universal Audio Representations
• Sound Event Detection with Adaptive Frequency Selection