Pruning vs XNOR-Net: A Comprehensive Study of Deep Learning for Audio Classification on Edge-devices

Abstract

Deep learning has celebrated resounding successes in many application areas of relevance to the Internet of Things (IoT), such as computer vision and machine listening. These technologies must ultimately be brought directly to the edge to fully harness the power of deep learning for the IoT. The obvious challenge is that deep learning techniques can only be implemented on strictly resource-constrained edge devices if the models are radically downsized. This task relies on different model compression techniques, such as network pruning, quantization, and the recent advancement of XNOR-Net. This study examines the suitability of these techniques for audio classification on microcontrollers. We present an application of XNOR-Net for end-to-end raw audio classification and a comprehensive empirical study comparing this approach with pruning-and-quantization methods. We show that raw audio classification with XNOR yields comparable performance to regular full precision networks for small numbers of classes while reducing memory requirements 32-fold and computation requirements 58-fold. However, as the number of classes increases significantly, performance degrades, and pruning-and-quantization based compression techniques take over as the preferred technique being able to satisfy the same space constraints but requiring approximately 8x more computation. We show that these insights are consistent between raw audio classification and image classification using standard benchmark sets. To the best of our knowledge, this is the first study to apply XNOR to end-to-end audio classification and evaluate it in the context of alternative techniques. All codes are publicly available on GitHub.

arXiv comment13 pages

References in corpus (13)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Distilling the Knowledge in a Neural Network
• Pruning Convolutional Neural Networks for Resource Efficient Inference
• Learning from Between-class Examples for Deep Sound Recognition
• Audio Spectrogram Representations for Processing with Convolutional Neural Networks
• Deep Convolutional Neural Networks and Data Augmentation for Acoustic Event Detection
• A Multi-view CNN-based Acoustic Classification System for Automatic Animal Species Identification
• Sound Event Detection with Binary Neural Networks on Tightly Power-Constrained IoT Devices
• AST: Audio Spectrogram Transformer
• Neural Style Transfer for Audio Spectograms
• AclNet: efficient end-to-end audio classification CNN
• A Sequential Self Teaching Approach for Improving Generalization in Sound Event Recognition
• Efficient Convolutional Neural Network For Audio Event Detection

Cited by in corpus (3)

• Few-shot Class-incremental Audio Classification Using Dynamically Expanded Classifier with Self-attention Modified Prototypes
• XnODR and XnIDR: Two Accurate and Fast Fully Connected Layers For Convolutional Neural Networks
• Deep Active Audio Feature Learning in Resource-Constrained Environments