Big-Little Net: An Efficient Multi-Scale Feature Representation for Visual and Speech Recognition

Abstract

In this paper, we propose a novel Convolutional Neural Network (CNN) architecture for learning multi-scale feature representations with good tradeoffs between speed and accuracy. This is achieved by using a multi-branch network, which has different computational complexity at different branches. Through frequent merging of features from branches at distinct scales, our model obtains multi-scale features while using less computation. The proposed approach demonstrates improvement of model efficiency and performance on both object recognition and speech recognition tasks,using popular architectures including ResNet and ResNeXt. For object recognition, our approach reduces computation by 33% on object recognition while improving accuracy with 0.9%. Furthermore, our model surpasses state-of-the-art CNN acceleration approaches by a large margin in accuracy and FLOPs reduction. On the task of speech recognition, our proposed multi-scale CNNs save 30% FLOPs with slightly better word error rates, showing good generalization across domains. The codes are available at https://github.com/IBM/BigLittleNet

arXiv commentgit repo: https://github.com/IBM/BigLittleNet

References in corpus (12)

• Distilling the Knowledge in a Neural Network
• Show and Tell: Lessons learned from the 2015 MSCOCO Image Captioning Challenge
• Binarized Neural Networks
• Ternary Weight Networks
• Compressing Neural Networks with the Hashing Trick
• Achieving Human Parity in Conversational Speech Recognition
• Multi-Scale Dense Networks for Resource Efficient Image Classification
• Convolutional Neural Fabrics
• Snapshot Ensembles: Train 1, get M for free
• Structured Transforms for Small-Footprint Deep Learning
• Dense Prediction on Sequences with Time-Dilated Convolutions for Speech Recognition
• Dual Path Networks

Cited by in corpus (9)

• Res2Net: A New Multi-scale Backbone Architecture
• Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution
• Hybrid Residual Attention Network for Single Image Super Resolution
• QU-BraTS: MICCAI BraTS 2020 Challenge on Quantifying Uncertainty in Brain Tumor Segmentation - Analysis of Ranking Scores and Benchmarking Results
• Compounding the Performance Improvements of Assembled Techniques in a Convolutional Neural Network
• AZTR: Aerial Video Action Recognition with Auto Zoom and Temporal Reasoning
• Busy-Quiet Video Disentangling for Video Classification
• Improved Res2Net model for Person re-identification
• MOOD: Multi-level Out-of-distribution Detection