N2N Learning: Network to Network Compression via Policy Gradient Reinforcement Learning

Abstract

While bigger and deeper neural network architectures continue to advance the state-of-the-art for many computer vision tasks, real-world adoption of these networks is impeded by hardware and speed constraints. Conventional model compression methods attempt to address this problem by modifying the architecture manually or using pre-defined heuristics. Since the space of all reduced architectures is very large, modifying the architecture of a deep neural network in this way is a difficult task. In this paper, we tackle this issue by introducing a principled method for learning reduced network architectures in a data-driven way using reinforcement learning. Our approach takes a larger `teacher' network as input and outputs a compressed `student' network derived from the `teacher' network. In the first stage of our method, a recurrent policy network aggressively removes layers from the large `teacher' model. In the second stage, another recurrent policy network carefully reduces the size of each remaining layer. The resulting network is then evaluated to obtain a reward -- a score based on the accuracy and compression of the network. Our approach uses this reward signal with policy gradients to train the policies to find a locally optimal student network. Our experiments show that we can achieve compression rates of more than 10x for models such as ResNet-34 while maintaining similar performance to the input `teacher' network. We also present a valuable transfer learning result which shows that policies which are pre-trained on smaller `teacher' networks can be used to rapidly speed up training on larger `teacher' networks.

References in corpus (10)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Distilling the Knowledge in a Neural Network
• Neural Architecture Search with Reinforcement Learning
• FitNets: Hints for Thin Deep Nets
• Do Deep Nets Really Need to be Deep?
• All you need is a good init
• Do Deep Convolutional Nets Really Need to be Deep and Convolutional?
• Diversity Networks: Neural Network Compression Using Determinantal Point Processes
• Structured Pruning of Deep Convolutional Neural Networks
• Self-informed neural network structure learning

Cited by in corpus (32)

• Knowledge Distillation: A Survey
• Neural Architecture Search: A Survey
• A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions
• Path-Level Network Transformation for Efficient Architecture Search
• Distilled Siamese Networks for Visual Tracking
• Once-for-All: Train One Network and Specialize it for Efficient Deployment
• EmBench: Quantifying Performance Variations of Deep Neural Networks across Modern Commodity Devices
• Cluster Pruning: An Efficient Filter Pruning Method for Edge AI Vision Applications
• PoPS: Policy Pruning and Shrinking for Deep Reinforcement Learning
• Group Sparsity: The Hinge Between Filter Pruning and Decomposition for Network Compression
• Knowledge Amalgamation for Object Detection with Transformers
• Cross-Resolution Face Recognition via Prior-Aided Face Hallucination and Residual Knowledge Distillation
• SS-Auto: A Single-Shot, Automatic Structured Weight Pruning Framework of DNNs with Ultra-High Efficiency
• CoCoPIE: Making Mobile AI Sweet As PIE --Compression-Compilation Co-Design Goes a Long Way
• An Embarrassingly Simple Approach for Knowledge Distillation
• Edge-Cloud Cooperation for DNN Inference via Reinforcement Learning and Supervised Learning
• Effective Training of Convolutional Neural Networks with Low-bitwidth Weights and Activations
• Data Efficient Stagewise Knowledge Distillation
• Towards Oracle Knowledge Distillation with Neural Architecture Search
• Resource Constrained Neural Network Architecture Search: Will a Submodularity Assumption Help?
• Efficient Neural Network Compression
• Data-Driven Compression of Convolutional Neural Networks
• Distilling Pixel-Wise Feature Similarities for Semantic Segmentation
• Compression and Localization in Reinforcement Learning for ATARI Games
• DECORE: Deep Compression with Reinforcement Learning
• Deep Reinforced Attention Learning for Quality-Aware Visual Recognition
• Mining the Weights Knowledge for Optimizing Neural Network Structures
• Parallel Blockwise Knowledge Distillation for Deep Neural Network Compression
• Learning to Prevent Leakage: Privacy-Preserving Inference in the Mobile Cloud
• Auto Deep Compression by Reinforcement Learning Based Actor-Critic Structure
• OICSR: Out-In-Channel Sparsity Regularization for Compact Deep Neural Networks
• Explore the Knowledge contained in Network Weights to Obtain Sparse Neural Networks