ProjectionNet: Learning Efficient On-Device Deep Networks Using Neural Projections

Abstract

Deep neural networks have become ubiquitous for applications related to visual recognition and language understanding tasks. However, it is often prohibitive to use typical neural networks on devices like mobile phones or smart watches since the model sizes are huge and cannot fit in the limited memory available on such devices. While these devices could make use of machine learning models running on high-performance data centers with CPUs or GPUs, this is not feasible for many applications because data can be privacy sensitive and inference needs to be performed directly "on" device. We introduce a new architecture for training compact neural networks using a joint optimization framework. At its core lies a novel objective that jointly trains using two different types of networks--a full trainer neural network (using existing architectures like Feed-forward NNs or LSTM RNNs) combined with a simpler "projection" network that leverages random projections to transform inputs or intermediate representations into bits. The simpler network encodes lightweight and efficient-to-compute operations in bit space with a low memory footprint. The two networks are trained jointly using backpropagation, where the projection network learns from the full network similar to apprenticeship learning. Once trained, the smaller network can be used directly for inference at low memory and computation cost. We demonstrate the effectiveness of the new approach at significantly shrinking the memory requirements of different types of neural networks while preserving good accuracy on visual recognition and text classification tasks. We also study the question "how many neural bits are required to solve a given task?" using the new framework and show empirical results contrasting model predictive capacity (in bits) versus accuracy on several datasets.

References in corpus (8)

• Neural Machine Translation by Jointly Learning to Align and Translate
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• Federated Optimization: Distributed Machine Learning for On-Device Intelligence
• Compressing Deep Convolutional Networks using Vector Quantization
• Predicting Parameters in Deep Learning
• Entropy-based Pruning of Backoff Language Models
• Smart Reply: Automated Response Suggestion for Email
• Neural Graph Machines: Learning Neural Networks Using Graphs

Cited by in corpus (17)

• Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better
• FermiNets: Learning generative machines to generate efficient neural networks via generative synthesis
• On-Device Machine Learning: An Algorithms and Learning Theory Perspective
• EdgeNet: Balancing Accuracy and Performance for Edge-based Convolutional Neural Network Object Detectors
• SEFR: A Fast Linear-Time Classifier for Ultra-Low Power Devices
• AttendNets: Tiny Deep Image Recognition Neural Networks for the Edge via Visual Attention Condensers
• PRGFlow: Benchmarking SWAP-Aware Unified Deep Visual Inertial Odometry
• Learning Compact Neural Networks Using Ordinary Differential Equations as Activation Functions
• Adaptive Selection of Deep Learning Models on Embedded Systems
• A Neural Network-Based On-device Learning Anomaly Detector for Edge Devices
• EdgeSegNet: A Compact Network for Semantic Segmentation
• AttendSeg: A Tiny Attention Condenser Neural Network for Semantic Segmentation on the Edge
• Understanding Chat Messages for Sticker Recommendation in Messaging Apps
• Signature-Graph Networks
• DiverseNet: When One Right Answer is not Enough
• Adversarially Robust and Explainable Model Compression with On-Device Personalization for Text Classification
• Tiny Neural Models for Seq2Seq