Why should we add early exits to neural networks?

Abstract

Deep neural networks are generally designed as a stack of differentiable layers, in which a prediction is obtained only after running the full stack. Recently, some contributions have proposed techniques to endow the networks with early exits, allowing to obtain predictions at intermediate points of the stack. These multi-output networks have a number of advantages, including: (i) significant reductions of the inference time, (ii) reduced tendency to overfitting and vanishing gradients, and (iii) capability of being distributed over multi-tier computation platforms. In addition, they connect to the wider themes of biological plausibility and layered cognitive reasoning. In this paper, we provide a comprehensive introduction to this family of neural networks, by describing in a unified fashion the way these architectures can be designed, trained, and actually deployed in time-constrained scenarios. We also describe in-depth their application scenarios in 5G and Fog computing environments, as long as some of the open research questions connected to them.

arXiv commentPublished in Cognitive Computation

References in corpus (7)

• Mobile Edge Computing: A Survey on Architecture and Computation Offloading
• Deep Visual Attention Prediction
• Neural Ordinary Differential Equations
• Learning Representations for Neural Network-Based Classification Using the Information Bottleneck Principle
• Training Neural Networks with Local Error Signals
• EcoMobiFog -- Design and Dynamic Optimization of a 5G Mobile-Fog-Cloud Multi-Tier Ecosystem for the Real-Time Distributed Execution of Stream Applications
• A Framework for Parallel and Distributed Training of Neural Networks

Cited by in corpus (9)

• Conditional computation in neural networks: principles and research trends
• Dynamic nsNet2: Efficient Deep Noise Suppression with Early Exiting
• StreamTinyNet: video streaming analysis with spatial-temporal TinyML
• Multi-Exit Kolmogorov-Arnold Networks: enhancing accuracy and parsimony
• Optimizing Attention and Cognitive Control Costs Using Temporally-Layered Architectures
• On-Sensor Convolutional Neural Networks with Early-Exits
• Adaptive Slimming for Scalable and Efficient Speech Enhancement
• Learning Task-preferred Inference Routes for Gradient De-conflict in Multi-output DNNs
• Training Multi-Layer Binary Neural Networks With Local Binary Error Signals