Limits of End-to-End Learning

Abstract

End-to-end learning refers to training a possibly complex learning system by applying gradient-based learning to the system as a whole. End-to-end learning system is specifically designed so that all modules are differentiable. In effect, not only a central learning machine, but also all "peripheral" modules like representation learning and memory formation are covered by a holistic learning process. The power of end-to-end learning has been demonstrated on many tasks, like playing a whole array of Atari video games with a single architecture. While pushing for solutions to more challenging tasks, network architectures keep growing more and more complex. In this paper we ask the question whether and to what extent end-to-end learning is a future-proof technique in the sense of scaling to complex and diverse data processing architectures. We point out potential inefficiencies, and we argue in particular that end-to-end learning does not make optimal use of the modular design of present neural networks. Our surprisingly simple experiments demonstrate these inefficiencies, up to the complete breakdown of learning.

References in corpus (4)

• Deep Learning in Neural Networks: An Overview
• ADADELTA: An Adaptive Learning Rate Method
• Natural Language Processing (almost) from Scratch
• Learning to Navigate in Complex Environments

Cited by in corpus (15)

• Automated Evolutionary Approach for the Design of Composite Machine Learning Pipelines
• A Review of Hidden Markov Models and Recurrent Neural Networks for Event Detection and Localization in Biomedical Signals
• The Neural Network Approach to Inverse Problems in Differential Equations
• Exploring Graph Neural Networks for Stock Market Predictions with Rolling Window Analysis
• Deep Reinforcement Learning for Contact-Rich Skills Using Compliant Movement Primitives
• Enhanced Behavioral Cloning Based self-driving Car Using Transfer Learning
• Potential Field: Interpretable and Unified Representation for Trajectory Prediction
• Dropout Prediction over Weeks in MOOCs by Learning Representations of Clicks and Videos
• Automating Vehicles by Deep Reinforcement Learning using Task Separation with Hill Climbing
• Multipurpose Intelligent Process Automation via Conversational Assistant
• Abductive Knowledge Induction From Raw Data
• Active Learning for Automated Visual Inspection of Manufactured Products
• RefSum: Refactoring Neural Summarization
• A Generative Neural Network Framework for Automated Software Testing
• Design of Complex Experiments Using Mixed Integer Linear Programming