AdaNet: Adaptive Structural Learning of Artificial Neural Networks
arXiv:1607.01097
Abstract
We present new algorithms for adaptively learning artificial neural networks. Our algorithms (AdaNet) adaptively learn both the structure of the network and its weights. They are based on a solid theoretical analysis, including data-dependent generalization guarantees that we prove and discuss in detail. We report the results of large-scale experiments with one of our algorithms on several binary classification tasks extracted from the CIFAR-10 dataset. The results demonstrate that our algorithm can automatically learn network structures with very competitive performance accuracies when compared with those achieved for neural networks found by standard approaches.
References in corpus (11)
- Neural Architecture Search with Reinforcement Learning
- Densely Connected Convolutional Networks
- Asynchronous Parallel Stochastic Gradient for Nonconvex Optimization
- Train faster, generalize better: Stability of stochastic gradient descent
- Provable Bounds for Learning Some Deep Representations
- The Power of Depth for Feedforward Neural Networks
- Toward Deeper Understanding of Neural Networks: The Power of Initialization and a Dual View on Expressivity
- Beating the Perils of Non-Convexity: Guaranteed Training of Neural Networks using Tensor Methods
- Architectural Complexity Measures of Recurrent Neural Networks
- Why are deep nets reversible: A simple theory, with implications for training
- Explorations on high dimensional landscapes
Cited by in corpus (18)
- Ensemble deep learning: A review
- Densely Connected Convolutional Networks
- Neural Architecture Search with Bayesian Optimisation and Optimal Transport
- Lifelong Learning with Dynamically Expandable Networks
- SpinalNet: Deep Neural Network with Gradual Input
- Lifelong Teacher-Student Network Learning
- Routing Networks and the Challenges of Modular and Compositional Computation
- Routing Networks: Adaptive Selection of Non-linear Functions for Multi-Task Learning
- Designing the Topology of Graph Neural Networks: A Novel Feature Fusion Perspective
- Learning Deep ResNet Blocks Sequentially using Boosting Theory
- Greedy Layerwise Learning Can Scale to ImageNet
- Evolutionary-Neural Hybrid Agents for Architecture Search
- A Framework for Searching for General Artificial Intelligence
- Learning to infer: RL-based search for DNN primitive selection on Heterogeneous Embedded Systems
- Convolutional Networks with Dense Connectivity
- Improving Neural Architecture Search Image Classifiers via Ensemble Learning
- Targeted Deep Learning: Framework, Methods, and Applications
- RSAC: Regularized Subspace Approximation Classifier for Lightweight Continuous Learning