AdaNet: Adaptive Structural Learning of Artificial Neural Networks

Abstract

We present new algorithms for adaptively learning artificial neural networks. Our algorithms (AdaNet) adaptively learn both the structure of the network and its weights. They are based on a solid theoretical analysis, including data-dependent generalization guarantees that we prove and discuss in detail. We report the results of large-scale experiments with one of our algorithms on several binary classification tasks extracted from the CIFAR-10 dataset. The results demonstrate that our algorithm can automatically learn network structures with very competitive performance accuracies when compared with those achieved for neural networks found by standard approaches.

References in corpus (2)

• Neural Architecture Search with Reinforcement Learning
• Explorations on high dimensional landscapes

Cited by in corpus (9)

• Lifelong Teacher-Student Network Learning
• Routing Networks and the Challenges of Modular and Compositional Computation
• Routing Networks: Adaptive Selection of Non-linear Functions for Multi-Task Learning
• Designing the Topology of Graph Neural Networks: A Novel Feature Fusion Perspective
• A Framework for Searching for General Artificial Intelligence
• Convolutional Networks with Dense Connectivity
• Improving Neural Architecture Search Image Classifiers via Ensemble Learning
• RSAC: Regularized Subspace Approximation Classifier for Lightweight Continuous Learning
• Targeted Deep Learning: Framework, Methods, and Applications