SGAS: Sequential Greedy Architecture Search
arXiv:1912.00195
Abstract
Architecture design has become a crucial component of successful deep learning. Recent progress in automatic neural architecture search (NAS) shows a lot of promise. However, discovered architectures often fail to generalize in the final evaluation. Architectures with a higher validation accuracy during the search phase may perform worse in the evaluation. Aiming to alleviate this common issue, we introduce sequential greedy architecture search (SGAS), an efficient method for neural architecture search. By dividing the search procedure into sub-problems, SGAS chooses and prunes candidate operations in a greedy fashion. We apply SGAS to search architectures for Convolutional Neural Networks (CNN) and Graph Convolutional Networks (GCN). Extensive experiments show that SGAS is able to find state-of-the-art architectures for tasks such as image classification, point cloud classification and node classification in protein-protein interaction graphs with minimal computational cost. Please visit https://www.deepgcns.org/auto/sgas for more information about SGAS.
Accepted at CVPR'2020. Project website: https://www.deepgcns.org/auto/sgas
References in corpus (20)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- Semi-Supervised Classification with Graph Convolutional Networks
- Inductive Representation Learning on Large Graphs
- Neural Architecture Search with Reinforcement Learning
- Fast Graph Representation Learning with PyTorch Geometric
- AutoAugment: Learning Augmentation Policies from Data
- Efficient Neural Architecture Search via Parameter Sharing
- Predicting multicellular function through multi-layer tissue networks
- Neural Architecture Optimization
- PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search
- SMASH: One-Shot Model Architecture Search through HyperNetworks
- SNAS: Stochastic Neural Architecture Search
- DeepGCNs: Making GCNs Go as Deep as CNNs
- Convolutional Neural Fabrics
- Evaluating the Search Phase of Neural Architecture Search
- Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluation
- DeepGCNs: Can GCNs Go as Deep as CNNs?
- Bilevel Programming for Hyperparameter Optimization and Meta-Learning
- Connectivity Learning in Multi-Branch Networks