A Generic Graph-based Neural Architecture Encoding Scheme for Predictor-based NAS

Abstract

This work proposes a novel Graph-based neural ArchiTecture Encoding Scheme, a.k.a. GATES, to improve the predictor-based neural architecture search. Specifically, different from existing graph-based schemes, GATES models the operations as the transformation of the propagating information, which mimics the actual data processing of neural architecture. GATES is a more reasonable modeling of the neural architectures, and can encode architectures from both the "operation on node" and "operation on edge" cell search spaces consistently. Experimental results on various search spaces confirm GATES's effectiveness in improving the performance predictor. Furthermore, equipped with the improved performance predictor, the sample efficiency of the predictor-based neural architecture search (NAS) flow is boosted. Codes are available at https://github.com/walkerning/aw_nas.

arXiv comment14 pages main text; 10 pages appendix

References in corpus (15)

• Adam: A Method for Stochastic Optimization
• Semi-Supervised Classification with Graph Convolutional Networks
• Neural Architecture Search with Reinforcement Learning
• Efficient Neural Architecture Search via Parameter Sharing
• Neural Architecture Optimization
• Hierarchical Representations for Efficient Architecture Search
• Evaluating the Search Phase of Neural Architecture Search
• DeepArchitect: Automatically Designing and Training Deep Architectures
• Graph HyperNetworks for Neural Architecture Search
• NAT: Neural Architecture Transformer for Accurate and Compact Architectures
• Bridging the Gap between Sample-based and One-shot Neural Architecture Search with BONAS
• AlphaX: eXploring Neural Architectures with Deep Neural Networks and Monte Carlo Tree Search
• Breaking the Curse of Space Explosion: Towards Efficient NAS with Curriculum Search
• ReNAS:Relativistic Evaluation of Neural Architecture Search
• Neural Architecture Search using Deep Neural Networks and Monte Carlo Tree Search

Cited by in corpus (10)

• Neural Predictor based Quantum Architecture Search
• How Powerful are Performance Predictors in Neural Architecture Search?
• Stronger NAS with Weaker Predictors
• NASGEM: Neural Architecture Search via Graph Embedding Method
• Fitting the Search Space of Weight-sharing NAS with Graph Convolutional Networks
• Self-supervised Representation Learning for Evolutionary Neural Architecture Search
• FLASH: Fast Neural Architecture Search with Hardware Optimization
• Efficient Sampling for Predictor-Based Neural Architecture Search
• Pretraining Neural Architecture Search Controllers with Locality-based Self-Supervised Learning
• Cascade Bagging for Accuracy Prediction with Few Training Samples