A Semi-Supervised Assessor of Neural Architectures

Abstract

Neural architecture search (NAS) aims to automatically design deep neural networks of satisfactory performance. Wherein, architecture performance predictor is critical to efficiently value an intermediate neural architecture. But for the training of this predictor, a number of neural architectures and their corresponding real performance often have to be collected. In contrast with classical performance predictor optimized in a fully supervised way, this paper suggests a semi-supervised assessor of neural architectures. We employ an auto-encoder to discover meaningful representations of neural architectures. Taking each neural architecture as an individual instance in the search space, we construct a graph to capture their intrinsic similarities, where both labeled and unlabeled architectures are involved. A graph convolutional neural network is introduced to predict the performance of architectures based on the learned representations and their relation modeled by the graph. Extensive experimental results on the NAS-Benchmark-101 dataset demonstrated that our method is able to make a significant reduction on the required fully trained architectures for finding efficient architectures.

References in corpus (17)

• Adam: A Method for Stochastic Optimization
• Semi-Supervised Classification with Graph Convolutional Networks
• Neural Architecture Search with Reinforcement Learning
• Efficient Neural Architecture Search via Parameter Sharing
• Large-Scale Evolution of Image Classifiers
• Neural Architecture Optimization
• SNAS: Stochastic Neural Architecture Search
• NAS-Bench-101: Towards Reproducible Neural Architecture Search
• Random Search and Reproducibility for Neural Architecture Search
• Single Path One-Shot Neural Architecture Search with Uniform Sampling
• Hierarchical Representations for Efficient Architecture Search
• Freeze-Thaw Bayesian Optimization
• GhostNet: More Features from Cheap Operations
• Peephole: Predicting Network Performance Before Training
• NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search
• CARS: Continuous Evolution for Efficient Neural Architecture Search
• ReNAS:Relativistic Evaluation of Neural Architecture Search

Cited by in corpus (5)

• Model Rubik's Cube: Twisting Resolution, Depth and Width for TinyNets
• Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization Gap
• CATE: Computation-aware Neural Architecture Encoding with Transformers
• GENNAPE: Towards Generalized Neural Architecture Performance Estimators
• DC-NAS: Divide-and-Conquer Neural Architecture Search