A Variational-Sequential Graph Autoencoder for Neural Architecture Performance Prediction

Abstract

In computer vision research, the process of automating architecture engineering, Neural Architecture Search (NAS), has gained substantial interest. In the past, NAS was hardly accessible to researchers without access to large-scale compute systems, due to very long compute times for the recurrent search and evaluation of new candidate architectures. The NAS-Bench-101 dataset facilitates a paradigm change towards classical methods such as supervised learning to evaluate neural architectures. In this paper, we propose a graph encoder built upon Graph Neural Networks (GNN). We demonstrate the effectiveness of the proposed encoder on NAS performance prediction for seen architecture types as well an unseen ones (i.e., zero shot prediction). We also provide a new variational-sequential graph autoencoder (VS-GAE) based on the proposed graph encoder. The VS-GAE is specialized on encoding and decoding graphs of varying length utilizing GNNs. Experiments on different sampling methods show that the embedding space learned by our VS-GAE increases the stability on the accuracy prediction task.

References in corpus (17)

• Semi-Supervised Classification with Graph Convolutional Networks
• Neural Architecture Search with Reinforcement Learning
• Neural Message Passing for Quantum Chemistry
• Geometric deep learning: going beyond Euclidean data
• Spectral Networks and Locally Connected Networks on Graphs
• Neural Architecture Search: A Survey
• Deep Convolutional Networks on Graph-Structured Data
• Fast Graph Representation Learning with PyTorch Geometric
• Variational Graph Auto-Encoders
• Junction Tree Variational Autoencoder for Molecular Graph Generation
• Large-Scale Evolution of Image Classifiers
• GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models
• Neural Architecture Optimization
• Learning Deep Generative Models of Graphs
• NAS-Bench-101: Towards Reproducible Neural Architecture Search
• Single Path One-Shot Neural Architecture Search with Uniform Sampling
• D-VAE: A Variational Autoencoder for Directed Acyclic Graphs

Cited by in corpus (4)

• Neural Predictor based Quantum Architecture Search
• Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization Gap
• Neural Architecture Optimization with Graph VAE
• Fitting the Search Space of Weight-sharing NAS with Graph Convolutional Networks