D-VAE: A Variational Autoencoder for Directed Acyclic Graphs

Abstract

Graph structured data are abundant in the real world. Among different graph types, directed acyclic graphs (DAGs) are of particular interest to machine learning researchers, as many machine learning models are realized as computations on DAGs, including neural networks and Bayesian networks. In this paper, we study deep generative models for DAGs, and propose a novel DAG variational autoencoder (D-VAE). To encode DAGs into the latent space, we leverage graph neural networks. We propose an asynchronous message passing scheme that allows encoding the computations on DAGs, rather than using existing simultaneous message passing schemes to encode local graph structures. We demonstrate the effectiveness of our proposed DVAE through two tasks: neural architecture search and Bayesian network structure learning. Experiments show that our model not only generates novel and valid DAGs, but also produces a smooth latent space that facilitates searching for DAGs with better performance through Bayesian optimization.

Accepted by 33rd Conference on Neural Information Processing Systems (NeurIPS 2019)

References in corpus (32)

• Adam: A Method for Stochastic Optimization
• Sequence to Sequence Learning with Neural Networks
• Semi-Supervised Classification with Graph Convolutional Networks
• Inductive Representation Learning on Large Graphs
• Neural Architecture Search with Reinforcement Learning
• Neural Message Passing for Quantum Chemistry
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Improved Regularization of Convolutional Neural Networks with Cutout
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Junction Tree Variational Autoencoder for Molecular Graph Generation
• Efficient Neural Architecture Search via Parameter Sharing
• GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models
• Neural Architecture Optimization
• MolGAN: An implicit generative model for small molecular graphs
• Learning Deep Generative Models of Graphs
• Gated Graph Sequence Neural Networks
• How Powerful are Graph Neural Networks?
• Constrained Graph Variational Autoencoders for Molecule Design
• Syntax-Directed Variational Autoencoder for Structured Data
• Hierarchical Representations for Efficient Architecture Search
• DAGs with NO TEARS: Continuous Optimization for Structure Learning
• GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution
• NetGAN: Generating Graphs via Random Walks
• Simple And Efficient Architecture Search for Convolutional Neural Networks
• DAG-GNN: DAG Structure Learning with Graph Neural Networks
• Finding Optimal Bayesian Networks
• TerpreT: A Probabilistic Programming Language for Program Induction
• Constrained Generation of Semantically Valid Graphs via Regularizing Variational Autoencoders
• Probabilistic Matrix Factorization for Automated Machine Learning
• Inductive Matrix Completion Based on Graph Neural Networks
• Quotient Normalized Maximum Likelihood Criterion for Learning Bayesian Network Structures
• Cluster Variational Approximations for Structure Learning of Continuous-Time Bayesian Networks from Incomplete Data

Cited by in corpus (8)

• Neural Predictor based Quantum Architecture Search
• Causality-based CTR Prediction using Graph Neural Networks
• Towards Efficient Local Causal Structure Learning
• CODEBench: A Neural Architecture and Hardware Accelerator Co-Design Framework
• A Variational-Sequential Graph Autoencoder for Neural Architecture Performance Prediction
• Rapid Neural Architecture Search by Learning to Generate Graphs from Datasets
• Methods for Recovering Conditional Independence Graphs: A Survey
• Scalable Bayesian Optimization with Sparse Gaussian Process Models