Graphite: Iterative Generative Modeling of Graphs

Abstract

Graphs are a fundamental abstraction for modeling relational data. However, graphs are discrete and combinatorial in nature, and learning representations suitable for machine learning tasks poses statistical and computational challenges. In this work, we propose Graphite, an algorithmic framework for unsupervised learning of representations over nodes in large graphs using deep latent variable generative models. Our model parameterizes variational autoencoders (VAE) with graph neural networks, and uses a novel iterative graph refinement strategy inspired by low-rank approximations for decoding. On a wide variety of synthetic and benchmark datasets, Graphite outperforms competing approaches for the tasks of density estimation, link prediction, and node classification. Finally, we derive a theoretical connection between message passing in graph neural networks and mean-field variational inference.

arXiv commentICML 2019

References in corpus (22)

• Scikit-learn: Machine Learning in Python
• Emergence of scaling in random networks
• The structure and function of complex networks
• Distributed Representations of Words and Phrases and their Compositionality
• Auto-Encoding Variational Bayes
• Inductive Representation Learning on Large Graphs
• Neural Message Passing for Quantum Chemistry
• Molecular Graph Convolutions: Moving Beyond Fingerprints
• Semi-Supervised Learning with Deep Generative Models
• Variational Graph Auto-Encoders
• Junction Tree Variational Autoencoder for Molecular Graph Generation
• Discriminative Embeddings of Latent Variable Models for Structured Data
• GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models
• Learning Deep Generative Models of Graphs
• Few-Shot Learning with Graph Neural Networks
• Revisiting Semi-Supervised Learning with Graph Embeddings
• NetGAN: Generating Graphs via Random Walks
• GraphGAN: Graph Representation Learning with Generative Adversarial Nets
• Structured Generative Models of Natural Source Code
• The Weisfeiler-Lehman Method and Graph Isomorphism Testing
• Stochastic Optimization of Sorting Networks via Continuous Relaxations
• Deep Generative Models for Relational Data with Side Information

Cited by in corpus (19)

• MolGAN: An implicit generative model for small molecular graphs
• Learning Discrete Structures for Graph Neural Networks
• Generative Models for Automatic Chemical Design
• Hierarchical Graph-to-Graph Translation for Molecules
• Permutation Invariant Graph Generation via Score-Based Generative Modeling
• Pointer Graph Networks
• Generating Tertiary Protein Structures via an Interpretative Variational Autoencoder
• Deep Graph Neural Networks with Shallow Subgraph Samplers
• Graph Deconvolutional Generation
• Generative Graph Convolutional Network for Growing Graphs
• Graph Generators: State of the Art and Open Challenges
• SCGG: A Deep Structure-Conditioned Graph Generative Model
• Graph-Aware Transformer: Is Attention All Graphs Need?
• EvoNet: A Neural Network for Predicting the Evolution of Dynamic Graphs
• Signed Network Modeling Based on Structural Balance Theory
• Neural Variational Inference For Estimating Uncertainty in Knowledge Graph Embeddings
• Livewired Neural Networks: Making Neurons That Fire Together Wire Together
• Graph-Preserving Grid Layout: A Simple Graph Drawing Method for Graph Classification using CNNs
• Bipartite Link Prediction based on Topological Features via 2-hop Path