Machine Learning on Graphs: A Model and Comprehensive Taxonomy

Abstract

There has been a surge of recent interest in learning representations for graph-structured data. Graph representation learning methods have generally fallen into three main categories, based on the availability of labeled data. The first, network embedding (such as shallow graph embedding or graph auto-encoders), focuses on learning unsupervised representations of relational structure. The second, graph regularized neural networks, leverages graphs to augment neural network losses with a regularization objective for semi-supervised learning. The third, graph neural networks, aims to learn differentiable functions over discrete topologies with arbitrary structure. However, despite the popularity of these areas there has been surprisingly little work on unifying the three paradigms. Here, we aim to bridge the gap between graph neural networks, network embedding and graph regularization models. We propose a comprehensive taxonomy of representation learning methods for graph-structured data, aiming to unify several disparate bodies of work. Specifically, we propose a Graph Encoder Decoder Model (GRAPHEDM), which generalizes popular algorithms for semi-supervised learning on graphs (e.g. GraphSage, Graph Convolutional Networks, Graph Attention Networks), and unsupervised learning of graph representations (e.g. DeepWalk, node2vec, etc) into a single consistent approach. To illustrate the generality of this approach, we fit over thirty existing methods into this framework. We believe that this unifying view both provides a solid foundation for understanding the intuition behind these methods, and enables future research in the area.

References in corpus (47)

• Distributed Representations of Words and Phrases and their Compositionality
• Semi-Supervised Classification with Graph Convolutional Networks
• Neural Message Passing for Quantum Chemistry
• Geometric deep learning: going beyond Euclidean data
• Graph Embedding Techniques, Applications, and Performance: A Survey
• Graph Neural Networks: A Review of Methods and Applications
• Deep Convolutional Networks on Graph-Structured Data
• Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks
• Fast Graph Representation Learning with PyTorch Geometric
• Graph Convolutional Matrix Completion
• Learning Combinatorial Optimization Algorithms over Graphs
• Variational Graph Auto-Encoders
• Junction Tree Variational Autoencoder for Molecular Graph Generation
• Deep learning for molecular design - a review of the state of the art
• Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networks
• Pitfalls of Graph Neural Network Evaluation
• Interaction Networks for Learning about Objects, Relations and Physics
• Open Graph Benchmark: Datasets for Machine Learning on Graphs
• GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models
• MolGAN: An implicit generative model for small molecular graphs
• Learning Deep Generative Models of Graphs
• Few-Shot Learning with Graph Neural Networks
• Constrained Graph Variational Autoencoders for Molecule Design
• PyTorch-BigGraph: A Large-scale Graph Embedding System
• Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them
• Benchmarking Graph Neural Networks
• Machine learning meets network science: dimensionality reduction for fast and efficient embedding of networks in the hyperbolic space
• Learning a SAT Solver from Single-Bit Supervision
• Network Mapping by Replaying Hyperbolic Growth
• Poincaré GloVe: Hyperbolic Word Embeddings
• Riemannian Adaptive Optimization Methods
• Watch Your Step: Learning Node Embeddings via Graph Attention
• NetLSD: Hearing the Shape of a Graph
• HARP: Hierarchical Representation Learning for Networks
• Is a Single Embedding Enough? Learning Node Representations that Capture Multiple Social Contexts
• Invariant and Equivariant Graph Networks
• Compositional Fairness Constraints for Graph Embeddings
• Hyperbolic Graph Neural Networks
• DDGK: Learning Graph Representations for Deep Divergence Graph Kernels
• Neural Embeddings of Graphs in Hyperbolic Space
• Generalization and Representational Limits of Graph Neural Networks
• A Tutorial on Network Embeddings
• Multi-relational Poincaré Graph Embeddings
• On the Equivalence between Positional Node Embeddings and Structural Graph Representations
• Just SLaQ When You Approximate: Accurate Spectral Distances for Web-Scale Graphs
• Evaluating Logical Generalization in Graph Neural Networks
• MONET: Debiasing Graph Embeddings via the Metadata-Orthogonal Training Unit

Cited by in corpus (33)

• Scaling Graph Neural Networks with Approximate PageRank
• TUDataset: A collection of benchmark datasets for learning with graphs
• Benchmarking Graph Neural Networks
• Beyond Homophily in Graph Neural Networks: Current Limitations and Effective Designs
• Adversarial Graph Augmentation to Improve Graph Contrastive Learning
• The Evolution of Distributed Systems for Graph Neural Networks and their Origin in Graph Processing and Deep Learning: A Survey
• Graph Clustering with Graph Neural Networks
• Graph neural network-based fault diagnosis: a review
• Grale: Designing Networks for Graph Learning
• From Trees to Continuous Embeddings and Back: Hyperbolic Hierarchical Clustering
• Socially Enhanced Situation Awareness from Microblogs using Artificial Intelligence: A Survey
• SLAPS: Self-Supervision Improves Structure Learning for Graph Neural Networks
• Shift-Robust GNNs: Overcoming the Limitations of Localized Graph Training Data
• Visualizing Graph Neural Networks with CorGIE: Corresponding a Graph to Its Embedding
• A Meta-Learning Approach for Graph Representation Learning in Multi-Task Settings
• Graph Neural Networks with Local Graph Parameters
• Schemes of Propagation Models and Source Estimators for Rumor Source Detection in Online Social Networks: A Short Survey of a Decade of Research
• Size-Invariant Graph Representations for Graph Classification Extrapolations
• Graph Learning for Combinatorial Optimization: A Survey of State-of-the-Art
• Utilising Graph Machine Learning within Drug Discovery and Development
• Interpretable Stability Bounds for Spectral Graph Filters
• Reconstruction for Powerful Graph Representations
• Wasserstein Embedding for Graph Learning
• Accelerating science with human versus alien artificial intelligences
• Financial Crime & Fraud Detection Using Graph Computing: Application Considerations & Outlook
• CopulaGNN: Towards Integrating Representational and Correlational Roles of Graphs in Graph Neural Networks
• Graph Computing for Financial Crime and Fraud Detection: Trends, Challenges and Outlook
• Non-separable Spatio-temporal Graph Kernels via SPDEs
• Graph Traversal with Tensor Functionals: A Meta-Algorithm for Scalable Learning
• Deep Constraint-based Propagation in Graph Neural Networks
• Community detection using low-dimensional network embedding algorithms
• Does your graph need a confidence boost? Convergent boosted smoothing on graphs with tabular node features
• A Literature Review of Recent Graph Embedding Techniques for Biomedical Data