Implicit Graph Neural Networks

Abstract

Graph Neural Networks (GNNs) are widely used deep learning models that learn meaningful representations from graph-structured data. Due to the finite nature of the underlying recurrent structure, current GNN methods may struggle to capture long-range dependencies in underlying graphs. To overcome this difficulty, we propose a graph learning framework, called Implicit Graph Neural Networks (IGNN), where predictions are based on the solution of a fixed-point equilibrium equation involving implicitly defined "state" vectors. We use the Perron-Frobenius theory to derive sufficient conditions that ensure well-posedness of the framework. Leveraging implicit differentiation, we derive a tractable projected gradient descent method to train the framework. Experiments on a comprehensive range of tasks show that IGNNs consistently capture long-range dependencies and outperform the state-of-the-art GNN models.

Accepted by NeurIPS 2020 at: https://papers.nips.cc/paper/2020/hash/8b5c8441a8ff8e151b191c53c1842a38-Abstract.html

References in corpus (5)

• Semi-Supervised Classification with Graph Convolutional Networks
• Equality of Opportunity in Supervised Learning
• Simplifying Graph Convolutional Networks
• Deep Equilibrium Models
• Geom-GCN: Geometric Graph Convolutional Networks

Cited by in corpus (16)

• Nonlocal Kernel Network (NKN): a Stable and Resolution-Independent Deep Neural Network
• UltraGCN: Ultra Simplification of Graph Convolutional Networks for Recommendation
• Adversarial Attack Framework on Graph Embedding Models with Limited Knowledge
• Path-based Explanation for Knowledge Graph Completion
• Nonsmooth Implicit Differentiation for Machine Learning and Optimization
• 3D-IDS: Doubly Disentangled Dynamic Intrusion Detection
• On Training Implicit Models
• Graph Neural Networks Inspired by Classical Iterative Algorithms
• On Positional and Structural Node Features for Graph Neural Networks on Non-attributed Graphs
• Towards Lightweight Graph Neural Network Search with Curriculum Graph Sparsification
• Multiset-Equivariant Set Prediction with Approximate Implicit Differentiation
• One-step differentiation of iterative algorithms
• Convergent Graph Solvers
• Reversible and irreversible bracket-based dynamics for deep graph neural networks
• Separated Contrastive Learning for Matching in Cross-domain Recommendation with Curriculum Scheduling
• Implicit vs Unfolded Graph Neural Networks