Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges

Abstract

The last decade has witnessed an experimental revolution in data science and machine learning, epitomised by deep learning methods. Indeed, many high-dimensional learning tasks previously thought to be beyond reach -- such as computer vision, playing Go, or protein folding -- are in fact feasible with appropriate computational scale. Remarkably, the essence of deep learning is built from two simple algorithmic principles: first, the notion of representation or feature learning, whereby adapted, often hierarchical, features capture the appropriate notion of regularity for each task, and second, learning by local gradient-descent type methods, typically implemented as backpropagation. While learning generic functions in high dimensions is a cursed estimation problem, most tasks of interest are not generic, and come with essential pre-defined regularities arising from the underlying low-dimensionality and structure of the physical world. This text is concerned with exposing these regularities through unified geometric principles that can be applied throughout a wide spectrum of applications. Such a 'geometric unification' endeavour, in the spirit of Felix Klein's Erlangen Program, serves a dual purpose: on one hand, it provides a common mathematical framework to study the most successful neural network architectures, such as CNNs, RNNs, GNNs, and Transformers. On the other hand, it gives a constructive procedure to incorporate prior physical knowledge into neural architectures and provide principled way to build future architectures yet to be invented.

156 pages. Work in progress -- comments welcome!

References in corpus (35)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Sequence to Sequence Learning with Neural Networks
• Semi-Supervised Classification with Graph Convolutional Networks
• On the difficulty of training Recurrent Neural Networks
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Bootstrap your own latent: A new approach to self-supervised Learning
• Language Models are Few-Shot Learners
• Striving for Simplicity: The All Convolutional Net
• Discrete Signal Processing on Graphs
• Simplifying Graph Convolutional Networks
• Variational Graph Auto-Encoders
• A simple neural network module for relational reasoning
• Deep Graph Contrastive Representation Learning
• A Generalization of Transformer Networks to Graphs
• Neural Machine Translation in Linear Time
• Fake News Detection on Social Media using Geometric Deep Learning
• E(n) Equivariant Graph Neural Networks
• Position-aware Graph Neural Networks
• Hamiltonian Graph Networks with ODE Integrators
• Learning Graph Representations with Embedding Propagation
• Norm-Based Capacity Control in Neural Networks
• Lorentz Group Equivariant Neural Network for Particle Physics
• Learning Mesh-Based Simulation with Graph Networks
• Fanaroff-Riley classification of radio galaxies using group-equivariant convolutional neural networks
• Representation Learning via Invariant Causal Mechanisms
• Hierarchical Inter-Message Passing for Learning on Molecular Graphs
• Learning Symbolic Physics with Graph Networks
• Learning with invariances in random features and kernel models
• Size-Invariant Graph Representations for Graph Classification Extrapolations
• Scattering Networks on the Sphere for Scalable and Rotationally Equivariant Spherical CNNs
• Graph Networks with Spectral Message Passing
• Implicit Regularization in ReLU Networks with the Square Loss
• XLVIN: eXecuted Latent Value Iteration Nets
• Interpretable Stability Bounds for Spectral Graph Filters

Cited by in corpus (117)

• Challenges and Opportunities in Quantum Machine Learning
• Everything is Connected: Graph Neural Networks
• Learning Disentangled Representations in the Imaging Domain
• Informing Geometric Deep Learning with Electronic Interactions to Accelerate Quantum Chemistry
• 3D Infomax improves GNNs for Molecular Property Prediction
• Deep learning for reconstructing protein structures from cryo-EM density maps: recent advances and future directions
• A newcomer's guide to deep learning for inverse design in nano-photonics
• On Representation Knowledge Distillation for Graph Neural Networks
• Three-dimensional microstructure generation using generative adversarial neural networks in the context of continuum micromechanics
• Learning physics-constrained subgrid-scale closures in the small-data regime for stable and accurate LES
• Three-dimensional deep learning-based reduced order model for unsteady flow dynamics with variable Reynolds number
• Laplacian2Mesh: Laplacian-Based Mesh Understanding
• Building spatial symmetries into parameterized quantum circuits for faster training
• Graph neural network initialisation of quantum approximate optimisation
• Reflection Equivariant Quantum Neural Networks for Enhanced Image Classification
• Toward Accurate Interpretable Predictions of Materials Properties within Transformer Language Models
• How Attentive are Graph Attention Networks?
• Graph Neural Networks for Graph Drawing
• The Cosmic Graph: Optimal Information Extraction from Large-Scale Structure using Catalogues
• DeltaConv: Anisotropic Operators for Geometric Deep Learning on Point Clouds
• RadarGNN: Transformation Invariant Graph Neural Network for Radar-based Perception
• Direction of Arrival Estimation of Sound Sources Using Icosahedral CNNs
• Predicting waves in fluids with deep neural network
• Discovering Sparse Interpretable Dynamics from Partial Observations
• An equivariant graph neural network for the elasticity tensors of all seven crystal systems
• Understanding over-squashing and bottlenecks on graphs via curvature
• Known Operator Learning and Hybrid Machine Learning in Medical Imaging -- A Review of the Past, the Present, and the Future
• A Principled Design of Image Representation: Towards Forensic Tasks
• Toward Dynamic Stability Assessment of Power Grid Topologies using Graph Neural Networks
• Convolution, aggregation and attention based deep neural networks for accelerating simulations in mechanics
• Completeness of Atomic Structure Representations
• A Meta-Learning Approach for Training Explainable Graph Neural Networks
• Laplace HypoPINN: Physics-Informed Neural Network for hypocenter localization and its predictive uncertainty
• Automatic Symmetry Discovery with Lie Algebra Convolutional Network
• Machine learning approaches for automatic defect detection in photovoltaic systems
• Spiers Memorial Lecture: How to do impactful research in artificial intelligence for chemistry and materials science
• Nonequilibrium Statistical Mechanics and Optimal Prediction of Partially-Observed Complex Systems
• A Lorentz-Equivariant Transformer for All of the LHC
• Cardinality Estimation over Knowledge Graphs with Embeddings and Graph Neural Networks
• nuScenes Knowledge Graph -- A comprehensive semantic representation of traffic scenes for trajectory prediction
• Approximately Equivariant Quantum Neural Network for $p4m$ Group Symmetries in Images
• Beltrami Flow and Neural Diffusion on Graphs
• Symmetric Tensor Networks for Generative Modeling and Constrained Combinatorial Optimization
• On the universality of $S_n$-equivariant $k$-body gates
• Geometric Transformers for Protein Interface Contact Prediction
• Boosting Heterogeneous Catalyst Discovery by Structurally Constrained Deep Learning Models
• Large-scale graph representation learning with very deep GNNs and self-supervision
• Similarity Equivariant Graph Neural Networks for Homogenization of Metamaterials
• Relating Graph Neural Networks to Structural Causal Models
• Homogeneous vector bundles and $G$-equivariant convolutional neural networks
• Identification of vortex in unstructured mesh with graph neural networks
• Equivariant Subgraph Aggregation Networks
• Because Every Sensor Is Unique, so Is Every Pair: Handling Dynamicity in Traffic Forecasting
• Rotation-equivariant Graph Neural Networks for Learning Glassy Liquids Representations
• Forecasting West Nile Virus with Graph Neural Networks: Harnessing Spatial Dependence in Irregularly Sampled Geospatial Data
• Monocular visual simultaneous localization and mapping: (r)evolution from geometry to deep learning-based pipelines
• Conditional Injective Flows for Bayesian Imaging
• Coordinate Independent Convolutional Networks -- Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds
• ESP: Exploiting Symmetry Prior for Multi-Agent Reinforcement Learning
• SO(2)-Equivariant Downwash Models for Close Proximity Flight
• Self-supervised graph neural networks for accurate prediction of Néel temperature
• A Survey of Methods for Converting Unstructured Data to CSG Models
• Metric Flows with Neural Networks
• Geometric Methods for Sampling, Optimisation, Inference and Adaptive Agents
• On discrete symmetries of robotics systems: A group-theoretic and data-driven analysis
• E(2)-Equivariant Graph Planning for Navigation
• Field-level simulation-based inference with galaxy catalogs: the impact of systematic effects
• Bilateral Trade Flow Prediction by Gravity-informed Graph Auto-encoder
• Symmetry and Complexity in Object-Centric Deep Active Inference Models
• Deep learning complete intersection Calabi-Yau manifolds
• SITReg: Multi-resolution architecture for symmetric, inverse consistent, and topology preserving image registration
• Solving the Wide-band Inverse Scattering Problem via Equivariant Neural Networks
• Trade-off between Gradient Measurement Efficiency and Expressivity in Deep Quantum Neural Networks
• Rapidly Encoding Generalizable Dynamics in a Euclidean Symmetric Neural Network
• Learning Equivariant Energy Based Models with Equivariant Stein Variational Gradient Descent
• Disentangling intrinsic motion from neighbourhood effects in heterogeneous collective motion
• Reply to: Modern graph neural networks do worse than classical greedy algorithms in solving combinatorial optimization problems like maximum independent set
• Permutation-equivariant quantum convolutional neural networks
• Partition Pooling for Convolutional Graph Network Applications in Particle Physics
• Invariance Measures for Neural Networks
• MC-LSTM: Mass-Conserving LSTM
• EuclidNet: Deep Visual Reasoning for Constructible Problems in Geometry
• A unifying primary framework for quantum graph neural networks from quantum graph states
• Symmetry constrained neural networks for detection and localization of damage in metal plates
• Group-Convolutional Extended Dynamic Mode Decomposition
• Inferring halo masses with Graph Neural Networks
• Some observations on the ambivalent role of symmetries in Bayesian inference problems
• MimicBot: Combining Imitation and Reinforcement Learning to win in Bot Bowl
• Kinematically consistent recurrent neural networks for learning inverse problems in wave propagation
• Amortized Probabilistic Detection of Communities in Graphs
• Stability of Entropic Wasserstein Barycenters and application to random geometric graphs
• Weighing the Milky Way and Andromeda with Artificial Intelligence
• Optimal message passing for molecular prediction is simple, attentive and spatial
• Reply to: Inability of a graph neural network heuristic to outperform greedy algorithms in solving combinatorial optimization problems
• Data-Centric AI Requires Rethinking Data Notion
• Top-N: Equivariant set and graph generation without exchangeability
• On Graph Neural Network Ensembles for Large-Scale Molecular Property Prediction
• Data efficiency in graph networks through equivariance
• Space-Time Graph Neural Networks
• Reversible and irreversible bracket-based dynamics for deep graph neural networks
• Loss it right: Euclidean and Riemannian Metrics in Learning-based Visual Odometry
• Jointly Learnable Data Augmentations for Self-Supervised GNNs
• Scaling of symmetry-restricted quantum circuits
• Noether's Learning Dynamics: Role of Symmetry Breaking in Neural Networks
• Irregular Metamaterial Networks
• ChebLieNet: Invariant Spectral Graph NNs Turned Equivariant by Riemannian Geometry on Lie Groups
• Interpreting Equivariant Representations
• EarthGAN: Can we visualize the Earth's mantle convection using a surrogate model?
• Towards a Taxonomy of Graph Learning Datasets
• Properties from Mechanisms: An Equivariance Perspective on Identifiable Representation Learning
• On Inductive Biases for Machine Learning in Data Constrained Settings
• Equivariant Contrastive Learning
• Graph-LDA: Graph Structure Priors to Improve the Accuracy in Few-Shot Classification
• Geometric Algebra Attention Networks for Small Point Clouds
• Symmetry-driven graph neural networks
• Image recognition via Vietoris-Rips complex
• Equivariant plug-and-play image reconstruction