Graph Element Networks: adaptive, structured computation and memory

Abstract

We explore the use of graph neural networks (GNNs) to model spatial processes in which there is no a priori graphical structure. Similar to finite element analysis, we assign nodes of a GNN to spatial locations and use a computational process defined on the graph to model the relationship between an initial function defined over a space and a resulting function in the same space. We use GNNs as a computational substrate, and show that the locations of the nodes in space as well as their connectivity can be optimized to focus on the most complex parts of the space. Moreover, this representational strategy allows the learned input-output relationship to generalize over the size of the underlying space and run the same model at different levels of precision, trading computation for accuracy. We demonstrate this method on a traditional PDE problem, a physical prediction problem from robotics, and learning to predict scene images from novel viewpoints.

arXiv commentAccepted to ICML 2019

References in corpus (18)

• Semi-Supervised Classification with Graph Convolutional Networks
• Representation Learning with Contrastive Predictive Coding
• Neural Message Passing for Quantum Chemistry
• Geometric deep learning: going beyond Euclidean data
• Spectral Networks and Locally Connected Networks on Graphs
• Graph Neural Networks: A Review of Methods and Applications
• DRAW: A Recurrent Neural Network For Image Generation
• Interaction Networks for Learning about Objects, Relations and Physics
• Neural Ordinary Differential Equations
• Few-Shot Learning with Graph Neural Networks
• Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids
• Flexible Neural Representation for Physics Prediction
• Relational Neural Expectation Maximization: Unsupervised Discovery of Objects and their Interactions
• Neural Map: Structured Memory for Deep Reinforcement Learning
• Neural Episodic Control
• Relational inductive bias for physical construction in humans and machines
• A Taxonomy for Neural Memory Networks
• Attentive Neural Processes

Cited by in corpus (4)

• Combining Differentiable PDE Solvers and Graph Neural Networks for Fluid Flow Prediction
• Multi-Fidelity Machine Learning Applied to Steady Fluid Flows
• Reinforcement Learning for Adaptive Mesh Refinement
• Simulating Continuum Mechanics with Multi-Scale Graph Neural Networks