Hamiltonian Neural Networks

Abstract

Even though neural networks enjoy widespread use, they still struggle to learn the basic laws of physics. How might we endow them with better inductive biases? In this paper, we draw inspiration from Hamiltonian mechanics to train models that learn and respect exact conservation laws in an unsupervised manner. We evaluate our models on problems where conservation of energy is important, including the two-body problem and pixel observations of a pendulum. Our model trains faster and generalizes better than a regular neural network. An interesting side effect is that our model is perfectly reversible in time.

arXiv commentConference paper at NeurIPS 2019. Main paper has 8 pages and 5 figures

References in corpus (15)

• Playing Atari with Deep Reinforcement Learning
• Fast and Accurate Modeling of Molecular Atomization Energies with Machine Learning
• ANI-1: An extensible neural network potential with DFT accuracy at force field computational cost
• Machine Learning of Accurate Energy-Conserving Molecular Force Fields
• Interaction Networks for Learning about Objects, Relations and Physics
• Neural Ordinary Differential Equations
• Discovering physical concepts with neural networks
• Recurrent World Models Facilitate Policy Evolution
• Accelerating Eulerian Fluid Simulation With Convolutional Networks
• Learning Dexterous In-Hand Manipulation
• The Reversible Residual Network: Backpropagation Without Storing Activations
• Relational inductive bias for physical construction in humans and machines
• i-RevNet: Deep Invertible Networks
• Reversible Recurrent Neural Networks
• Learning Symmetries of Classical Integrable Systems

Cited by in corpus (133)

• Discovering Symbolic Models from Deep Learning with Inductive Biases
• Machine Learning for Microcontroller-Class Hardware: A Review
• Physics Informed Neural Networks for Control Oriented Thermal Modeling of Buildings
• A Review of Machine Learning Methods Applied to Structural Dynamics and Vibroacoustic
• Symplectic ODE-Net: Learning Hamiltonian Dynamics with Control
• Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solvers
• AI Poincaré: Machine Learning Conservation Laws from Trajectories
• Graph Neural Networks Meet Neural-Symbolic Computing: A Survey and Perspective
• Characterizing possible failure modes in physics-informed neural networks
• Augmenting Physical Models with Deep Networks for Complex Dynamics Forecasting
• Deep learning of thermodynamics-aware reduced-order models from data
• Dissecting Neural ODEs
• Natural quantum reservoir computing for temporal information processing
• Physics enhanced neural networks predict order and chaos
• Continuous-time system identification with neural networks: Model structures and fitting criteria
• Hamiltonian Graph Networks with ODE Integrators
• Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data
• Symplectic Recurrent Neural Networks
• Lagrangian Neural Networks
• GFINNs: GENERIC Formalism Informed Neural Networks for Deterministic and Stochastic Dynamical Systems
• Interpretable Sequence Learning for COVID-19 Forecasting
• The Ramanujan Machine: Automatically Generated Conjectures on Fundamental Constants
• Thermodynamics-informed graph neural networks
• Symbolic Pregression: Discovering Physical Laws from Distorted Video
• Physics-Informed Supervised Residual Learning for Electromagnetic Modeling
• Parameterized Neural Ordinary Differential Equations: Applications to Computational Physics Problems
• Neural Network Augmented Physics Models for Systems with Partially Unknown Dynamics: Application to Slider-Crank Mechanism
• Symplectic Neural Networks in Taylor Series Form for Hamiltonian Systems
• Port-Hamiltonian Neural Networks for Learning Explicit Time-Dependent Dynamical Systems
• A Survey on Physics Informed Reinforcement Learning: Review and Open Problems
• Learning Hamiltonian dynamics by reservoir computer
• Dissipative SymODEN: Encoding Hamiltonian Dynamics with Dissipation and Control into Deep Learning
• SymmetryGAN: Symmetry Discovery with Deep Learning
• Symplectic Learning for Hamiltonian Neural Networks
• Forecasting Sequential Data using Consistent Koopman Autoencoders
• An introduction to variational inference in Geophysical inverse problems
• Hamiltonian Generative Networks
• Neural Canonical Transformation with Symplectic Flows
• Machine-Learning Non-Conservative Dynamics for New-Physics Detection
• Hybrid Learning of Time-Series Inverse Dynamics Models for Locally Isotropic Robot Motion
• Learning Neural Event Functions for Ordinary Differential Equations
• Lagrangian operator inference enhanced with structure-preserving machine learning for nonintrusive model reduction of mechanical systems
• Meta-Learning Symmetries by Reparameterization
• Image Inversion and Uncertainty Quantification for Constitutive Laws of Pattern Formation
• Data-driven Identification of 2D Partial Differential Equations using extracted physical features
• Data-driven Prediction of General Hamiltonian Dynamics via Learning Exactly-Symplectic Maps
• Hypersolvers: Toward Fast Continuous-Depth Models
• RelaxNet: A structure-preserving neural network to approximate the Boltzmann collision operator
• Gaussian Process Port-Hamiltonian Systems: Bayesian Learning with Physics Prior
• Linearly Constrained Neural Networks
• LagNetViP: A Lagrangian Neural Network for Video Prediction
• Coupled Oscillatory Recurrent Neural Network (coRNN): An accurate and (gradient) stable architecture for learning long time dependencies
• Towards Complex Dynamic Physics System Simulation with Graph Neural ODEs
• Locally-symplectic neural networks for learning volume-preserving dynamics
• Integrability ex machina
• Augmenting Differentiable Simulators with Neural Networks to Close the Sim2Real Gap
• Learning Physical Constraints with Neural Projections
• PDE-Driven Spatiotemporal Disentanglement
• Stable Neural Flows
• UnICORNN: A recurrent model for learning very long time dependencies
• Encoding Physical Constraints in Differentiable Newton-Euler Algorithm
• Exact conservation laws for neural network integrators of dynamical systems
• Learning Generalized Quasi-Geostrophic Models Using Deep Neural Numerical Models
• Learning Group Structure and Disentangled Representations of Dynamical Environments
• Deep Potential: Recovering the gravitational potential from a snapshot of phase space
• Physics-Integrated Variational Autoencoders for Robust and Interpretable Generative Modeling
• Sparse Symplectically Integrated Neural Networks
• Machine-learning Kohn-Sham potential from dynamics in time-dependent Kohn-Sham systems
• Forward Prediction for Physical Reasoning
• Sparsely constrained neural networks for model discovery of PDEs
• Discovering Efficient Periodic Behaviours in Mechanical Systems via Neural Approximators
• Weak Collocation Regression method: fast reveal hidden stochastic dynamics from high-dimensional aggregate data
• Neural Networks with Physics-Informed Architectures and Constraints for Dynamical Systems Modeling
• Physics-Informed Graph-Mesh Networks for PDEs: A hybrid approach for complex problems
• Predicting Change, Not States: An Alternate Framework for Neural PDE Surrogates
• Learning Stochastic Behaviour from Aggregate Data
• Noisy Recurrent Neural Networks
• Extending Lagrangian and Hamiltonian Neural Networks with Differentiable Contact Models
• PCE-PINNs: Physics-Informed Neural Networks for Uncertainty Propagation in Ocean Modeling
• One-Shot Transfer Learning of Physics-Informed Neural Networks
• A hybrid approach for solving the gravitational N-body problem with Artificial Neural Networks
• Inverse modified differential equations for discovery of dynamics
• Certified machine learning: Rigorous a posteriori error bounds for PDE defined PINNs
• Learning Potentials of Quantum Systems using Deep Neural Networks
• HamNet: Conformation-Guided Molecular Representation with Hamiltonian Neural Networks
• A Differentiable Newton Euler Algorithm for Multi-body Model Learning
• Tailoring: encoding inductive biases by optimizing unsupervised objectives at prediction time
• Neural Vortex Method: from Finite Lagrangian Particles to Infinite Dimensional Eulerian Dynamics
• Continuous-Time Model-Based Reinforcement Learning
• On Computational Poisson Geometry II: Numerical Methods
• Climate Modeling with Neural Diffusion Equations
• Unsupervised Learning of Lagrangian Dynamics from Images for Prediction and Control
• Composed Physics- and Data-driven System Identification for Non-autonomous Systems in Control Engineering
• DDPNOpt: Differential Dynamic Programming Neural Optimizer
• Efficient PINNs via Multi-Head Unimodular Regularization of the Solutions Space
• MC-LSTM: Mass-Conserving LSTM
• Continuous-Depth Neural Models for Dynamic Graph Prediction
• Benchmarking Energy-Conserving Neural Networks for Learning Dynamics from Data
• Physics-aware Spatiotemporal Modules with Auxiliary Tasks for Meta-Learning
• The Autodidactic Universe
• Controlling Neural Networks with Rule Representations
• Physics-enhanced Neural Networks in the Small Data Regime
• Meta-Solver for Neural Ordinary Differential Equations
• Hamiltonian reduction using a convolutional auto-encoder coupled to an Hamiltonian neural network
• Nested Mixture of Experts: Cooperative and Competitive Learning of Hybrid Dynamical System
• Learning Dynamics Models with Stable Invariant Sets
• Deep Learning Alternative to Explicit Model Predictive Control for Unknown Nonlinear Systems
• Kinematically consistent recurrent neural networks for learning inverse problems in wave propagation
• Time-Reversal Symmetric ODE Network
• Neural Lyapunov Redesign
• Model discovery in the sparse sampling regime
• Hamiltonian-based Neural ODE Networks on the SE(3) Manifold For Dynamics Learning and Control
• SyMetric: Measuring the Quality of Learnt Hamiltonian Dynamics Inferred from Vision
• Neural Symplectic Integrator with Hamiltonian Inductive Bias for the Gravitational $N$-body Problem
• Learning the Dynamics of Sparsely Observed Interacting Systems
• Reversible and irreversible bracket-based dynamics for deep graph neural networks
• Derivative-free discrete gradient methods
• A Grid-Structured Model of Tubular Reactors
• Rodent: Relevance determination in differential equations
• Differential Euler: Designing a Neural Network approximator to solve the Chaotic Three Body Problem
• Learning to Reach, Swim, Walk and Fly in One Trial: Data-Driven Control with Scarce Data and Side Information
• GEM: Group Enhanced Model for Learning Dynamical Control Systems
• Learning second order coupled differential equations that are subject to non-conservative forces
• Nonseparable Symplectic Neural Networks
• Training Structured Mechanical Models by Minimizing Discrete Euler-Lagrange Residual
• Learning Deep Energy Shaping Policies for Stability-Guaranteed Manipulation
• (Thesis) Reservoir Computing With Dynamical Systems
• Towards Conditional Generation of Minimal Action Potential Pathways for Molecular Dynamics
• Extracting Dynamical Models from Data
• SymmetryLens: Unsupervised Symmetry Learning via Locality and Density Preservation
• SLIDE: A machine-learning based method for forced dynamic response estimation of multibody systems
• A Comparative Evaluation of Additive Separability Tests for Physics-Informed Machine Learning
• Fixed-kinetic Neural Hamiltonian Flows for enhanced interpretability and reduced complexity