Detect the Interactions that Matter in Matter: Geometric Attention for Many-Body Systems

Abstract

Attention mechanisms are developing into a viable alternative to convolutional layers as elementary building block of NNs. Their main advantage is that they are not restricted to capture local dependencies in the input, but can draw arbitrary connections. This unprecedented capability coincides with the long-standing problem of modeling global atomic interactions in molecular force fields and other many-body problems. In its original formulation, however, attention is not applicable to the continuous domains in which the atoms live. For this purpose we propose a variant to describe geometric relations for arbitrary atomic configurations in Euclidean space that also respects all relevant physical symmetries. We furthermore demonstrate, how the successive application of our learned attention matrices effectively translates the molecular geometry into a set of individual atomic contributions on-the-fly.

References in corpus (23)

• Adam: A Method for Stochastic Optimization
• Neural Machine Translation by Jointly Learning to Align and Translate
• Neural Message Passing for Quantum Chemistry
• Gaussian Approximation Potentials: the accuracy of quantum mechanics, without the electrons
• On representing chemical environments
• SchNet - a deep learning architecture for molecules and materials
• ANI-1: An extensible neural network potential with DFT accuracy at force field computational cost
• Molecular Graph Convolutions: Moving Beyond Fingerprints
• Quantum-Chemical Insights from Deep Tensor Neural Networks
• Machine Learning of Accurate Energy-Conserving Molecular Force Fields
• PhysNet: A Neural Network for Predicting Energies, Forces, Dipole Moments and Partial Charges
• Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds
• SpookyNet: Learning Force Fields with Electronic Degrees of Freedom and Nonlocal Effects
• FCHL revisited: faster and more accurate quantum machine learning
• Embedded Atom Neural Network Potentials: Efficient and Accurate Machine Learning with a Physically Inspired Representation
• Equivariant message passing for the prediction of tensorial properties and molecular spectra
• SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks
• On the Completeness of Atomic Structure Representations
• Cormorant: Covariant Molecular Neural Networks
• Rethinking Attention with Performers
• Reactive Dynamics and Spectroscopy of Hydrogen Transfer from Neural Network-Based Reactive Potential Energy Surfaces
• Nyströmformer: A Nyström-Based Algorithm for Approximating Self-Attention
• Sparse and Continuous Attention Mechanisms