Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds

Abstract

We introduce tensor field neural networks, which are locally equivariant to 3D rotations, translations, and permutations of points at every layer. 3D rotation equivariance removes the need for data augmentation to identify features in arbitrary orientations. Our network uses filters built from spherical harmonics; due to the mathematical consequences of this filter choice, each layer accepts as input (and guarantees as output) scalars, vectors, and higher-order tensors, in the geometric sense of these terms. We demonstrate the capabilities of tensor field networks with tasks in geometry, physics, and chemistry.

arXiv commentchanges for NIPS submission

References in corpus (10)

• Neural Message Passing for Quantum Chemistry
• PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
• PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
• Quantum-Chemical Insights from Deep Tensor Neural Networks
• Machine learning prediction errors better than DFT accuracy
• AtomNet: A Deep Convolutional Neural Network for Bioactivity Prediction in Structure-based Drug Discovery
• On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups
• Spherical CNNs
• N-body Networks: a Covariant Hierarchical Neural Network Architecture for Learning Atomic Potentials
• Intertwiners between Induced Representations (with Applications to the Theory of Equivariant Neural Networks)

Cited by in corpus (186)

• E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials
• Combining Machine Learning and Computational Chemistry for Predictive Insights Into Chemical Systems
• Deep learning for molecular design - a review of the state of the art
• Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
• SpookyNet: Learning Force Fields with Electronic Degrees of Freedom and Nonlocal Effects
• A Comprehensive Survey on Deep Graph Representation Learning
• Everything is Connected: Graph Neural Networks
• SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks
• On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups
• 3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data
• Deep-Learning Density Functional Theory Hamiltonian for Efficient ab initio Electronic-Structure Calculation
• Learning from Protein Structure with Geometric Vector Perceptrons
• Gauge Equivariant Convolutional Networks and the Icosahedral CNN
• Enforcing Statistical Constraints in Generative Adversarial Networks for Modeling Chaotic Dynamical Systems
• Representations of molecules and materials for interpolation of quantum-mechanical simulations via machine learning
• Self-Consistent Determination of Long-Range Electrostatics in Neural Network Potentials
• Cormorant: Covariant Molecular Neural Networks
• DPA-2: a large atomic model as a multi-task learner
• Ab-initio quantum chemistry with neural-network wavefunctions
• General framework for E(3)-equivariant neural network representation of density functional theory Hamiltonian
• Machine Learning on Neutron and X-Ray Scattering
• Deep Learning and Knowledge-Based Methods for Computer Aided Molecular Design -- Toward a Unified Approach: State-of-the-Art and Future Directions
• TeaNet: universal neural network interatomic potential inspired by iterative electronic relaxations
• A physics-informed operator regression framework for extracting data-driven continuum models
• Crystal Diffusion Variational Autoencoder for Periodic Material Generation
• E(n) Equivariant Graph Neural Networks
• Informing Geometric Deep Learning with Electronic Interactions to Accelerate Quantum Chemistry
• Theoretical Guarantees for Permutation-Equivariant Quantum Neural Networks
• SchNetPack 2.0: A neural network toolbox for atomistic machine learning
• Design and Analysis of Machine Learning Exchange-Correlation Functionals via Rotationally Invariant Convolutional Descriptors
• Transferable E(3) equivariant parameterization for Hamiltonian of molecules and solids
• Recursive evaluation and iterative contraction of $N$-body equivariant features
• Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data
• Lorentz Group Equivariant Neural Network for Particle Physics
• Equivariant graph neural networks for fast electron density estimation of molecules, liquids, and solids
• Accurate large-scale simulations of siliceous zeolites by neural network potentials
• Deep-learning electronic-structure calculation of magnetic superstructures
• Hierarchical, rotation-equivariant neural networks to select structural models of protein complexes
• Machine-learning hidden symmetries
• On Learning Sets of Symmetric Elements
• Equivariant Graph Neural Networks for 3D Macromolecular Structure
• Rotation Invariant Graph Neural Networks using Spin Convolutions
• Deep Learning for 3D Point Clouds: A Survey
• Predicting tensorial molecular properties with equivariant machine-learning models
• A deep neural network for molecular wave functions in quasi-atomic minimal basis representation
• Relevance of Rotationally Equivariant Convolutions for Predicting Molecular Properties
• Equivariant Energy Flow Networks for Jet Tagging
• Fast Neural Network Approach for Direct Covariant Forces Prediction in Complex Multi-Element Extended Systems
• Deep Scale-spaces: Equivariance Over Scale
• Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphs
• Scale-Equivariant Steerable Networks
• Learning Gradient Fields for Molecular Conformation Generation
• DeltaConv: Anisotropic Operators for Geometric Deep Learning on Point Clouds
• Scale equivariance in CNNs with vector fields
• Soft Metamaterials: Adaptation and Intelligence
• A General Theory of Equivariant CNNs on Homogeneous Spaces
• E(n)-Equivariant Cartesian Tensor Passing Potential
• LieTransformer: Equivariant self-attention for Lie Groups
• An equivariant graph neural network for the elasticity tensors of all seven crystal systems
• Hydration free energies from kernel-based machine learning: Compound-database bias
• Learning from the Density to Correct Total Energy and Forces in First Principle Simulations
• On the design space between molecular mechanics and machine learning force fields
• Attentive Group Equivariant Convolutional Networks
• Neural Networks Enforcing Physical Symmetries in Nonlinear Dynamical Lattices: The Case Example of the Ablowitz-Ladik Model
• Temporally coherent backmapping of molecular trajectories from coarse-grained to atomistic resolution
• Completeness of Atomic Structure Representations
• FlexConv: Continuous Kernel Convolutions with Differentiable Kernel Sizes
• A method to challenge symmetries in data with self-supervised learning
• Denoise Pretraining on Nonequilibrium Molecules for Accurate and Transferable Neural Potentials
• Independent SE(3)-Equivariant Models for End-to-End Rigid Protein Docking
• Theoretical Aspects of Group Equivariant Neural Networks
• A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groups
• SE(3)-equivariant prediction of molecular wavefunctions and electronic densities
• Geometry-enhanced Pre-training on Interatomic Potentials
• Fast and flexible long-range models for atomistic machine learning
• Reinforcement learning for traversing chemical structure space: Optimizing transition states and minimum energy paths of molecules
• On the Universality of Rotation Equivariant Point Cloud Networks
• Data efficiency and extrapolation trends in neural network interatomic potentials
• A Wigner-Eckart Theorem for Group Equivariant Convolution Kernels
• Surrogate- and invariance-boosted contrastive learning for data-scarce applications in science
• Scale-invariant scale-channel networks: Deep networks that generalise to previously unseen scales
• Unifying machine learning and quantum chemistry -- a deep neural network for molecular wavefunctions
• Spiers Memorial Lecture: How to do impactful research in artificial intelligence for chemistry and materials science
• MGNN: Moment Graph Neural Network for Universal Molecular Potentials
• Machine learning approaches for automatic defect detection in photovoltaic systems
• Integrating Expert ODEs into Neural ODEs: Pharmacology and Disease Progression
• Stress and heat flux via automatic differentiation
• Learning 3D Representations of Molecular Chirality with Invariance to Bond Rotations
• Particle Convolution for High Energy Physics
• KineticNet: Deep learning a transferable kinetic energy functional for orbital-free density functional theory
• Algorithmic Differentiation for Automated Modeling of Machine Learned Force Fields
• Wavelet Scattering Networks for Atomistic Systems with Extrapolation of Material Properties
• Heterogeneous Molecular Graph Neural Networks for Predicting Molecule Properties
• Rotation-Equivariant Deep Learning for Diffusion MRI
• B-Spline CNNs on Lie Groups
• Predicting The One-Particle Density Matrix With Machine Learning
• Similarity Equivariant Graph Neural Networks for Homogenization of Metamaterials
• Accelerating superconductor discovery through tempered deep learning of the electron-phonon spectral function
• Rotation-equivariant Graph Neural Networks for Learning Glassy Liquids Representations
• Supervised Learning and the Finite-Temperature String Method for Computing Committor Functions and Reaction Rates
• 3D-Rotation-Equivariant Quaternion Neural Networks
• CKConv: Continuous Kernel Convolution For Sequential Data
• Machine learning and invariant theory
• Geometric and Physical Quantities Improve E(3) Equivariant Message Passing
• Leveraging SE(3) Equivariance for Self-Supervised Category-Level Object Pose Estimation
• Scaling-Translation-Equivariant Networks with Decomposed Convolutional Filters
• Frame Averaging for Invariant and Equivariant Network Design
• Spin-Weighted Spherical CNNs
• Coordinate Independent Convolutional Networks -- Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds
• Quaternion Equivariant Capsule Networks for 3D Point Clouds
• Towards Symbolic XAI -- Explanation Through Human Understandable Logical Relationships Between Features
• A smooth basis for atomistic machine learning
• Improving Transformation Invariance in Contrastive Representation Learning
• Machine learning at the atomic-scale
• Symmetry-Aware Actor-Critic for 3D Molecular Design
• From predicting to learning dissipation from pair correlations of active liquids
• Equivariant Neural Networks for Spin Dynamics Simulations of Itinerant Magnets
• Graph-neural-network predictions of solid-state NMR parameters from spherical tensor decomposition
• $\Delta$-model correction of Foundation Model based on the models own understanding
• Quantum algorithms for group convolution, cross-correlation, and equivariant transformations
• Efficient ensemble uncertainty estimation in Gaussian Processes Regression
• Investigating Ionic Diffusivity in Amorphous Solid Electrolytes using Machine Learned Interatomic Potentials
• Equivariant Machine Learning Interatomic Potentials with Global Charge Redistribution
• Probabilistic Numeric Convolutional Neural Networks
• TetraSphere: A Neural Descriptor for O(3)-Invariant Point Cloud Analysis
• Scaling Graph Neural Networks to Large Proteins
• E(n) Equivariant Normalizing Flows
• Embed Me If You Can: A Geometric Perceptron
• Equivariant Maps for Hierarchical Structures
• Geometry Sharing Network for 3D Point Cloud Classification and Segmentation
• Ab-Initio Potential Energy Surfaces by Pairing GNNs with Neural Wave Functions
• A simple approach to rotationally invariant machine learning of avector quantity
• Energy-GNoME: A Living Database of Selected Materials for Energy Applications
• Rapidly Encoding Generalizable Dynamics in a Euclidean Symmetric Neural Network
• Geometric Prediction: Moving Beyond Scalars
• Protein model quality assessment using rotation-equivariant, hierarchical neural networks
• Fine-Tuning Unifies Foundational Machine-learned Interatomic Potential Architectures at ab initio Accuracy
• Trajectory Prediction using Equivariant Continuous Convolution
• PRIN/SPRIN: On Extracting Point-wise Rotation Invariant Features
• Group Equivariant Stand-Alone Self-Attention For Vision
• Deep Positional and Relational Feature Learning for Rotation-Invariant Point Cloud Analysis
• Discrete Rotation Equivariance for Point Cloud Recognition
• Leveraging binding-site structure for drug discovery with point-cloud methods
• Automated and Autonomous Experiment in Electron and Scanning Probe Microscopy
• Complete Neural Networks for Complete Euclidean Graphs
• HEroBM: a deep equivariant graph neural network for universal backmapping from coarse-grained to all-atom representations
• Geometric Wavelet Scattering Networks on Compact Riemannian Manifolds
• Isometric Transformation Invariant and Equivariant Graph Convolutional Networks
• Physics-Constrained Predictive Molecular Latent Space Discovery with Graph Scattering Variational Autoencoder
• Quaternion Product Units for Deep Learning on 3D Rotation Groups
• A simple equivariant machine learning method for dynamics based on scalars
• Deep Learning for 2D and 3D Rotatable Data: An Overview of Methods
• Tranquil Clouds: Neural Networks for Learning Temporally Coherent Features in Point Clouds
• Leveraging SO(3)-steerable convolutions for pose-robust semantic segmentation in 3D medical data
• Sparse Steerable Convolutions: An Efficient Learning of SE(3)-Equivariant Features for Estimation and Tracking of Object Poses in 3D Space
• GeoT: A Geometry-aware Transformer for Reliable Molecular Property Prediction and Chemically Interpretable Representation Learning
• Affine Self Convolution
• Learning physics confers pose-sensitivity in structure-based virtual screening
• DV-ConvNet: Fully Convolutional Deep Learning on Point Clouds with Dynamic Voxelization and 3D Group Convolution
• Equivariant Networks for Pixelized Spheres
• Discriminative Learning of Similarity and Group Equivariant Representations
• RISA-Net: Rotation-Invariant Structure-Aware Network for Fine-Grained 3D Shape Retrieval
• Scale Equivariant Neural Networks with Morphological Scale-Spaces
• Learning Augmentation Distributions using Transformed Risk Minimization
• Deep Learning on Attributed Graphs: A Journey from Graphs to Their Embeddings and Back
• Efficient Generalized Spherical CNNs
• Rotational and Reflectional Equivariant Convolutional Neural Network for data-limited applications: Multiphase Flow demonstration
• Data efficiency in graph networks through equivariance
• Deformation Robust Roto-Scale-Translation Equivariant CNNs
• Beyond permutation equivariance in graph networks
• Physics-Augmented Learning: A New Paradigm Beyond Physics-Informed Learning
• libmolgrid: GPU Accelerated Molecular Gridding for Deep Learning Applications
• Detect the Interactions that Matter in Matter: Geometric Attention for Many-Body Systems
• Learning Equivariant Representations
• UniHam: A Large-Scale SOC-Complete Dataset and Benchmark for Hamiltonian Learning in Materials
• Rotation Equivariant 3D Hand Mesh Generation from a Single RGB Image
• Equivariant Point Network for 3D Point Cloud Analysis
• Compressing local atomic neighbourhood descriptors
• Neural Mode Jump Monte Carlo
• Rotation-Invariant Gait Identification with Quaternion Convolutional Neural Networks
• Group Equivariant Subsampling
• Neural Polarization: Toward Electron Density for Molecules by Extending Equivariant Networks
• Symmetry-driven graph neural networks
• Geometric Algebra Attention Networks for Small Point Clouds
• Knowledge Distillation of a Protein Language Model Yields a Foundational Implicit Solvent Model
• Training or Architecture? How to Incorporate Invariance in Neural Networks