Graph Networks as a Universal Machine Learning Framework for Molecules and Crystals

Abstract

Graph networks are a new machine learning (ML) paradigm that supports both relational reasoning and combinatorial generalization. Here, we develop universal MatErials Graph Network (MEGNet) models for accurate property prediction in both molecules and crystals. We demonstrate that the MEGNet models outperform prior ML models such as the SchNet in 11 out of 13 properties of the QM9 molecule data set. Similarly, we show that MEGNet models trained on $\sim 60,000$ crystals in the Materials Project substantially outperform prior ML models in the prediction of the formation energies, band gaps and elastic moduli of crystals, achieving better than DFT accuracy over a much larger data set. We present two new strategies to address data limitations common in materials science and chemistry. First, we demonstrate a physically-intuitive approach to unify four separate molecular MEGNet models for the internal energy at 0 K and room temperature, enthalpy and Gibbs free energy into a single free energy MEGNet model by incorporating the temperature, pressure and entropy as global state inputs. Second, we show that the learned element embeddings in MEGNet models encode periodic chemical trends and can be transfer-learned from a property model trained on a larger data set (formation energies) to improve property models with smaller amounts of data (band gaps and elastic moduli).

References in corpus (24)

• Neural Message Passing for Quantum Chemistry
• Gaussian Approximation Potentials: the accuracy of quantum mechanics, without the electrons
• Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties
• Fast and Accurate Modeling of Molecular Atomization Energies with Machine Learning
• SchNet - a deep learning architecture for molecules and materials
• A General-Purpose Machine Learning Framework for Predicting Properties of Inorganic Materials
• Molecular Graph Convolutions: Moving Beyond Fingerprints
• AFLOW: An automatic framework for high-throughput materials discovery
• Quantum-Chemical Insights from Deep Tensor Neural Networks
• A Spectral Analysis Method for Automated Generation of Quantum-Accurate Interatomic Potentials
• Machine Learning Unifies the Modelling of Materials and Molecules
• Machine learning prediction errors better than DFT accuracy
• Universal Fragment Descriptors for Predicting Electronic Properties of Inorganic Crystals
• Gated Graph Sequence Neural Networks
• Efficient and Accurate Machine-Learning Interpolation of Atomic Energies in Compositions with Many Species
• Deep Neural Networks for Accurate Predictions of Garnet Stability
• Data-driven learning of total and local energies in elemental boron
• Atom2Vec: learning atoms for materials discovery
• Machine learning with force-field inspired descriptors for materials: fast screening and mapping energy landscape
• Order Matters: Sequence to sequence for sets
• Accurate Force Field for Molybdenum by Machine Learning Large Materials Data
• Feature Optimization for Atomistic Machine Learning Yields A Data-Driven Construction of the Periodic Table of the Elements
• Neural Message Passing with Edge Updates for Predicting Properties of Molecules and Materials
• Hierarchical Visualization of Materials Space with Graph Convolutional Neural Networks

Cited by in corpus (177)

• Recent Advances and Applications of Deep Learning Methods in Materials Science
• A Universal Graph Deep Learning Interatomic Potential for the Periodic Table
• DScribe: Library of Descriptors for Machine Learning in Materials Science
• Atomistic Line Graph Neural Network for Improved Materials Property Predictions
• Physics-inspired structural representations for molecules and materials
• Opportunities and Challenges for Machine Learning in Materials Science
• GPUMD: A package for constructing accurate machine-learned potentials and performing highly efficient atomistic simulations
• Towards Universal Neural Network Potential for Material Discovery Applicable to Arbitrary Combination of 45 Elements
• Predicting materials properties without crystal structure: Deep representation learning from stoichiometry
• Benchmarking Materials Property Prediction Methods: The Matbench Test Set and Automatminer Reference Algorithm
• Developing an improved Crystal Graph Convolutional Neural Network framework for accelerated materials discovery
• Global Attention based Graph Convolutional Neural Networks for Improved Materials Property Prediction
• A critical examination of compound stability predictions from machine-learned formation energies
• Learning Properties of Ordered and Disordered Materials from Multi-fidelity Data
• A practical guide to machine learning interatomic potentials -- Status and future
• An invertible crystallographic representation for general inverse design of inorganic crystals with targeted properties
• Graph Neural Networks for an Accurate and Interpretable Prediction of the Properties of Polycrystalline Materials
• Neural Operator: Graph Kernel Network for Partial Differential Equations
• InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization
• High-throughput discovery of novel cubic crystal materials using deep generative neural networks
• A critical examination of robustness and generalizability of machine learning prediction of materials properties
• A Survey on Graph-Based Deep Learning for Computational Histopathology
• Accelerating computational materials discovery with artificial intelligence and cloud high-performance computing: from large-scale screening to experimental validation
• Machine Learning on Neutron and X-Ray Scattering
• Unified Graph Neural Network Force-field for the Periodic Table
• Deep Learning and Knowledge-Based Methods for Computer Aided Molecular Design -- Toward a Unified Approach: State-of-the-Art and Future Directions
• Message-passing neural networks for high-throughput polymer screening
• Performance Assessment of Universal Machine Learning Interatomic Potentials: Challenges and Directions for Materials' Surfaces
• MODNet -- accurate and interpretable property predictions for limited materials datasets by feature selection and joint-learning
• Advances of Machine Learning in Materials Science: Ideas and Techniques
• Crystal Diffusion Variational Autoencoder for Periodic Material Generation
• Transfer learning for materials informatics using crystal graph convolutional neural network
• Graph convolutional neural networks as "general-purpose" property predictors: the universality and limits of applicability
• Density of States Prediction for Materials Discovery via Contrastive Learning from Probabilistic Embeddings
• PiNN: A Python Library for Building Atomic Neural Networks of Molecules and Materials
• A Deep Learning Based Automatic Defect Analysis Framework for In-situ TEM Ion Irradiations
• Designing High-Tc Superconductors with BCS-inspired Screening, Density Functional Theory and Deep-learning
• AtomSets -- A Hierarchical Transfer Learning Framework for Small and Large Materials Datasets
• End-to-end Wind Turbine Wake Modelling with Deep Graph Representation Learning
• AI-driven inverse design of materials: Past, present and future
• Materials Representation and Transfer Learning for Multi-Property Prediction
• Direct prediction of phonon density of states with Euclidean neural networks
• Graph Convolutional Neural Networks for Body Force Prediction
• Modelling the dielectric constants of crystals using machine learning
• Artificial Intelligence in Material Engineering: A review on applications of AI in Material Engineering
• The Evolution of Machine Learning Potentials for Molecules, Reactions and Materials
• Graph Neural Networks Accelerated Molecular Dynamics
• Physical Pooling Functions in Graph Neural Networks for Molecular Property Prediction
• Inverse design of glass structure with deep graph neural networks
• Permutationally invariant polynomial regression for energies and gradients, using reverse differentiation, achieves orders of magnitude speed-up with high precision compared to other machine learning methods
• Molecular Mechanics-Driven Graph Neural Network with Multiplex Graph for Molecular Structures
• Developments and applications of the OPTIMADE API for materials discovery, design, and data exchange
• Geometric Deep Learning for Molecular Crystal Structure Prediction
• GPTFF: A high-accuracy out-of-the-box universal AI force field for arbitrary inorganic materials
• Autonomous decision making for solid-state synthesis of inorganic materials
• Machine-learning structural reconstructions for accelerated point defect calculations
• A Unified Understanding of Minimum Lattice Thermal Conductivity
• Toward Accurate Interpretable Predictions of Materials Properties within Transformer Language Models
• Unveiling the complex structure-property correlation of defects in 2D materials based on high throughput datasets
• Predicting Elastic Properties of Materials from Electronic Charge Density Using 3D Deep Convolutional Neural Networks
• RawlsGCN: Towards Rawlsian Difference Principle on Graph Convolutional Network
• Structure motif centric learning framework for inorganic crystalline systems
• Propagation Tree Is Not Deep: Adaptive Graph Contrastive Learning Approach for Rumor Detection
• Recent progress in the JARVIS infrastructure for next-generation data-driven materials design
• Machine Learning - Driven Materials Discovery: Unlocking Next-Generation Functional Materials - A review
• The hBN defects database: a theoretical compilation of color centers in hexagonal boron nitride
• Multimodal Foundation Models for Material Property Prediction and Discovery
• Predicting the phase diagram of titanium dioxide with random search and pattern recognition
• DenseGNN: universal and scalable deeper graph neural networks for high-performance property prediction in crystals and molecules
• An equivariant graph neural network for the elasticity tensors of all seven crystal systems
• Deciphering Chemical Ordering in High Entropy Materials: A Machine Learning-Accelerated High-throughput Cluster Expansion Approach
• A Quantum-Chemical Bonding Database for Solid-State Materials
• Piezoelectric modulus prediction using machine learning and graph neural networks
• Robust model benchmarking and bias-imbalance in data-driven materials science: a case study on MODNet
• Artificial Intelligence in Materials Science and Engineering: Current Landscape, Key Challenges, and Future Trajectorie
• High-dimensional Bayesian Optimization of Hyperparameters for an Attention-based Network to Predict Materials Property: a Case Study on CrabNet using Ax and SAASBO
• Formula graph self-attention network for representation-domain independent materials discovery
• Detecting dynamic domains and local fluctuations in complex molecular systems via timelapse neighbors shuffling
• Implementing graph neural networks with TensorFlow-Keras
• Accelerated chemical space search using a quantum-inspired cluster expansion approach
• AI-Driven Accelerated Discovery of Intercalation-type Cathode Materials for Magnesium Batteries
• A Physically-informed Graph-based Order Parameter for the Universal Characterization of Atomic Structures
• A Universal Framework for Accurate and Efficient Geometric Deep Learning of Molecular Systems
• Data-Driven Approach to Encoding and Decoding 3-D Crystal Structures
• A community-powered search of machine learning strategy space to find NMR property prediction models
• Machine Learned Interatomic Potential for Dispersion Strengthened Plasma Facing Components
• Formulation Graphs for Mapping Structure-Composition of Battery Electrolytes to Device Performance
• Accelerating discrete dislocation dynamics simulations with graph neural networks
• Fast and Accurate Prediction of Material Properties with Three-Body Tight-Binding Model for the Periodic Table
• Multi-task graph neural networks for simultaneous prediction of global and atomic properties in ferromagnetic systems
• Hardness and fracture toughness models by symbolic regression
• Representations and Strategies for Transferable Machine Learning Models in Chemical Discovery
• InvDesFlow: An AI-driven materials inverse design workflow to explore possible high-temperature superconductors
• Constructing and evaluating machine-learned interatomic potentials for Li-based disordered rocksalts
• Deep learning of spectra: Predicting the dielectric function of semiconductors
• Machine Learning Enabled Discovery of Application Dependent Design Principles for Two-dimensional Materials
• Optimal pre-train/fine-tune strategies for accurate material property predictions
• Learning to Optimise Wind Farms with Graph Transformers
• Artificial Intelligence for Science in Quantum, Atomistic, and Continuum Systems
• Prediction of Large Magnetic Moment Materials With Graph Neural Networks and Random Forests
• Graph Nets for Partial Charge Prediction
• Predicting ionic conductivity in solids from the machine-learned potential energy landscape
• Training Algorithm Matters for the Performance of Neural Network Potential: A Case Study of Adam and the Kalman Filter Optimizers
• Dimensionalities and multiplicities determination of crystal nets
• Universal machine learning interatomic potentials poised to supplant DFT in modeling general defects in metals and random alloys
• Rotation-equivariant Graph Neural Networks for Learning Glassy Liquids Representations
• Study of Deep Generative Models for Inorganic Chemical Compositions
• General time-reversal equivariant neural network potential for magnetic materials
• Facilitating {\it ab initio} configurational sampling of multicomponent solids using an on-lattice neural network model and active learning
• A Tailored Convolutional Neural Network for Nonlinear Manifold Learning of Computational Physics Data using Unstructured Spatial Discretizations
• DPmoire: A tool for constructing accurate machine learning force fields in moiré systems
• Differentiable Physics: A Position Piece
• Accelerating Material Property Prediction using Generically Complete Isometry Invariants
• Towards Foundation Models for Materials Science: The Open MatSci ML Toolkit
• Machine Learning for Screening Large Organic Molecules
• End-to-end AI framework for interpretable prediction of molecular and crystal properties
• Discovery of structure-property relations for molecules via hypothesis-driven active learning over the chemical space
• Material Property Prediction using Graphs based on Generically Complete Isometry Invariants
• Self-supervised graph neural networks for accurate prediction of Néel temperature
• Crystalline Material Discovery in the Era of Artificial Intelligence
• CARL-G: Clustering-Accelerated Representation Learning on Graphs
• Optical materials discovery and design with federated databases and machine learning
• PINK: physical-informed machine learning for lattice thermal conductivity
• Accelerating the discovery of high-performance nonlinear optical materials using active learning and high-throughput screening
• Space Group Informed Transformer for Crystalline Materials Generation
• Learning the Crystal Structure Genome for Property Classification
• Enhancing crystal structure prediction by combining computational and experimental data via graph networks
• Coarse-grained crystal graph neural networks for reticular materials design
• Machine-learned models for magnetic materials
• Efficient Structure-Informed Featurization and Property Prediction of Ordered, Dilute, and Random Atomic Structures
• Online Graph Learning from Social Interactions
• EOSnet: Embedded Overlap Structures for Graph Neural Networks in Predicting Material Properties
• Self-Supervised Generative Models for Crystal Structures
• Materials discovery acceleration by using condition generative methodology
• AI for Scientific Discovery is a Social Problem
• A Cartesian Encoding Graph Neural Network for Crystal Structures Property Prediction: Application to Thermal Ellipsoid Estimation
• On the equivalence of molecular graph convolution and molecular wave function with poor basis set
• Benchmarking Inverse Optimization Algorithms for Materials Design
• Interpretable discovery of new semiconductors with machine learning
• Discovery of sustainable energy materials via the machine-learned material space
• Atomistic Simulations of Oxide-Water Interfaces using Machine Learning Potentials
• Equivariant Electronic Hamiltonian Prediction with Many-Body Message Passing
• Molecular distance matrix prediction based on graph convolutional networks
• Atomistic Graph Neural Networks for metals: Application to bcc iron
• Physical machine learning outperforms "human learning" in Quantum Chemistry
• All-in-one foundational models learning across quantum chemical levels
• Soliquidy: a descriptor for atomic geometrical confusion
• Neural Structure Fields with Application to Crystal Structure Autoencoders
• Developing a Complete AI-Accelerated Workflow for Superconductor Discovery
• Data Fusion of Deep Learned Molecular Embeddings for Property Prediction
• DeFecT-FF: a machine learning force field framework for high throughput defect modeling in CdTe-based solar cells
• Physics-Constrained Predictive Molecular Latent Space Discovery with Graph Scattering Variational Autoencoder
• Artificial Intelligence based Autonomous Molecular Design for Medical Therapeutic: A Perspective
• CHILI: Chemically-Informed Large-scale Inorganic Nanomaterials Dataset for Advancing Graph Machine Learning
• Site-specific graph neural network for predicting protonation energy of oxygenate molecules
• Efficient Learning of Long-Range and Equivariant Quantum Systems
• Reinforcement Fine-Tuning for Materials Design
• Bridging Text and Crystal Structures: Literature-driven Contrastive Learning for Materials Science
• Flexible dual-branched message passing neural network for quantum mechanical property prediction with molecular conformation
• FastCHGNet: Training one Universal Interatomic Potential to 1.5 Hours with 32 GPUs
• Self-supervised Representations and Node Embedding Graph Neural Networks for Accurate and Multi-scale Analysis of Materials
• Predicting Kovats Retention Indices Using Graph Neural Networks
• LiteGEM: Lite Geometry Enhanced Molecular Representation Learning for Quantum Property Prediction
• General Learning of the Electric Response of Inorganic Materials
• Towards explainable message passing networks for predicting carbon dioxide adsorption in metal-organic frameworks
• Mutual Information Maximization in Graph Neural Networks
• Predicting Material Properties Using a 3D Graph Neural Network with Invariant Local Descriptors
• Scalable Data-Driven Basis Selection for Linear Machine Learning Interatomic Potentials
• Entropy-based Active Learning of Graph Neural Network Surrogate Models for Materials Properties
• Incorporating electronic information into Machine Learning potential energy surfaces via approaching the ground-state electronic energy as a function of atom-based electronic populations
• Predicting Many Crystal Properties via an Adaptive Transformer-based Framework
• Scalable Training of Neural Network Potentials for Complex Interfaces Through Data Augmentation
• Atoms to Processes: The Role of Artificial Intelligence and Machine Learning in Chemical Engineering
• Learning Atomic Multipoles: Prediction of the Electrostatic Potential with Equivariant Graph Neural Networks
• Graph Convolutional Neural Networks for (QM)ML/MM Molecular Dynamics Simulations
• EBSD Grain Knowledge Graph Representation Learning for Material Structure-Property Prediction
• Electron-Informed Coarse-Graining Molecular Representation Learning for Real-World Molecular Physics