A critical examination of compound stability predictions from machine-learned formation energies

Abstract

Machine learning has emerged as a novel tool for the efficient prediction of materials properties, and claims have been made that machine-learned models for the formation energy of compounds can approach the accuracy of Density Functional Theory (DFT). The models tested in this work include five recently published compositional models, a baseline model using stoichiometry alone, and a structural model. By testing seven machine learning models for formation energy on stability predictions using the Materials Project database of DFT calculations for 85,014 unique chemical compositions, we show that while formation energies can indeed be predicted well, all compositional models perform poorly on predicting the stability of compounds, making them considerably less useful than DFT for the discovery and design of new solids. Most critically, in sparse chemical spaces where few stoichiometries have stable compounds, only the structural model is capable of efficiently detecting which materials are stable. The non-incremental improvement of structural models compared with compositional models is noteworthy and encourages the use of structural models for materials discovery, with the constraint that for any new composition, the ground-state structure is not known a priori. This work demonstrates that accurate predictions of formation energy do not imply accurate predictions of stability, emphasizing the importance of assessing model performance on stability predictions, for which we provide a set of publicly available tests.

References in corpus (14)

• XGBoost: A Scalable Tree Boosting System
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties
• On representing chemical environments
• A General-Purpose Machine Learning Framework for Predicting Properties of Inorganic Materials
• Graph Networks as a Universal Machine Learning Framework for Molecules and Crystals
• AFLOW: An automatic framework for high-throughput materials discovery
• Data-driven materials science: status, challenges and perspectives
• Universal Fragment Descriptors for Predicting Electronic Properties of Inorganic Crystals
• AiiDA: Automated Interactive Infrastructure and Database for Computational Science
• Machine Learning Energies of 2 M Elpasolite (ABC${}_2$D${}_6$) Crystals
• Physical descriptor for the Gibbs energy of inorganic crystalline solids and temperature-dependent materials chemistry
• Performance of the strongly constrained and appropriately normed density functional for solid-state materials
• The role of decomposition reactions in assessing first-principles predictions of solid stability

Cited by in corpus (24)

• Recent Advances and Applications of Deep Learning Methods in Materials Science
• A critical examination of robustness and generalizability of machine learning prediction of materials properties
• Synthetic accessibility and stability rules of NASICONs
• A Deep Dive into Machine Learning Density Functional Theory for Materials Science and Chemistry
• Uncertainty Prediction for Machine Learning Models of Material Properties
• Machine-learning rationalization and prediction of solid-state synthesis conditions
• AtomSets -- A Hierarchical Transfer Learning Framework for Small and Large Materials Datasets
• JARVIS-Leaderboard: A Large Scale Benchmark of Materials Design Methods
• Machine Learning Enabled Prediction of Cathode Materials for Zn ion Batteries
• Calibrating DFT formation enthalpy calculations by multi-fidelity machine learning
• Toward Accurate Interpretable Predictions of Materials Properties within Transformer Language Models
• Analogical discovery of disordered perovskite oxides by crystal structure information hidden in unsupervised material fingerprints
• Linear Jacobi-Legendre expansion of the charge density for machine learning-accelerated electronic structure calculations
• Deep learning of spectra: Predicting the dielectric function of semiconductors
• An ${\ell }_0{\ell }_2$-norm regularized regression model for construction of robust cluster expansions in multicomponent systems
• Transferable and Robust Machine Learning Model for Predicting Stability of Si Anodes for Multivalent Cation Batteries
• Beyond Independent Error Assumptions in Large GNN Atomistic Models
• Coarse-grained crystal graph neural networks for reticular materials design
• Interpretation of Crystal Energy Landscapes with Kolmogorov-Arnold Networks
• Formation Energy Prediction of Material Crystal Structures using Deep Learning
• A charge-density machine-learning workflow for computing the infrared spectrum of molecules
• Glassy magnetic freezing of interacting clusters in LK-99-family materials
• GEWUM: General Exploration Workflow for the Utopia of Materials: A Unified Platform for Automated Structure Generation, Selection, and Validation
• Assessing the accuracy of compound formation energies with quantum Monte Carlo