NOMAD 2018 Kaggle Competition: Solving Materials Science Challenges Through Crowd Sourcing

Abstract

Machine learning (ML) is increasingly used in the field of materials science, where statistical estimates of computed properties are employed to rapidly examine the chemical space for new compounds. However, a systematic comparison of several ML models for this domain has been hindered by the scarcity of appropriate datasets of materials properties, as well as the lack of thorough benchmarking studies. To address this, a public data-analytics competition was organized by the Novel Materials Discovery (NOMAD) Centre of Excellence and hosted by the on-line platform Kaggle using a dataset of $3000$ (Al${}_x$ Ga${}_y$ In${}_z$)${}_2$ O${}_3$ compounds (with $x+y+z=1$). The aim of this challenge was to identify the best ML model for the prediction of two key physical properties that are relevant for optoelectronic applications: the electronic band gap energy and the crystalline formation energy. In this contribution, we present a summary of the top three ML approaches of the competition including the 1st place solution based on a crystal graph representation that is new for ML of the properties of materials. The 2nd place model combined many candidate descriptors from a set of compositional, atomic environment-based, and average structural properties with the light gradient-boosting machine regression model. The 3rd place model employed the smooth overlap of atomic positions representation with a neural network. To gain insight into whether the representation or the regression model determines the overall model performance, nine ML models obtained by combining the representations and regression models of the top three approaches were compared by looking at the correlations among prediction errors. At fixed representation, the largest correlation is observed in predictions made with kernel ridge regression and neural network, reflecting a similar performance on the same test set samples.

References in corpus (19)

• XGBoost: A Scalable Tree Boosting System
• Gaussian Approximation Potentials: the accuracy of quantum mechanics, without the electrons
• Popular Ensemble Methods: An Empirical Study
• Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties
• On representing chemical environments
• Fast and Accurate Modeling of Molecular Atomization Energies with Machine Learning
• New Tolerance Factor to Predict the Stability of Perovskite Oxides and Halides
• The Alloy Theoretic Automated Toolkit: A User Guide
• Big Data of Materials Science - Critical Role of the Descriptor
• SISSO: a compressed-sensing method for identifying the best low-dimensional descriptor in an immensity of offered candidates
• How to represent crystal structures for machine learning: towards fast prediction of electronic properties
• Insightful classification of crystal structures using deep learning
• Prediction model of band-gap for AX binary compounds by combination of density functional theory calculations and machine learning techniques
• Representation of compounds for machine-learning prediction of physical properties
• Materials Cartography: Representing and Mining Material Space Using Structural and Electronic Fingerprints
• Physical descriptor for the Gibbs energy of inorganic crystalline solids and temperature-dependent materials chemistry
• Cluster expansion made easy with Bayesian compressive sensing
• BOPfox program for tight-binding and analytic bond-order potential calculations
• Electronic structure based descriptor for characterizing local atomic environments

Cited by in corpus (9)

• Representations of molecules and materials for interpolation of quantum-mechanical simulations via machine learning
• Developments and Further Applications of Ephemeral Data Derived Potentials
• Machine Learning Benchmarks for the Classification of Equivalent Circuit Models from Electrochemical Impedance Spectra
• Sensitivity and Dimensionality of Atomic Environment Representations used for Machine Learning Interatomic Potentials
• A community-powered search of machine learning strategy space to find NMR property prediction models
• Tight-binding bond parameters for dimers across the periodic table from density-functional theory
• Advancing descriptor search in materials science: feature engineering and selection strategies
• Building Domain-Specific Machine Learning Workflows: A Conceptual Framework for the State-of-the-Practice
• Statistical hypothesis testing versus machine-learning binary classification: distinctions and guidelines