IRNet: A General Purpose Deep Residual Regression Framework for Materials Discovery

Abstract

Materials discovery is crucial for making scientific advances in many domains. Collections of data from experiments and first-principle computations have spurred interest in applying machine learning methods to create predictive models capable of mapping from composition and crystal structures to materials properties. Generally, these are regression problems with the input being a 1D vector composed of numerical attributes representing the material composition and/or crystal structure. While neural networks consisting of fully connected layers have been applied to such problems, their performance often suffers from the vanishing gradient problem when network depth is increased. In this paper, we study and propose design principles for building deep regression networks composed of fully connected layers with numerical vectors as input. We introduce a novel deep regression network with individual residual learning, IRNet, that places shortcut connections after each layer so that each layer learns the residual mapping between its output and input. We use the problem of learning properties of inorganic materials from numerical attributes derived from material composition and/or crystal structure to compare IRNet's performance against that of other machine learning techniques. Using multiple datasets from the Open Quantum Materials Database (OQMD) and Materials Project for training and evaluation, we show that IRNet provides significantly better prediction performance than the state-of-the-art machine learning approaches currently used by domain scientists. We also show that IRNet's use of individual residual learning leads to better convergence during the training phase than when shortcut connections are between multi-layer stacks while maintaining the same number of parameters.

9 pages, under publication at KDD'19

References in corpus (18)

• Adam: A Method for Stochastic Optimization
• Scikit-learn: Machine Learning in Python
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Fast and Accurate Modeling of Molecular Atomization Energies with Machine Learning
• SchNet - a deep learning architecture for molecules and materials
• A General-Purpose Machine Learning Framework for Predicting Properties of Inorganic Materials
• Training Very Deep Networks
• Big Data of Materials Science - Critical Role of the Descriptor
• Machine Learning of Molecular Electronic Properties in Chemical Compound Space
• How to represent crystal structures for machine learning: towards fast prediction of electronic properties
• Machine Learning Energies of 2 M Elpasolite (ABC${}_2$D${}_6$) Crystals
• Representation of compounds for machine-learning prediction of physical properties
• Atom2Vec: learning atoms for materials discovery
• SMILES2Vec: An Interpretable General-Purpose Deep Neural Network for Predicting Chemical Properties
• Spatially - resolved study of the Meissner effect in superconductors using NV-centers-in-diamond optical magnetometry
• CheMixNet: Mixed DNN Architectures for Predicting Chemical Properties using Multiple Molecular Representations
• An Improved Residual LSTM Architecture for Acoustic Modeling

Cited by in corpus (6)

• Recent Advances and Applications of Deep Learning Methods in Materials Science
• Opportunities and Challenges for Machine Learning in Materials Science
• Advances of Machine Learning in Materials Science: Ideas and Techniques
• AtomSets -- A Hierarchical Transfer Learning Framework for Small and Large Materials Datasets
• JARVIS-Leaderboard: A Large Scale Benchmark of Materials Design Methods
• CHILI: Chemically-Informed Large-scale Inorganic Nanomaterials Dataset for Advancing Graph Machine Learning