Predicting Many Crystal Properties via an Adaptive Transformer-based Framework

Abstract

Machine learning has revolutionized many fields, including materials science. However, predicting properties of crystalline materials using machine learning faces challenges in input encoding, output versatility, and interpretability. We introduce CrystalBERT, an adaptable transformer-based framework integrating space group, elemental, and unit cell information. This novel structure can seamlessly combine diverse features and accurately predict various physical properties, including topological properties, superconducting transition temperatures, dielectric constants, and more. CrystalBERT provides insightful interpretations of features influencing target properties. Our results indicate that space group and elemental information are crucial for predicting topological and superconducting properties, underscoring their intricate nature. By incorporating these features, we achieve 91\% accuracy in topological classification, surpassing prior studies and identifying previously misclassified materials. This research demonstrates that integrating diverse material information enhances the prediction of complex material properties, paving the way for more accurate and interpretable machine learning models in materials science.

38+20 pages, 5+12 figures. The codes are available upon reasonable request

References in corpus (37)

• Topological Insulators in Three Dimensions
• Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties
• Machine learning and the physical sciences
• SchNet - a deep learning architecture for molecules and materials
• Topological quantum chemistry
• Electron-phonon interactions from first principles
• Graph Networks as a Universal Machine Learning Framework for Molecules and Crystals
• Recent Advances and Applications of Deep Learning Methods in Materials Science
• The (High Quality) Topological Materials In The World
• Catalogue of Topological Electronic Materials
• Symmetry-based Indicators of Band Topology in the 230 Space Groups
• Towards ideal topological materials: Comprehensive database searches using symmetry indicators
• Atomistic Line Graph Neural Network for Improved Materials Property Predictions
• Finding Density Functionals with Machine Learning
• Topological classification of crystalline insulators through band structure combinatorics
• Phase transitions of hybrid perovskites simulated by machine-learning force fields trained on-the-fly with Bayesian inference
• Benchmarking Materials Property Prediction Methods: The Matbench Test Set and Automatminer Reference Algorithm
• Symmetry-Adapted Machine-Learning for Tensorial Properties of Atomistic Systems
• Magnetic Topological Quantum Chemistry
• Quantum Loop Topography for Machine Learning
• Deep-Learning Density Functional Theory Hamiltonian for Efficient ab initio Electronic-Structure Calculation
• Data-driven learning of total and local energies in elemental boron
• All Topological Bands of All Nonmagnetic Stoichiometric Materials
• Efficient Topological Materials Discovery Using Symmetry Indicators
• Machine learning with force-field inspired descriptors for materials: fast screening and mapping energy landscape
• Orbital Graph Convolutional Neural Network for Material Property Prediction
• Machine learning out-of-equilibrium phases of matter
• Density of States Prediction for Materials Discovery via Contrastive Learning from Probabilistic Embeddings
• Machine learning for materials discovery: two-dimensional topological insulators
• Detection of Topological Materials with Machine Learning
• Deep-learning density functional perturbation theory
• Computational Search for Magnetic and Non-magnetic 2D Topological Materials using Unified Spin-orbit Spillage Screening
• Artificial Intelligence for High-Throughput Discovery of Topological Insulators: the Example of Alloyed Tetradymites
• Topogivity: A Machine-Learned Chemical Rule for Discovering Topological Materials
• Discovering two-dimensional magnetic topological insulators by machine learning
• Beyond Independent Error Assumptions in Large GNN Atomistic Models
• Towards Foundation Models for Materials Science: The Open MatSci ML Toolkit