Toward Accurate Interpretable Predictions of Materials Properties within Transformer Language Models

Abstract

Property prediction accuracy has long been a key parameter of machine learning in materials informatics. Accordingly, advanced models showing state-of-the-art performance turn into highly parameterized black boxes missing interpretability. Here, we present an elegant way to make their reasoning transparent. Human-readable text-based descriptions automatically generated within a suite of open-source tools are proposed as materials representation. Transformer language models pretrained on 2 million peer-reviewed articles take as input well-known terms, e.g., chemical composition, crystal symmetry, and site geometry. Our approach outperforms crystal graph networks by classifying four out of five analyzed properties if one considers all available reference data. Moreover, fine-tuned text-based models show high accuracy in the ultra-small data limit. Explanations of their internal machinery are produced using local interpretability techniques and are faithful and consistent with domain expert rationales. This language-centric framework makes accurate property predictions accessible to people without artificial-intelligence expertise.

17 pages, 5 figures, 1 table

References in corpus (9)

• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Towards A Rigorous Science of Interpretable Machine Learning
• Methods for Interpreting and Understanding Deep Neural Networks
• Big Data of Materials Science - Critical Role of the Descriptor
• Captum: A unified and generic model interpretability library for PyTorch
• Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey
• Understanding Neural Networks through Representation Erasure
• Beyond scaling relations for the description of catalytic materials
• Parameterized Explainer for Graph Neural Network

Cited by in corpus (4)

• Machine Learning - Driven Materials Discovery: Unlocking Next-Generation Functional Materials - A review
• Artificial Intelligence in Materials Science and Engineering: Current Landscape, Key Challenges, and Future Trajectorie
• LLM4Mat-Bench: Benchmarking Large Language Models for Materials Property Prediction
• Coarse-grained crystal graph neural networks for reticular materials design