Reinforcement Fine-Tuning for Materials Design

Abstract

Reinforcement fine-tuning played an instrumental role in enhancing the instruction-following and reasoning abilities of large language models. In this work, we employ reinforcement fine-tuning for materials design, in which discriminative machine learning models are used to provide rewards to the autoregressive transformer-based materials generative model CrystalFormer. By optimizing the reward signals-such as energy above the convex hull and material properties figures of merit-reinforcement fine-tuning infuses knowledge from discriminative models into generative models. The resulting model, CrystalFormer-RL, shows enhanced stability in generated crystals and successfully discovers crystals with desirable yet conflicting material properties, such as substantial dielectric constant and band gap simultaneously. Notably, we observe that reinforcement fine-tuning not only enables the property-guided material design but also unlocks property-based material retrieval behavior of pretrained generative model. The present framework opens an exciting gateway to the synergies of the machine learning ecosystem for materials design.

10 pages, 7 figures

References in corpus (17)

• Automatic chemical design using a data-driven continuous representation of molecules
• Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties
• Deep Potential Molecular Dynamics: a scalable model with the accuracy of quantum mechanics
• E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials
• Graph Networks as a Universal Machine Learning Framework for Molecules and Crystals
• Deep Reinforcement Learning for De-Novo Drug Design
• Quasiparticle Self-Consistent GW Theory
• A Universal Graph Deep Learning Interatomic Potential for the Periodic Table
• Solving Statistical Mechanics Using Variational Autoregressive Networks
• Learning Properties of Ordered and Disordered Materials from Multi-fidelity Data
• Neural Network Renormalization Group
• DPA-2: a large atomic model as a multi-task learner
• Deep learning generative model for crystal structure prediction
• The quantum geometric origin of capacitance in insulators
• Con-CDVAE: A method for the conditional generation of crystal structures
• Universal relation between energy gap and dielectric constant
• InvDesFlow: An AI-driven materials inverse design workflow to explore possible high-temperature superconductors