Self-Supervised Generative Models for Crystal Structures

Abstract

Drawing inspiration from the achievements of natural language processing, we adopt self-supervised learning and utilize an equivariant graph neural network to develop a unified platform designed for training generative models capable of generating crystal structures, as well as efficiently adapting to downstream tasks in material property prediction. To mitigate the challenge of incorporating large-scale assessment on the reliability of generated structures into the training process, we utilize the generative adversarial network (GAN) with its discriminator being a cost-effective evaluator for the generated structures, resulting in notable improvements in model performance. We demonstrate the utility of our model in finding the optimal crystal structure under predefined conditions. Without reliance on properties acquired experimentally or numerically, our model further displays its capability to comprehend the mechanism of crystal structure formation through its ability to grouping chemically similar elements. Therefore, this paper extends an invitation to explore deeper into the scientific understanding of material structures through generative models, offering a fresh perspective on broadening the scope and efficacy of machine learning in material science.

References in corpus (17)

• Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties
• Graph Networks as a Universal Machine Learning Framework for Molecules and Crystals
• Recent Advances and Applications of Deep Learning Methods in Materials Science
• Attention in Natural Language Processing
• Computation and data driven discovery of topological phononic materials
• Constrained crystals deep convolutional generative adversarial network for the inverse design of crystal structures
• Challenges in Large Scale Quantum Mechanical Calculations
• Data-driven discovery of novel 2D materials by deep generative models
• Crystal Diffusion Variational Autoencoder for Periodic Material Generation
• Density of States Prediction for Materials Discovery via Contrastive Learning from Probabilistic Embeddings
• EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representations
• MatterGen: a generative model for inorganic materials design
• Data Distributional Properties Drive Emergent In-Context Learning in Transformers
• Machine learning spectral indicators of topology
• Large scale quantum chemistry with Tensor Processing Units
• Scalable Diffusion for Materials Generation
• The FHI-aims Code: All-electron, ab initio materials simulations towards the exascale