Generative Discovery of Novel Chemical Designs using Diffusion Modeling and Transformer Deep Neural Networks with Application to Deep Eutectic Solvents

Abstract

We report a series of deep learning models to solve complex forward and inverse design problems in molecular modeling and design. Using both diffusion models inspired by nonequilibrium thermodynamics and attention-based transformer architectures, we demonstrate a flexible framework to capture complex chemical structures. First trained on the QM9 dataset and a series of quantum mechanical properties (e.g. homo, lumo, free energy, heat capacity, etc.), we then generalize the model to study and design key properties of deep eutectic solvents. In addition to separate forward and inverse models, we also report an integrated fully prompt-based multi-task generative pretrained transformer model that solves multiple forward, inverse design, and prediction tasks, flexibly and within one model. We show that the multi-task generative model has the overall best performance and allows for flexible integration of multiple objectives, within one model, and for distinct chemistries, suggesting that synergies emerge during training of this large language model. Trained jointly in tasks related to the QM9 dataset and deep eutectic solvents (DESs), the model can predict various quantum mechanical properties and critical properties to achieve deep eutectic solvent behavior. Several novel combinations of DESs are proposed based on this framework.

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• Denoising Diffusion Probabilistic Models
• Categorical Reparameterization with Gumbel-Softmax
• Sparks of Artificial General Intelligence: Early experiments with GPT-4
• Deep Reinforcement Learning for De-Novo Drug Design
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• Elucidating the Design Space of Diffusion-Based Generative Models
• Computational Protein Design with Deep Learning Neural Networks
• Alchemy: A Quantum Chemistry Dataset for Benchmarking AI Models

Cited by in corpus (3)

• AI-accelerated Discovery of Altermagnetic Materials
• A Data-Driven Approach to Morphogenesis under Structural Instability
• Decoratypes: An Extensible Crystal Taxonomy for Machine Learning-Guided Materials Discovery