Materials Discovery with Extreme Properties via Reinforcement Learning-Guided Combinatorial Chemistry

Abstract

The goal of most materials discovery is to discover materials that are superior to those currently known. Fundamentally, this is close to extrapolation, which is a weak point for most machine learning models that learn the probability distribution of data. Herein, we develop reinforcement learning-guided combinatorial chemistry, which is a rule-based molecular designer driven by trained policy for selecting subsequent molecular fragments to get a target molecule. Since our model has the potential to generate all possible molecular structures that can be obtained from combinations of molecular fragments, unknown molecules with superior properties can be discovered. We theoretically and empirically demonstrate that our model is more suitable for discovering better compounds than probability distribution-learning models. In an experiment aimed at discovering molecules that hit seven extreme target properties, our model discovered 1,315 of all target-hitting molecules and 7,629 of five target-hitting molecules out of 100,000 trials, whereas the probability distribution-learning models failed. Moreover, it has been confirmed that every molecule generated under the binding rules of molecular fragments is 100% chemically valid. To illustrate the performance in actual problems, we also demonstrate that our models work well on two practical applications: discovering protein docking molecules and HIV inhibitors.

18 pages, 8 figures

References in corpus (9)

• Sequence to Sequence Learning with Neural Networks
• Conditional Generative Adversarial Nets
• Generative Adversarial Networks
• Objective-Reinforced Generative Adversarial Networks (ORGAN) for Sequence Generation Models
• Generative adversarial networks (GAN) based efficient sampling of chemical space for inverse design of inorganic materials
• High-Dimensional Materials and Process Optimization using Data-driven Experimental Design with Well-Calibrated Uncertainty Estimates
• Constrained crystals deep convolutional generative adversarial network for the inverse design of crystal structures
• Generative Chemical Transformer: Neural Machine Learning of Molecular Geometric Structures from Chemical Language via Attention
• Improving Generalization in Meta Reinforcement Learning using Learned Objectives

Cited by in corpus (1)

• Machine-Learning-Assisted Photonic Device Development: A Multiscale Approach from Theory to Characterization