Deep Reinforcement Learning for De-Novo Drug Design

Abstract

We propose a novel computational strategy for de novo design of molecules with desired properties termed ReLeaSE (Reinforcement Learning for Structural Evolution). Based on deep and reinforcement learning approaches, ReLeaSE integrates two deep neural networks - generative and predictive - that are trained separately but employed jointly to generate novel targeted chemical libraries. ReLeaSE employs simple representation of molecules by their SMILES strings only. Generative models are trained with stack-augmented memory network to produce chemically feasible SMILES strings, and predictive models are derived to forecast the desired properties of the de novo generated compounds. In the first phase of the method, generative and predictive models are trained separately with a supervised learning algorithm. In the second phase, both models are trained jointly with the reinforcement learning approach to bias the generation of new chemical structures towards those with the desired physical and/or biological properties. In the proof-of-concept study, we have employed the ReLeaSE method to design chemical libraries with a bias toward structural complexity or biased toward compounds with either maximal, minimal, or specific range of physical properties such as melting point or hydrophobicity, as well as to develop novel putative inhibitors of JAK2. The approach proposed herein can find a general use for generating targeted chemical libraries of novel compounds optimized for either a single desired property or multiple properties.

References in corpus (2)

• ANI-1: An extensible neural network potential with DFT accuracy at force field computational cost
• Protein-Ligand Scoring with Convolutional Neural Networks

Cited by in corpus (63)

• Molecular Transformer - A Model for Uncertainty-Calibrated Chemical Reaction Prediction
• Machine learning for molecular simulation
• Combining Machine Learning and Computational Chemistry for Predictive Insights Into Chemical Systems
• GuacaMol: Benchmarking Models for De Novo Molecular Design
• Deep learning for molecular design - a review of the state of the art
• Big-Data Science in Porous Materials: Materials Genomics and Machine Learning
• Deep Reinforcement Learning for Cyber Security
• Accelerating Antimicrobial Discovery with Controllable Deep Generative Models and Molecular Dynamics
• Optimization of Molecules via Deep Reinforcement Learning
• Autonomous discovery in the chemical sciences part I: Progress
• SELFIES and the future of molecular string representations
• Mol-CycleGAN - a generative model for molecular optimization
• First return, then explore
• Perspective on integrating machine learning into computational chemistry and materials science
• Exploring Chemical Space using Natural Language Processing Methodologies for Drug Discovery
• Generative chemistry: drug discovery with deep learning generative models
• Towards Explainable Anticancer Compound Sensitivity Prediction via Multimodal Attention-based Convolutional Encoders
• Recent Advances and Applications of Machine Learning in Experimental Solid Mechanics: A Review
• Scaffold-based molecular design using graph generative model
• Computer-Aided Multi-Objective Optimization in Small Molecule Discovery
• Deep Learning and Knowledge-Based Methods for Computer Aided Molecular Design -- Toward a Unified Approach: State-of-the-Art and Future Directions
• A Universal Density Matrix Functional from Molecular Orbital-Based Machine Learning: Transferability across Organic Molecules
• Generative Models for Automatic Chemical Design
• Deep Reinforcement Learning Optimizes Graphene Nanopores for Efficient Desalination
• Population-based de novo molecule generation, using grammatical evolution
• Scaffold-constrained molecular generation
• Regression-clustering for Improved Accuracy and Training Cost with Molecular-Orbital-Based Machine Learning
• Reinforcement learning for autonomous preparation of Floquet-engineered states: Inverting the quantum Kapitza oscillator
• Quantum enhancements for deep reinforcement learning in large spaces
• Generative Discovery of Novel Chemical Designs using Diffusion Modeling and Transformer Deep Neural Networks with Application to Deep Eutectic Solvents
• Using Molecular Embeddings in QSAR Modeling: Does it Make a Difference?
• Flowsheet synthesis through hierarchical reinforcement learning and graph neural networks
• Conformal Prediction Under Feedback Covariate Shift for Biomolecular Design
• CRYSPNet: Crystal Structure Predictions via Neural Network
• Atomistic structure learning
• Inverse Design of Composite Metal Oxide Optical Materials based on Deep Transfer Learning
• Deep reinforcement learning-designed radiofrequency waveform in MRI
• Learning with Delayed Rewards -- A case study on inverse defect design in 2D materials
• Accurate Molecular-Orbital-Based Machine Learning Energies via Unsupervised Clustering of Chemical Space
• Recent advances in the Self-Referencing Embedding Strings (SELFIES) library
• Reinforcement learning for traversing chemical structure space: Optimizing transition states and minimum energy paths of molecules
• Application and Assessment of Deep Learning for the Generation of Potential NMDA Receptor Antagonists
• Language models in molecular discovery
• ChemSpaceAL: An Efficient Active Learning Methodology Applied to Protein-Specific Molecular Generation
• Analytical Gradients for Molecular-Orbital-Based Machine Learning
• Molecular-orbital-based Machine Learning for Open-shell and Multi-reference Systems with Kernel Addition Gaussian Process Regression
• Adversarial Learned Molecular Graph Inference and Generation
• Reversible Upper Confidence Bound Algorithm to Generate Diverse Optimized Candidates
• Discovery of structure-property relations for molecules via hypothesis-driven active learning over the chemical space
• SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learning
• Full Gradient DQN Reinforcement Learning: A Provably Convergent Scheme
• Molecular Dipole Moment Learning via Rotationally Equivariant Gaussian Process Regression with Derivatives in Molecular-orbital-based Machine Learning
• Hybrid Quantum Generative Adversarial Networks for Molecular Simulation and Drug Discovery
• Optimizing thermodynamic trajectories using evolutionary and gradient-based reinforcement learning
• Computational Rational Engineering and Development: Synergies and Opportunities
• A Bayesian Flow Network Framework for Chemistry Tasks
• How Evaluation Choices Distort the Outcome of Generative Drug Discovery
• MLSolv-A: A Novel Machine Learning-Based Prediction of Solvation Free Energies from Pairwise Atomistic Interactions
• Reinforcement Fine-Tuning for Materials Design
• Multi-Constraint Molecular Generation using Sparsely Labelled Training Data for Localized High-Concentration Electrolyte Diluent Screening
• Learn Quasi-stationary Distributions of Finite State Markov Chain
• Curiosity as a Self-Supervised Method to Improve Exploration in De novo Drug Design
• Chemical classification program synthesis using generative artificial intelligence