Curiosity in exploring chemical space: Intrinsic rewards for deep molecular reinforcement learning

Abstract

Computer-aided design of molecules has the potential to disrupt the field of drug and material discovery. Machine learning, and deep learning, in particular, have been topics where the field has been developing at a rapid pace. Reinforcement learning is a particularly promising approach since it allows for molecular design without prior knowledge. However, the search space is vast and efficient exploration is desirable when using reinforcement learning agents. In this study, we propose an algorithm to aid efficient exploration. The algorithm is inspired by a concept known in the literature as curiosity. We show on three benchmarks that a curious agent finds better performing molecules. This indicates an exciting new research direction for reinforcement learning agents that can explore the chemical space out of their own motivation. This has the potential to eventually lead to unexpected new molecules that no human has thought about so far.

arXiv comment9 pages, 2 figures; comments welcome

References in corpus (13)

• Automatic chemical design using a data-driven continuous representation of molecules
• GuacaMol: Benchmarking Models for De Novo Molecular Design
• Junction Tree Variational Autoencoder for Molecular Graph Generation
• Self-Referencing Embedded Strings (SELFIES): A 100% robust molecular string representation
• Objective-Reinforced Generative Adversarial Networks (ORGAN) for Sequence Generation Models
• Parameter Space Noise for Exploration
• Large-Scale Study of Curiosity-Driven Learning
• Incentivizing Exploration In Reinforcement Learning With Deep Predictive Models
• Go-Explore: a New Approach for Hard-Exploration Problems
• Augmenting Genetic Algorithms with Deep Neural Networks for Exploring the Chemical Space
• A survey on intrinsic motivation in reinforcement learning
• JANUS: Parallel Tempered Genetic Algorithm Guided by Deep Neural Networks for Inverse Molecular Design
• Amortized Tree Generation for Bottom-up Synthesis Planning and Synthesizable Molecular Design

Cited by in corpus (5)

• On scientific understanding with artificial intelligence
• Reinforcement Learning for Generative AI: State of the Art, Opportunities and Open Research Challenges
• Generative diffusion model for surface structure discovery
• Evolutionary Monte Carlo of QM properties in chemical space: Electrolyte design
• Quantum Chemistry Driven Molecular Inverse Design with Data-free Reinforcement Learning