Objective-Reinforced Generative Adversarial Networks (ORGAN) for Sequence Generation Models

Abstract

In unsupervised data generation tasks, besides the generation of a sample based on previous observations, one would often like to give hints to the model in order to bias the generation towards desirable metrics. We propose a method that combines Generative Adversarial Networks (GANs) and reinforcement learning (RL) in order to accomplish exactly that. While RL biases the data generation process towards arbitrary metrics, the GAN component of the reward function ensures that the model still remembers information learned from data. We build upon previous results that incorporated GANs and RL in order to generate sequence data and test this model in several settings for the generation of molecules encoded as text sequences (SMILES) and in the context of music generation, showing for each case that we can effectively bias the generation process towards desired metrics.

10 pages, 7 figures

References in corpus (6)

• Practical Bayesian Optimization of Machine Learning Algorithms
• Learning to Generate Reviews and Discovering Sentiment
• Maximum-Likelihood Augmented Discrete Generative Adversarial Networks
• Boundary-Seeking Generative Adversarial Networks
• In silico generation of novel, drug-like chemical matter using the LSTM neural network
• Learning to Fuse Music Genres with Generative Adversarial Dual Learning

Cited by in corpus (91)

• Combining Machine Learning and Computational Chemistry for Predictive Insights Into Chemical Systems
• Junction Tree Variational Autoencoder for Molecular Graph Generation
• Deep learning for molecular design - a review of the state of the art
• MolGAN: An implicit generative model for small molecular graphs
• How Generative Adversarial Networks and Their Variants Work: An Overview
• Accelerating Materials Development via Automation, Machine Learning, and High-Performance Computing
• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation
• Syntax-Directed Variational Autoencoder for Structured Data
• Conditional molecular design with deep generative models
• Exploring Chemical Space using Natural Language Processing Methodologies for Drug Discovery
• Learning Multimodal Graph-to-Graph Translation for Molecular Optimization
• Machine Learning and Big Scientific Data
• Deep Learning and Knowledge-Based Methods for Computer Aided Molecular Design -- Toward a Unified Approach: State-of-the-Art and Future Directions
• Generating and designing DNA with deep generative models
• Hierarchical Generation of Molecular Graphs using Structural Motifs
• ChemGAN challenge for drug discovery: can AI reproduce natural chemical diversity?
• Generative Models for Automatic Chemical Design
• Multi-Objective Molecule Generation using Interpretable Substructures
• Augmenting Genetic Algorithms with Deep Neural Networks for Exploring the Chemical Space
• Overcoming data scarcity with transfer learning
• Deeply learning molecular structure-property relationships using attention- and gate-augmented graph convolutional network
• Symmetry-adapted generation of 3d point sets for the targeted discovery of molecules
• Optimizing Molecules using Efficient Queries from Property Evaluations
• ChemBO: Bayesian Optimization of Small Organic Molecules with Synthesizable Recommendations
• Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations
• MolecularRNN: Generating realistic molecular graphs with optimized properties
• Learning To Navigate The Synthetically Accessible Chemical Space Using Reinforcement Learning
• Constrained Bayesian Optimization for Automatic Chemical Design
• Reinforcement Learning for Generative AI: State of the Art, Opportunities and Open Research Challenges
• Deep Molecular Dreaming: Inverse machine learning for de-novo molecular design and interpretability with surjective representations
• Quantum Machine Learning in Drug Discovery: Applications in Academia and Pharmaceutical Industries
• Creativity and Machine Learning: A Survey
• All SMILES Variational Autoencoder
• Generating valid Euclidean distance matrices
• Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted Retraining
• CogMol: Target-Specific and Selective Drug Design for COVID-19 Using Deep Generative Models
• Guiding Deep Molecular Optimization with Genetic Exploration
• Hierarchical Graph-to-Graph Translation for Molecules
• Generating 3D Molecular Structures Conditional on a Receptor Binding Site with Deep Generative Models
• Accelerated chemical space search using a quantum-inspired cluster expansion approach
• A Model to Search for Synthesizable Molecules
• Reinforcement Learning for Molecular Design Guided by Quantum Mechanics
• Materials Discovery with Extreme Properties via Reinforcement Learning-Guided Combinatorial Chemistry
• Curiosity in exploring chemical space: Intrinsic rewards for deep molecular reinforcement learning
• JANUS: Parallel Tempered Genetic Algorithm Guided by Deep Neural Networks for Inverse Molecular Design
• SafeCritic: Collision-Aware Trajectory Prediction
• C5T5: Controllable Generation of Organic Molecules with Transformers
• Uncertainty quantification of molecular property prediction with Bayesian neural networks
• PcDGAN: A Continuous Conditional Diverse Generative Adversarial Network For Inverse Design
• Actively Learning what makes a Discrete Sequence Valid
• Quantum Generative Models for Small Molecule Drug Discovery
• Efficient Generation of Structured Objects with Constrained Adversarial Networks
• Study of Deep Generative Models for Inorganic Chemical Compositions
• Improving Neural Machine Translation with Conditional Sequence Generative Adversarial Nets
• Learning a Continuous Representation of 3D Molecular Structures with Deep Generative Models
• Adversarial Learned Molecular Graph Inference and Generation
• Black Box Recursive Translations for Molecular Optimization
• Bioinformatics and Medicine in the Era of Deep Learning
• Adaptive Generation of Unrestricted Adversarial Inputs
• Symmetry-Aware Actor-Critic for 3D Molecular Design
• Generative Adversarial Networks for Crystal Structure Prediction
• Application of quantum-inspired generative models to small molecular datasets
• Spatial Graph Attention and Curiosity-driven Policy for Antiviral Drug Discovery
• Artificial Intelligence in Drug Discovery: Applications and Techniques
• AI-driven Inverse Design System for Organic Molecules
• Molecular De Novo Design through Deep Reinforcement Learning
• HelixMO: Sample-Efficient Molecular Optimization in Scene-Sensitive Latent Space
• Semi-Recurrent CNN-based VAE-GAN for Sequential Data Generation
• Multiple-objective Reinforcement Learning for Inverse Design and Identification
• Artificial Intelligence based Autonomous Molecular Design for Medical Therapeutic: A Perspective
• Goal-directed Generation of Discrete Structures with Conditional Generative Models
• Goal directed molecule generation using Monte Carlo Tree Search
• Training Generative Adversarial Networks with Adaptive Composite Gradient
• Molecule Generation Experience: An Open Platform of Material Design for Public Users
• An In-depth Summary of Recent Artificial Intelligence Applications in Drug Design
• Controlled Molecule Generator for Optimizing Multiple Chemical Properties
• Research and development of MolAICal for drug design via deep learning and classical programming
• Fréchet ChemNet Distance: A metric for generative models for molecules in drug discovery
• Learn distributed GAN with Temporary Discriminators
• A Binded VAE for Inorganic Material Generation
• Uncertainty quantification of molecular property prediction using Bayesian neural network models
• Graph Energy-based Model for Substructure Preserving Molecular Design
• Reward-Reinforced Reinforcement Learning for Multi-agent Systems
• A Classification Scheme for Inverse Design of Molecules: from Targeted Electronic Properties to Atomicity
• Game of GANs: Game-Theoretical Models for Generative Adversarial Networks
• Predicting Nanorobot Shapes via Generative Models
• RL-Duet: Online Music Accompaniment Generation Using Deep Reinforcement Learning
• Generating Multi-type Temporal Sequences to Mitigate Class-imbalanced Problem
• Learning to Sample Hard Instances for Graph Algorithms
• Probabilistic hypergraph grammars for efficient molecular optimization