Syntax-Directed Variational Autoencoder for Structured Data

Abstract

Deep generative models have been enjoying success in modeling continuous data. However it remains challenging to capture the representations for discrete structures with formal grammars and semantics, e.g., computer programs and molecular structures. How to generate both syntactically and semantically correct data still remains largely an open problem. Inspired by the theory of compiler where the syntax and semantics check is done via syntax-directed translation (SDT), we propose a novel syntax-directed variational autoencoder (SD-VAE) by introducing stochastic lazy attributes. This approach converts the offline SDT check into on-the-fly generated guidance for constraining the decoder. Comparing to the state-of-the-art methods, our approach enforces constraints on the output space so that the output will be not only syntactically valid, but also semantically reasonable. We evaluate the proposed model with applications in programming language and molecules, including reconstruction and program/molecule optimization. The results demonstrate the effectiveness in incorporating syntactic and semantic constraints in discrete generative models, which is significantly better than current state-of-the-art approaches.

to appear in ICLR 2018

References in corpus (8)

• WaveNet: A Generative Model for Raw Audio
• Progressive Growing of GANs for Improved Quality, Stability, and Variation
• Objective-Reinforced Generative Adversarial Networks (ORGAN) for Sequence Generation Models
• SMILES Enumeration as Data Augmentation for Neural Network Modeling of Molecules
• Grammar Variational Autoencoder
• ChemGAN challenge for drug discovery: can AI reproduce natural chemical diversity?
• Deriving Neural Architectures from Sequence and Graph Kernels
• Actively Learning what makes a Discrete Sequence Valid

Cited by in corpus (56)

• Explainable Machine Learning for Scientific Insights and Discoveries
• Combining Machine Learning and Computational Chemistry for Predictive Insights Into Chemical Systems
• Junction Tree Variational Autoencoder for Molecular Graph Generation
• Deep learning for molecular design - a review of the state of the art
• Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation
• Conditional molecular design with deep generative models
• Keeping it Simple: Language Models can learn Complex Molecular Distributions
• Molecular graph generation with Graph Neural Networks
• Learning Multimodal Graph-to-Graph Translation for Molecular Optimization
• Machine learning and AI-based approaches for bioactive ligand discovery and GPCR-ligand recognition
• Hierarchical Generation of Molecular Graphs using Structural Motifs
• Generative Models for Automatic Chemical Design
• D-VAE: A Variational Autoencoder for Directed Acyclic Graphs
• Symmetry-adapted generation of 3d point sets for the targeted discovery of molecules
• A Two-Step Graph Convolutional Decoder for Molecule Generation
• Constrained Bayesian Optimization for Automatic Chemical Design
• A Deep Generative Model for Fragment-Based Molecule Generation
• All SMILES Variational Autoencoder
• Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted Retraining
• Latent Molecular Optimization for Targeted Therapeutic Design
• CogMol: Target-Specific and Selective Drug Design for COVID-19 Using Deep Generative Models
• Guiding Deep Molecular Optimization with Genetic Exploration
• A Tutorial on Deep Latent Variable Models of Natural Language
• Decoding Molecular Graph Embeddings with Reinforcement Learning
• Hierarchical Graph-to-Graph Translation for Molecules
• Generating 3D Molecular Structures Conditional on a Receptor Binding Site with Deep Generative Models
• Generating equilibrium molecules with deep neural networks
• Improving black-box optimization in VAE latent space using decoder uncertainty
• A Model to Search for Synthesizable Molecules
• Efficient Generator of Mathematical Expressions for Symbolic Regression
• Differentiable Scaffolding Tree for Molecular Optimization
• Neural Attribute Machines for Program Generation
• A generative model for molecule generation based on chemical reaction trees
• Bayesian Variational Autoencoders for Unsupervised Out-of-Distribution Detection
• Graph Deconvolutional Generation
• MIMOSA: Multi-constraint Molecule Sampling for Molecule Optimization
• Adversarial Learned Molecular Graph Inference and Generation
• Spatial Graph Attention and Curiosity-driven Policy for Antiviral Drug Discovery
• Concept-Oriented Deep Learning: Generative Concept Representations
• Artificial Intelligence in Drug Discovery: Applications and Techniques
• Combining Latent Space and Structured Kernels for Bayesian Optimization over Combinatorial Spaces
• Building LEGO Using Deep Generative Models of Graphs
• Multiple-objective Reinforcement Learning for Inverse Design and Identification
• Controlled Molecule Generator for Optimizing Multiple Chemical Properties
• CORE: Automatic Molecule Optimization Using Copy & Refine Strategy
• Goal-directed Generation of Discrete Structures with Conditional Generative Models
• Discrete Object Generation with Reversible Inductive Construction
• ast2vec: Utilizing Recursive Neural Encodings of Python Programs
• Learning Space Partitions for Path Planning
• Generative network complex (GNC) for drug discovery
• Learning Markov Random Fields for Combinatorial Structures via Sampling through Lovász Local Lemma
• Text Modeling with Syntax-Aware Variational Autoencoders
• Neural Program Generation Modulo Static Analysis
• Mining for Potent Inhibitors through Artificial Intelligence and Physics: A Unified Methodology for Ligand Based and Structure Based Drug Design
• Generate Novel Molecules With Target Properties Using Conditional Generative Models
• Molecular Attributes Transfer from Non-Parallel Data