Neural Attribute Machines for Program Generation
arXiv:1705.09231
Abstract
Recurrent neural networks have achieved remarkable success at generating sequences with complex structures, thanks to advances that include richer embeddings of input and cures for vanishing gradients. Trained only on sequences from a known grammar, though, they can still struggle to learn rules and constraints of the grammar. Neural Attribute Machines (NAMs) are equipped with a logical machine that represents the underlying grammar, which is used to teach the constraints to the neural machine by (i) augmenting the input sequence, and (ii) optimizing a custom loss function. Unlike traditional RNNs, NAMs are exposed to the grammar, as well as samples from the language of the grammar. During generation, NAMs make significantly fewer violations of the constraints of the underlying grammar than RNNs trained only on samples from the language of the grammar.
References in corpus (15)
- Adam: A Method for Stochastic Optimization
- Sequence to Sequence Learning with Neural Networks
- Pixel Recurrent Neural Networks
- A Neural Representation of Sketch Drawings
- MolGAN: An implicit generative model for small molecular graphs
- Syntax-Directed Variational Autoencoder for Structured Data
- Mining Idioms from Source Code
- Parallel Multi-Dimensional LSTM, With Application to Fast Biomedical Volumetric Image Segmentation
- Learning to Transduce with Unbounded Memory
- Structured Generative Models of Natural Source Code
- Grammar Variational Autoencoder
- Latent Predictor Networks for Code Generation
- Tree-to-tree Neural Networks for Program Translation
- Discriminative Neural Sentence Modeling by Tree-Based Convolution
- Neural Sketch Learning for Conditional Program Generation