Latent Attention For If-Then Program Synthesis

Abstract

Automatic translation from natural language descriptions into programs is a longstanding challenging problem. In this work, we consider a simple yet important sub-problem: translation from textual descriptions to If-Then programs. We devise a novel neural network architecture for this task which we train end-to-end. Specifically, we introduce Latent Attention, which computes multiplicative weights for the words in the description in a two-stage process with the goal of better leveraging the natural language structures that indicate the relevant parts for predicting program elements. Our architecture reduces the error rate by 28.57% compared to prior art. We also propose a one-shot learning scenario of If-Then program synthesis and simulate it with our existing dataset. We demonstrate a variation on the training procedure for this scenario that outperforms the original procedure, significantly closing the gap to the model trained with all data.

Accepted by NIPS 2016

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Recurrent Neural Network Regularization
• Language to Logical Form with Neural Attention
• Latent Predictor Networks for Code Generation

Cited by in corpus (3)

• A Survey on Machine Learning Techniques for Source Code Analysis
• From Natural Language Instructions to Complex Processes: Issues in Chaining Trigger Action Rules
• Evaluating Sequence-to-Sequence Learning Models for If-Then Program Synthesis