From Language to Programs: Bridging Reinforcement Learning and Maximum Marginal Likelihood

Abstract

Our goal is to learn a semantic parser that maps natural language utterances into executable programs when only indirect supervision is available: examples are labeled with the correct execution result, but not the program itself. Consequently, we must search the space of programs for those that output the correct result, while not being misled by spurious programs: incorrect programs that coincidentally output the correct result. We connect two common learning paradigms, reinforcement learning (RL) and maximum marginal likelihood (MML), and then present a new learning algorithm that combines the strengths of both. The new algorithm guards against spurious programs by combining the systematic search traditionally employed in MML with the randomized exploration of RL, and by updating parameters such that probability is spread more evenly across consistent programs. We apply our learning algorithm to a new neural semantic parser and show significant gains over existing state-of-the-art results on a recent context-dependent semantic parsing task.

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (2017)

References in corpus (1)

• Reward Augmented Maximum Likelihood for Neural Structured Prediction

Cited by in corpus (21)

• Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning
• Deep Reinforcement Learning: An Overview
• Multi-Hop Knowledge Graph Reasoning with Reward Shaping
• Weakly-Supervised Deep Learning of Heat Transport via Physics Informed Loss
• Semantic Parsing with Syntax- and Table-Aware SQL Generation
• An Encoder-Decoder Framework Translating Natural Language to Database Queries
• Unified Pragmatic Models for Generating and Following Instructions
• Macro Grammars and Holistic Triggering for Efficient Semantic Parsing
• Around the GLOBE: Numerical Aggregation Question-Answering on Heterogeneous Genealogical Knowledge Graphs with Deep Neural Networks
• Policy Shaping and Generalized Update Equations for Semantic Parsing from Denotations
• Question Generation from SQL Queries Improves Neural Semantic Parsing
• An Overview of Natural Language State Representation for Reinforcement Learning
• Search from History and Reason for Future: Two-stage Reasoning on Temporal Knowledge Graphs
• Program Enhanced Fact Verification with Verbalization and Graph Attention Network
• Translating Natural Language to SQL using Pointer-Generator Networks and How Decoding Order Matters
• NeurIPS 2020 NLC2CMD Competition: Translating Natural Language to Bash Commands
• Neural Abstructions: Abstractions that Support Construction for Grounded Language Learning
• Learning to Organize Knowledge and Answer Questions with N-Gram Machines
• Neural Sequence Model Training via $\alpha$-divergence Minimization
• Mapping Natural Language Commands to Web Elements
• Learning to Generate Structured Queries from Natural Language with Indirect Supervision