Grammar as a Foreign Language

Abstract

Syntactic constituency parsing is a fundamental problem in natural language processing and has been the subject of intensive research and engineering for decades. As a result, the most accurate parsers are domain specific, complex, and inefficient. In this paper we show that the domain agnostic attention-enhanced sequence-to-sequence model achieves state-of-the-art results on the most widely used syntactic constituency parsing dataset, when trained on a large synthetic corpus that was annotated using existing parsers. It also matches the performance of standard parsers when trained only on a small human-annotated dataset, which shows that this model is highly data-efficient, in contrast to sequence-to-sequence models without the attention mechanism. Our parser is also fast, processing over a hundred sentences per second with an unoptimized CPU implementation.

References in corpus (6)

• Sequence to Sequence Learning with Neural Networks
• End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results
• Learning to Execute
• A Linear Observed Time Statistical Parser Based on Maximum Entropy Models
• Addressing the Rare Word Problem in Neural Machine Translation
• Three Generative, Lexicalised Models for Statistical Parsing

Cited by in corpus (56)

• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• TensorFlow: A system for large-scale machine learning
• Deep Learning with Differential Privacy
• Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
• A Neural Conversational Model
• Attention in Natural Language Processing
• Reasoning about Entailment with Neural Attention
• A Simple Way to Initialize Recurrent Networks of Rectified Linear Units
• Learning Deep Generative Models of Graphs
• Neural GPUs Learn Algorithms
• A Hierarchical Neural Autoencoder for Paragraphs and Documents
• Contextual LSTM (CLSTM) models for Large scale NLP tasks
• On Extended Long Short-term Memory and Dependent Bidirectional Recurrent Neural Network
• Order Matters: Sequence to sequence for sets
• Database Meets Deep Learning: Challenges and Opportunities
• When Are Tree Structures Necessary for Deep Learning of Representations?
• Linking the Neural Machine Translation and the Prediction of Organic Chemistry Reactions
• Towards Universal Paraphrastic Sentence Embeddings
• RobustFill: Neural Program Learning under Noisy I/O
• Advances in All-Neural Speech Recognition
• Compositional Vector Space Models for Knowledge Base Completion
• Paradigm Shift in Natural Language Processing
• AMR Dependency Parsing with a Typed Semantic Algebra
• Back to the Basics: Bayesian extensions of IRT outperform neural networks for proficiency estimation
• S-Net: From Answer Extraction to Answer Generation for Machine Reading Comprehension
• Structured Training for Neural Network Transition-Based Parsing
• Evaluating prose style transfer with the Bible
• An empirical study on the effectiveness of images in Multimodal Neural Machine Translation
• Wasserstein Generative Adversarial Uncertainty Quantification in Physics-Informed Neural Networks
• Character-Based Handwritten Text Transcription with Attention Networks
• A Deep Memory-based Architecture for Sequence-to-Sequence Learning
• What to talk about and how? Selective Generation using LSTMs with Coarse-to-Fine Alignment
• Bengali Abstractive News Summarization(BANS): A Neural Attention Approach
• Sequence Labeling Parsing by Learning Across Representations
• Unsupervised Pretraining for Sequence to Sequence Learning
• The Statistical Recurrent Unit
• Structured Prediction Energy Networks
• SVD-PINNs: Transfer Learning of Physics-Informed Neural Networks via Singular Value Decomposition
• User Modeling for Task Oriented Dialogues
• Dual-interest Factorization-heads Attention for Sequential Recommendation
• Constructive Type-Logical Supertagging with Self-Attention Networks
• A Linear Dynamical System Model for Text
• Visually Grounded Word Embeddings and Richer Visual Features for Improving Multimodal Neural Machine Translation
• Log-Linear RNNs: Towards Recurrent Neural Networks with Flexible Prior Knowledge
• Recurrent Models for Auditory Attention in Multi-Microphone Distance Speech Recognition
• The NLP Engine: A Universal Turing Machine for NLP
• Low-rank passthrough neural networks
• Deep Neural Network for Semantic-based Text Recognition in Images
• Read, Tag, and Parse All at Once, or Fully-neural Dependency Parsing
• Differentially Private Variational Dropout
• Differentially Private Dropout
• Dependency Parsing with LSTMs: An Empirical Evaluation
• KoSpeech: Open-Source Toolkit for End-to-End Korean Speech Recognition
• Neural Multi-Source Morphological Reinflection
• Use Generalized Representations, But Do Not Forget Surface Features
• On Multilingual Training of Neural Dependency Parsers