Unsupervised Latent Tree Induction with Deep Inside-Outside Recursive Autoencoders

Abstract

We introduce deep inside-outside recursive autoencoders (DIORA), a fully-unsupervised method for discovering syntax that simultaneously learns representations for constituents within the induced tree. Our approach predicts each word in an input sentence conditioned on the rest of the sentence and uses inside-outside dynamic programming to consider all possible binary trees over the sentence. At test time the CKY algorithm extracts the highest scoring parse. DIORA achieves a new state-of-the-art F1 in unsupervised binary constituency parsing (unlabeled) in two benchmark datasets, WSJ and MultiNLI.

14 pages, 8 figures, 8 tables. NAACL 2019

References in corpus (1)

• Jointly Learning Sentence Embeddings and Syntax with Unsupervised Tree-LSTMs

Cited by in corpus (11)

• Syntactically Supervised Transformers for Faster Neural Machine Translation
• Are Pre-trained Language Models Aware of Phrases? Simple but Strong Baselines for Grammar Induction
• Interpreting Deep Learning Models in Natural Language Processing: A Review
• Tree Transformer: Integrating Tree Structures into Self-Attention
• Efficient Second-Order TreeCRF for Neural Dependency Parsing
• Compound Probabilistic Context-Free Grammars for Grammar Induction
• StructFormer: Joint Unsupervised Induction of Dependency and Constituency Structure from Masked Language Modeling
• Unsupervised Recurrent Neural Network Grammars
• PaLM: A Hybrid Parser and Language Model
• What is Learned in Visually Grounded Neural Syntax Acquisition
• Video-aided Unsupervised Grammar Induction