Learning to Understand Phrases by Embedding the Dictionary

Abstract

Distributional models that learn rich semantic word representations are a success story of recent NLP research. However, developing models that learn useful representations of phrases and sentences has proved far harder. We propose using the definitions found in everyday dictionaries as a means of bridging this gap between lexical and phrasal semantics. Neural language embedding models can be effectively trained to map dictionary definitions (phrases) to (lexical) representations of the words defined by those definitions. We present two applications of these architectures: "reverse dictionaries" that return the name of a concept given a definition or description and general-knowledge crossword question answerers. On both tasks, neural language embedding models trained on definitions from a handful of freely-available lexical resources perform as well or better than existing commercial systems that rely on significant task-specific engineering. The results highlight the effectiveness of both neural embedding architectures and definition-based training for developing models that understand phrases and sentences.

References in corpus (8)

• ADADELTA: An Adaptive Learning Rate Method
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models
• Large-scale Simple Question Answering with Memory Networks
• An Autoencoder Approach to Learning Bilingual Word Representations
• Question Answering with Subgraph Embeddings
• Multilingual Distributed Representations without Word Alignment
• Evaluating Neural Word Representations in Tensor-Based Compositional Settings

Cited by in corpus (7)

• Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Features
• xSense: Learning Sense-Separated Sparse Representations and Textual Definitions for Explainable Word Sense Networks
• Implementing a Reverse Dictionary, based on word definitions, using a Node-Graph Architecture
• A Large-Scale Multilingual Disambiguation of Glosses
• Learning Word Embeddings from Intrinsic and Extrinsic Views
• Towards a Transformer-Based Reverse Dictionary Model for Quality Estimation of Definitions
• TFW, DamnGina, Juvie, and Hotsie-Totsie: On the Linguistic and Social Aspects of Internet Slang