sense2vec - A Fast and Accurate Method for Word Sense Disambiguation In Neural Word Embeddings

Abstract

Neural word representations have proven useful in Natural Language Processing (NLP) tasks due to their ability to efficiently model complex semantic and syntactic word relationships. However, most techniques model only one representation per word, despite the fact that a single word can have multiple meanings or "senses". Some techniques model words by using multiple vectors that are clustered based on context. However, recent neural approaches rarely focus on the application to a consuming NLP algorithm. Furthermore, the training process of recent word-sense models is expensive relative to single-sense embedding processes. This paper presents a novel approach which addresses these concerns by modeling multiple embeddings for each word based on supervised disambiguation, which provides a fast and accurate way for a consuming NLP model to select a sense-disambiguated embedding. We demonstrate that these embeddings can disambiguate both contrastive senses such as nominal and verbal senses as well as nuanced senses such as sarcasm. We further evaluate Part-of-Speech disambiguated embeddings on neural dependency parsing, yielding a greater than 8% average error reduction in unlabeled attachment scores across 6 languages.

References in corpus (3)

• Efficient Estimation of Word Representations in Vector Space
• Exploiting Similarities among Languages for Machine Translation
• Polyglot: Distributed Word Representations for Multilingual NLP

Cited by in corpus (14)

• Distributed Representations of Signed Networks
• Query Expansion with Locally-Trained Word Embeddings
• Multi-sense embeddings through a word sense disambiguation process
• Network representation learning systematic review: ancestors and current development state
• ArgRewrite V.2: an Annotated Argumentative Revisions Corpus
• Overlapping Word Removal is All You Need: Revisiting Data Imbalance in Hope Speech Detection
• Zero-shot Fact Verification by Claim Generation
• Extracting Fairness Policies from Legal Documents
• Using Multi-Sense Vector Embeddings for Reverse Dictionaries
• SLAM-Inspired Simultaneous Contextualization and Interpreting for Incremental Conversation Sentences
• Cross-Discourse and Multilingual Exploration of Textual Corpora with the DualNeighbors Algorithm
• Generating Sense Embeddings for Syntactic and Semantic Analogy for Portuguese
• Adverse Media Mining for KYC and ESG Compliance
• Going Beyond T-SNE: Exposing \texttt{whatlies} in Text Embeddings