A Survey of Word Embeddings Evaluation Methods

Abstract

Word embeddings are real-valued word representations able to capture lexical semantics and trained on natural language corpora. Models proposing these representations have gained popularity in the recent years, but the issue of the most adequate evaluation method still remains open. This paper presents an extensive overview of the field of word embeddings evaluation, highlighting main problems and proposing a typology of approaches to evaluation, summarizing 16 intrinsic methods and 12 extrinsic methods. I describe both widely-used and experimental methods, systematize information about evaluation datasets and discuss some key challenges.

26 pages

References in corpus (4)

• How to evaluate word embeddings? On importance of data efficiency and simple supervised tasks
• WordRep: A Benchmark for Research on Learning Word Representations
• An Exploration of Word Embedding Initialization in Deep-Learning Tasks
• Improving Reliability of Word Similarity Evaluation by Redesigning Annotation Task and Performance Measure

Cited by in corpus (16)

• Evaluating Word Embedding Models: Methods and Experimental Results
• Representation Learning for Natural Language Processing
• Using natural language processing techniques to extract information on the properties and functionalities of energetic materials from large text corpora
• COS960: A Chinese Word Similarity Dataset of 960 Word Pairs
• Literature review on vulnerability detection using NLP technology
• An Automated, End-to-End Framework for Modeling Attacks From Vulnerability Descriptions
• A Survey On Neural Word Embeddings
• Evaluation of Morphological Embeddings for English and Russian Languages
• Word Embedding Algorithms as Generalized Low Rank Models and their Canonical Form
• Exploring the Combination of Contextual Word Embeddings and Knowledge Graph Embeddings
• Evaluation of Morphological Embeddings for the Russian Language
• Assessing the Effectiveness of Syntactic Structure to Learn Code Edit Representations
• Evaluating Neural Word Embeddings for Sanskrit
• Laplacian Eigenmaps with variational circuits: a quantum embedding of graph data
• Shareable Representations for Search Query Understanding
• Sentence Embeddings for Russian NLU