A Survey of Word Embeddings Evaluation Methods

Abstract

Word embeddings are real-valued word representations able to capture lexical semantics and trained on natural language corpora. Models proposing these representations have gained popularity in the recent years, but the issue of the most adequate evaluation method still remains open. This paper presents an extensive overview of the field of word embeddings evaluation, highlighting main problems and proposing a typology of approaches to evaluation, summarizing 16 intrinsic methods and 12 extrinsic methods. I describe both widely-used and experimental methods, systematize information about evaluation datasets and discuss some key challenges.

arXiv comment26 pages

References in corpus (7)

• Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes
• A large annotated corpus for learning natural language inference
• Cross-lingual Models of Word Embeddings: An Empirical Comparison
• How to evaluate word embeddings? On importance of data efficiency and simple supervised tasks
• WordRep: A Benchmark for Research on Learning Word Representations
• An Exploration of Word Embedding Initialization in Deep-Learning Tasks
• Improving Reliability of Word Similarity Evaluation by Redesigning Annotation Task and Performance Measure

Cited by in corpus (20)

• Evaluating Word Embedding Models: Methods and Experimental Results
• Evaluation of sentence embeddings in downstream and linguistic probing tasks
• Representation Learning for Natural Language Processing
• Using natural language processing techniques to extract information on the properties and functionalities of energetic materials from large text corpora
• COS960: A Chinese Word Similarity Dataset of 960 Word Pairs
• Literature review on vulnerability detection using NLP technology
• An Automated, End-to-End Framework for Modeling Attacks From Vulnerability Descriptions
• Word Embeddings for Entity-annotated Texts
• Word Embeddings for Automatic Equalization in Audio Mixing
• A Survey On Neural Word Embeddings
• Evaluation of Morphological Embeddings for English and Russian Languages
• Word Embedding Algorithms as Generalized Low Rank Models and their Canonical Form
• Evaluation of Morphological Embeddings for the Russian Language
• Exploring the Combination of Contextual Word Embeddings and Knowledge Graph Embeddings
• Evaluating Neural Word Embeddings for Sanskrit
• Assessing the Effectiveness of Syntactic Structure to Learn Code Edit Representations
• Laplacian Eigenmaps with variational circuits: a quantum embedding of graph data
• Shareable Representations for Search Query Understanding
• Training Cross-Lingual embeddings for Setswana and Sepedi
• Sentence Embeddings for Russian NLU