Diachronic word embeddings and semantic shifts: a survey

Abstract

Recent years have witnessed a surge of publications aimed at tracing temporal changes in lexical semantics using distributional methods, particularly prediction-based word embedding models. However, this vein of research lacks the cohesion, common terminology and shared practices of more established areas of natural language processing. In this paper, we survey the current state of academic research related to diachronic word embeddings and semantic shifts detection. We start with discussing the notion of semantic shifts, and then continue with an overview of the existing methods for tracing such time-related shifts with word embedding models. We propose several axes along which these methods can be compared, and outline the main challenges before this emerging subfield of NLP, as well as prospects and possible applications.

Proceedings of COLING 2018

References in corpus (6)

• Efficient Estimation of Word Representations in Vector Space
• Distributed Representations of Words and Phrases and their Compositionality
• From Frequency to Meaning: Vector Space Models of Semantics
• Dynamic Word Embeddings
• Breaking Sticks and Ambiguities with Adaptive Skip-gram
• Modeling the dynamics of domain specific terminology in diachronic corpora

Cited by in corpus (33)

• COVID-19 sentiment analysis via deep learning during the rise of novel cases
• Analysing Lexical Semantic Change with Contextualised Word Representations
• Time-Out: Temporal Referencing for Robust Modeling of Lexical Semantic Change
• Survey of Computational Approaches to Lexical Semantic Change
• Bias in Data-driven AI Systems -- An Introductory Survey
• Statistically significant detection of semantic shifts using contextual word embeddings
• Semantic Journeys: Quantifying Change in Emoji Meaning from 2012-2018
• Same Words, Different Meanings: Semantic Polarization in Broadcast Media Language Forecasts Polarization on Social Media Discourse
• Temporal Embeddings and Transformer Models for Narrative Text Understanding
• Lexical semantic change for Ancient Greek and Latin
• Tracing cultural diachronic semantic shifts in Russian using word embeddings: test sets and baselines
• Dynamic Network Embeddings for Network Evolution Analysis
• A Survey On Neural Word Embeddings
• Compass-aligned Distributional Embeddings for Studying Semantic Differences across Corpora
• Term Interrelations and Trends in Software Engineering
• Time Masking for Temporal Language Models
• Back to the Future -- Sequential Alignment of Text Representations
• DRIFT: A Toolkit for Diachronic Analysis of Scientific Literature
• Autoencoding Word Representations through Time for Semantic Change Detection
• SChME at SemEval-2020 Task 1: A Model Ensemble for Detecting Lexical Semantic Change
• Learning Neural Models for Natural Language Processing in the Face of Distributional Shift
• Empirical Study of Diachronic Word Embeddings for Scarce Data
• Towards Explainable Fact Checking
• We went to look for meaning and all we got were these lousy representations: aspects of meaning representation for computational semantics
• Grammatical Profiling for Semantic Change Detection
• Diachronic Text Mining Investigation of Therapeutic Candidates for COVID-19
• Generating Timelines by Modeling Semantic Change
• Short-Term Meaning Shift: A Distributional Exploration
• Learning language variations in news corpora through differential embeddings
• Word Embeddings: Stability and Semantic Change
• Measuring prominence of scientific work in online news as a proxy for impact
• Semantic Change and Emerging Tropes In a Large Corpus of New High German Poetry
• Towards Using Diachronic Distributed Word Representations as Models of Lexical Development