Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change
arXiv:1605.09096
Abstract
Understanding how words change their meanings over time is key to models of language and cultural evolution, but historical data on meaning is scarce, making theories hard to develop and test. Word embeddings show promise as a diachronic tool, but have not been carefully evaluated. We develop a robust methodology for quantifying semantic change by evaluating word embeddings (PPMI, SVD, word2vec) against known historical changes. We then use this methodology to reveal statistical laws of semantic evolution. Using six historical corpora spanning four languages and two centuries, we propose two quantitative laws of semantic change: (i) the law of conformity---the rate of semantic change scales with an inverse power-law of word frequency; (ii) the law of innovation---independent of frequency, words that are more polysemous have higher rates of semantic change.
Association for Computational Linguistics (ACL), 2016. Minor corrections; improved methodology for Section 3
References in corpus (2)
Cited by in corpus (41)
- Inductive Representation Learning on Large Graphs
- DynGEM: Deep Embedding Method for Dynamic Graphs
- On the Dimensionality of Word Embedding
- Leveraging Contextual Embeddings for Detecting Diachronic Semantic Shift
- Inducing Domain-Specific Sentiment Lexicons from Unlabeled Corpora
- Cultural Shift or Linguistic Drift? Comparing Two Computational Measures of Semantic Change
- Multi-level computational methods for interdisciplinary research in the HathiTrust Digital Library
- LMMS Reloaded: Transformer-based Sense Embeddings for Disambiguation and Beyond
- MILE: A Multi-Level Framework for Scalable Graph Embedding
- Evolution of emotion semantics
- Simulating Lexical Semantic Change from Sense-Annotated Data
- Words are Malleable: Computing Semantic Shifts in Political and Media Discourse
- Semantic Variation in Online Communities of Practice
- Structured Black Box Variational Inference for Latent Time Series Models
- Leveraging Prompt-Based Large Language Models: Predicting Pandemic Health Decisions and Outcomes Through Social Media Language
- The Road to Success: Assessing the Fate of Linguistic Innovations in Online Communities
- Domain Adaptation for Named Entity Recognition in Online Media with Word Embeddings
- Structured Embedding Models for Grouped Data
- A Computational Approach to Historical Ontologies
- Language classification from bilingual word embedding graphs
- Communicative need modulates competition in language change
- Second-Order Word Embeddings from Nearest Neighbor Topological Features
- Aligning Vector-spaces with Noisy Supervised Lexicons
- Will This Idea Spread Beyond Academia? Understanding Knowledge Transfer of Scientific Concepts across Text Corpora
- Gender Bias, Social Bias and Representation: 70 Years of Bollywood
- Recurrent Point Review Models
- Enriching Word Embeddings with Temporal and Spatial Information
- The Typology of Polysemy: A Multilingual Distributional Framework
- Shared task: Lexical semantic change detection in German (Student Project Report)
- Creation and Evaluation of Datasets for Distributional Semantics Tasks in the Digital Humanities Domain
- Relation Extraction Datasets in the Digital Humanities Domain and their Evaluation with Word Embeddings
- Measuring diachronic sense change: new models and Monte Carlo methods for Bayesian inference
- SST-BERT at SemEval-2020 Task 1: Semantic Shift Tracing by Clustering in BERT-based Embedding Spaces
- SChME at SemEval-2020 Task 1: A Model Ensemble for Detecting Lexical Semantic Change
- The presence of occupational structure in online texts based on word embedding NLP models
- Generating Timelines by Modeling Semantic Change
- Diachronic Text Mining Investigation of Therapeutic Candidates for COVID-19
- Event-Driven Query Expansion
- Temporal Graph Offset Reconstruction: Towards Temporally Robust Graph Representation Learning
- Detecting Reliable Novel Word Senses: A Network-Centric Approach
- Tracing metaphors in time through self-distance in vector spaces