Temporal Analysis of Language through Neural Language Models

Abstract

We provide a method for automatically detecting change in language across time through a chronologically trained neural language model. We train the model on the Google Books Ngram corpus to obtain word vector representations specific to each year, and identify words that have changed significantly from 1900 to 2009. The model identifies words such as "cell" and "gay" as having changed during that time period. The model simultaneously identifies the specific years during which such words underwent change.

Cited by in corpus (15)

• Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change
• Dynamic Word Embeddings
• Leveraging Contextual Embeddings for Detecting Diachronic Semantic Shift
• Cultural Shift or Linguistic Drift? Comparing Two Computational Measures of Semantic Change
• Semantic Journeys: Quantifying Change in Emoji Meaning from 2012-2018
• Freshman or Fresher? Quantifying the Geographic Variation of Internet Language
• Structured Black Box Variational Inference for Latent Time Series Models
• Statistically Significant Detection of Linguistic Change
• Structured Embedding Models for Grouped Data
• dynnode2vec: Scalable Dynamic Network Embedding
• SST-BERT at SemEval-2020 Task 1: Semantic Shift Tracing by Clustering in BERT-based Embedding Spaces
• CoVeR: Learning Covariate-Specific Vector Representations with Tensor Decompositions
• Temporal Graph Offset Reconstruction: Towards Temporally Robust Graph Representation Learning
• Tracing metaphors in time through self-distance in vector spaces
• An Improved Historical Embedding without Alignment