Time-Out: Temporal Referencing for Robust Modeling of Lexical Semantic Change

Abstract

State-of-the-art models of lexical semantic change detection suffer from noise stemming from vector space alignment. We have empirically tested the Temporal Referencing method for lexical semantic change and show that, by avoiding alignment, it is less affected by this noise. We show that, trained on a diachronic corpus, the skip-gram with negative sampling architecture with temporal referencing outperforms alignment models on a synthetic task as well as a manual testset. We introduce a principled way to simulate lexical semantic change and systematically control for possible biases.

To appear in the 57th Annual Meeting of the Association for Computational Linguistics (ACL2019)

References in corpus (5)

• Distributed Representations of Words and Phrases and their Compositionality
• Dynamic Word Embeddings
• Diachronic word embeddings and semantic shifts: a survey
• Survey of Computational Approaches to Lexical Semantic Change
• GASC: Genre-Aware Semantic Change for Ancient Greek

Cited by in corpus (20)

• Time-Aware Language Models as Temporal Knowledge Bases
• Capturing Evolution in Word Usage: Just Add More Clusters?
• A Survey on Contextualised Semantic Shift Detection
• GASC: Genre-Aware Semantic Change for Ancient Greek
• Statistically significant detection of semantic shifts using contextual word embeddings
• Challenges in detecting evolutionary forces in language change using diachronic corpora
• Challenges for Computational Lexical Semantic Change
• Semantic Journeys: Quantifying Change in Emoji Meaning from 2012-2018
• Simulating Lexical Semantic Change from Sense-Annotated Data
• Let's Play Mono-Poly: BERT Can Reveal Words' Polysemy Level and Partitionability into Senses
• From Plenipotentiary to Puddingless: Users and Uses of New Words in Early English Letters
• Lexical semantic change for Ancient Greek and Latin
• SuperSim: a test set for word similarity and relatedness in Swedish
• Time Masking for Temporal Language Models
• Conceptual similarity and communicative need shape colexification: an experimental study
• Enriching Word Embeddings with Temporal and Spatial Information
• Autoencoding Word Representations through Time for Semantic Change Detection
• Measuring diachronic sense change: new models and Monte Carlo methods for Bayesian inference
• Shared task: Lexical semantic change detection in German (Student Project Report)
• Semantic coordinates analysis reveals language changes in the AI field