Capturing Evolution in Word Usage: Just Add More Clusters?
arXiv:2001.06629 · doi:10.1145/3366424.3382186
Abstract
The way the words are used evolves through time, mirroring cultural or technological evolution of society. Semantic change detection is the task of detecting and analysing word evolution in textual data, even in short periods of time. In this paper we focus on a new set of methods relying on contextualised embeddings, a type of semantic modelling that revolutionised the NLP field recently. We leverage the ability of the transformer-based BERT model to generate contextualised embeddings capable of detecting semantic change of words across time. Several approaches are compared in a common setting in order to establish strengths and weaknesses for each of them. We also propose several ideas for improvements, managing to drastically improve the performance of existing approaches.
References in corpus (9)
- Efficient Estimation of Word Representations in Vector Space
- Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
- Visualizing and Measuring the Geometry of BERT
- Analysing Lexical Semantic Change with Contextualised Word Representations
- Does BERT Make Any Sense? Interpretable Word Sense Disambiguation with Contextualized Embeddings
- Time-Out: Temporal Referencing for Robust Modeling of Lexical Semantic Change
- Towards better substitution-based word sense induction
- A State-of-the-Art of Semantic Change Computation
- Leveraging Contextual Embeddings for Detecting Diachronic Semantic Shift