Breaking Sticks and Ambiguities with Adaptive Skip-gram
arXiv:1502.07257
Abstract
Recently proposed Skip-gram model is a powerful method for learning high-dimensional word representations that capture rich semantic relationships between words. However, Skip-gram as well as most prior work on learning word representations does not take into account word ambiguity and maintain only single representation per word. Although a number of Skip-gram modifications were proposed to overcome this limitation and learn multi-prototype word representations, they either require a known number of word meanings or learn them using greedy heuristic approaches. In this paper we propose the Adaptive Skip-gram model which is a nonparametric Bayesian extension of Skip-gram capable to automatically learn the required number of representations for all words at desired semantic resolution. We derive efficient online variational learning algorithm for the model and empirically demonstrate its efficiency on word-sense induction task.
References in corpus (3)
Cited by in corpus (27)
- Diachronic word embeddings and semantic shifts: a survey
- Breaking Sticks and Ambiguities with Adaptive Skip-gram
- Jointly Learning Word Embeddings and Latent Topics
- RUSSE'2018: A Shared Task on Word Sense Induction for the Russian Language
- Beyond Bilingual: Multi-sense Word Embeddings using Multilingual Context
- Vector representations of text data in deep learning
- How much does a word weigh? Weighting word embeddings for word sense induction
- Bilingual Learning of Multi-sense Embeddings with Discrete Autoencoders
- Improving Interpretability of Word Embeddings by Generating Definition and Usage
- Sequence embeddings help to identify fraudulent cases in healthcare insurance
- Mark my Word: A Sequence-to-Sequence Approach to Definition Modeling
- MUSE: Modularizing Unsupervised Sense Embeddings
- Combining Neural Language Models for WordSense Induction
- Word Sense Disambiguation for 158 Languages using Word Embeddings Only
- Inducing and Embedding Senses with Scaled Gumbel Softmax
- Multi-sense Definition Modeling using Word Sense Decompositions
- Efficient Graph-based Word Sense Induction by Distributional Inclusion Vector Embeddings
- Data-driven models and computational tools for neurolinguistics: a language technology perspective
- Zero-Shot Clinical Acronym Expansion via Latent Meaning Cells
- Toward Cross-Lingual Definition Generation for Language Learners
- BCWS: Bilingual Contextual Word Similarity
- Russian word sense induction by clustering averaged word embeddings
- Conditional Generators of Words Definitions
- Measuring diachronic sense change: new models and Monte Carlo methods for Bayesian inference
- DialectGram: Detecting Dialectal Variation at Multiple Geographic Resolutions
- Improving Neural Sequence Labelling using Additional Linguistic Information
- CLUSE: Cross-Lingual Unsupervised Sense Embeddings