Beyond Bilingual: Multi-sense Word Embeddings using Multilingual Context
arXiv:1706.08160
Abstract
Word embeddings, which represent a word as a point in a vector space, have become ubiquitous to several NLP tasks. A recent line of work uses bilingual (two languages) corpora to learn a different vector for each sense of a word, by exploiting crosslingual signals to aid sense identification. We present a multi-view Bayesian non-parametric algorithm which improves multi-sense word embeddings by (a) using multilingual (i.e., more than two languages) corpora to significantly improve sense embeddings beyond what one achieves with bilingual information, and (b) uses a principled approach to learn a variable number of senses per word, in a data-driven manner. Ours is the first approach with the ability to leverage multilingual corpora efficiently for multi-sense representation learning. Experiments show that multilingual training significantly improves performance over monolingual and bilingual training, by allowing us to combine different parallel corpora to leverage multilingual context. Multilingual training yields comparable performance to a state of the art mono-lingual model trained on five times more training data.
ACL 2017 Repl4NLP workshop
References in corpus (4)
- Breaking Sticks and Ambiguities with Adaptive Skip-gram
- Cross-lingual Models of Word Embeddings: An Empirical Comparison
- A State of the Art of Word Sense Induction: A Way Towards Word Sense Disambiguation for Under-Resourced Languages
- Learning to Represent Words in Context with Multilingual Supervision
Cited by in corpus (6)
- Recent Trends in Deep Learning Based Natural Language Processing
- Learning Gender-Neutral Word Embeddings
- A Simple Approach to Learn Polysemous Word Embeddings
- Machine Learning Meets Natural Language Processing -- The story so far
- Towards Learning Cross-Modal Perception-Trace Models
- CLUSE: Cross-Lingual Unsupervised Sense Embeddings