Semantic Specialisation of Distributional Word Vector Spaces using Monolingual and Cross-Lingual Constraints
arXiv:1706.00374
Abstract
We present Attract-Repel, an algorithm for improving the semantic quality of word vectors by injecting constraints extracted from lexical resources. Attract-Repel facilitates the use of constraints from mono- and cross-lingual resources, yielding semantically specialised cross-lingual vector spaces. Our evaluation shows that the method can make use of existing cross-lingual lexicons to construct high-quality vector spaces for a plethora of different languages, facilitating semantic transfer from high- to lower-resource ones. The effectiveness of our approach is demonstrated with state-of-the-art results on semantic similarity datasets in six languages. We next show that Attract-Repel-specialised vectors boost performance in the downstream task of dialogue state tracking (DST) across multiple languages. Finally, we show that cross-lingual vector spaces produced by our algorithm facilitate the training of multilingual DST models, which brings further performance improvements.
arXiv commentAccepted for publication at TACL (to be presented at EMNLP 2017)
References in corpus (10)
- Exploiting Similarities among Languages for Machine Translation
- Reasoning about Entailment with Neural Attention
- BilBOWA: Fast Bilingual Distributed Representations without Word Alignments
- A Dual Embedding Space Model for Document Ranking
- Multilingual Models for Compositional Distributed Semantics
- Separated by an Un-common Language: Towards Judgment Language Informed Vector Space Modeling
- Cross-lingual Models of Word Embeddings: An Empirical Comparison
- Charagram: Embedding Words and Sentences via Character n-grams
- Learning Crosslingual Word Embeddings without Bilingual Corpora
- Non-distributional Word Vector Representations
Cited by in corpus (5)
- Neural Natural Language Inference Models Enhanced with External Knowledge
- A Survey Of Cross-lingual Word Embedding Models
- Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning
- XL-NBT: A Cross-lingual Neural Belief Tracking Framework
- Conformal retrofitting via Riemannian manifolds: distilling task-specific graphs into pretrained embeddings