Density Matching for Bilingual Word Embedding
arXiv:1904.02343
Abstract
Recent approaches to cross-lingual word embedding have generally been based on linear transformations between the sets of embedding vectors in the two languages. In this paper, we propose an approach that instead expresses the two monolingual embedding spaces as probability densities defined by a Gaussian mixture model, and matches the two densities using a method called normalizing flow. The method requires no explicit supervision, and can be learned with only a seed dictionary of words that have identical strings. We argue that this formulation has several intuitively attractive properties, particularly with the respect to improving robustness and generalization to mappings between difficult language pairs or word pairs. On a benchmark data set of bilingual lexicon induction and cross-lingual word similarity, our approach can achieve competitive or superior performance compared to state-of-the-art published results, with particularly strong results being found on etymologically distant and/or morphologically rich languages.
arXiv commentAccepted by NAACL-HLT 2019
References in corpus (13)
- Adam: A Method for Stochastic Optimization
- Exploiting Similarities among Languages for Machine Translation
- Variational Inference with Normalizing Flows
- Density estimation using Real NVP
- A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings
- Unsupervised Statistical Machine Translation
- BilBOWA: Fast Bilingual Distributed Representations without Word Alignments
- Word Translation Without Parallel Data
- Word Representations via Gaussian Embedding
- Universal Neural Machine Translation for Extremely Low Resource Languages
- Cross-lingual Models of Word Embeddings: An Empirical Comparison
- Unsupervised Alignment of Embeddings with Wasserstein Procrustes
- On Difficulties of Cross-Lingual Transfer with Order Differences: A Case Study on Dependency Parsing