Unsupervised Hyperalignment for Multilingual Word Embeddings

Abstract

We consider the problem of aligning continuous word representations, learned in multiple languages, to a common space. It was recently shown that, in the case of two languages, it is possible to learn such a mapping without supervision. This paper extends this line of work to the problem of aligning multiple languages to a common space. A solution is to independently map all languages to a pivot language. Unfortunately, this degrades the quality of indirect word translation. We thus propose a novel formulation that ensures composable mappings, leading to better alignments. We evaluate our method by jointly aligning word vectors in eleven languages, showing consistent improvement with indirect mappings while maintaining competitive performance on direct word translation.

arXiv commentICLR 2019

References in corpus (12)

• Distributed Representations of Words and Phrases and their Compositionality
• Natural Language Processing (almost) from Scratch
• Exploiting Similarities among Languages for Machine Translation
• A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings
• Enriching Word Vectors with Subword Information
• Near-linear time approximation algorithms for optimal transport via Sinkhorn iteration
• Word Translation Without Parallel Data
• Computational Optimal Transport
• Unsupervised Alignment of Embeddings with Wasserstein Procrustes
• Loss in Translation: Learning Bilingual Word Mapping with a Retrieval Criterion
• Non-Adversarial Unsupervised Word Translation
• Unsupervised Multilingual Word Embeddings

Cited by in corpus (12)

• A Benchmarking Study of Embedding-based Entity Alignment for Knowledge Graphs
• embComp: Visual Interactive Comparison of Vector Embeddings
• Quantitative stability of optimal transport maps and linearization of the 2-Wasserstein space
• Linear Time Sinkhorn Divergences using Positive Features
• Unsupervised Multilingual Alignment using Wasserstein Barycenter
• Regularized Optimal Transport is Ground Cost Adversarial
• Linear-Time Gromov Wasserstein Distances using Low Rank Couplings and Costs
• Regularity as Regularization: Smooth and Strongly Convex Brenier Potentials in Optimal Transport
• On a Novel Application of Wasserstein-Procrustes for Unsupervised Cross-Lingual Learning
• Nearly Tight Convergence Bounds for Semi-discrete Entropic Optimal Transport
• Semi-Supervised Bilingual Lexicon Induction with Two-way Interaction
• A Relaxed Matching Procedure for Unsupervised BLI