Unsupervised Hyperalignment for Multilingual Word Embeddings

Abstract

We consider the problem of aligning continuous word representations, learned in multiple languages, to a common space. It was recently shown that, in the case of two languages, it is possible to learn such a mapping without supervision. This paper extends this line of work to the problem of aligning multiple languages to a common space. A solution is to independently map all languages to a pivot language. Unfortunately, this degrades the quality of indirect word translation. We thus propose a novel formulation that ensures composable mappings, leading to better alignments. We evaluate our method by jointly aligning word vectors in eleven languages, showing consistent improvement with indirect mappings while maintaining competitive performance on direct word translation.

ICLR 2019

References in corpus (3)

• Natural Language Processing (almost) from Scratch
• Word Translation Without Parallel Data
• Unsupervised Multilingual Word Embeddings

Cited by in corpus (12)

• A Benchmarking Study of Embedding-based Entity Alignment for Knowledge Graphs
• embComp: Visual Interactive Comparison of Vector Embeddings
• Quantitative stability of optimal transport maps and linearization of the 2-Wasserstein space
• Linear Time Sinkhorn Divergences using Positive Features
• Unsupervised Multilingual Alignment using Wasserstein Barycenter
• Regularized Optimal Transport is Ground Cost Adversarial
• Linear-Time Gromov Wasserstein Distances using Low Rank Couplings and Costs
• Regularity as Regularization: Smooth and Strongly Convex Brenier Potentials in Optimal Transport
• On a Novel Application of Wasserstein-Procrustes for Unsupervised Cross-Lingual Learning
• Nearly Tight Convergence Bounds for Semi-discrete Entropic Optimal Transport
• Semi-Supervised Bilingual Lexicon Induction with Two-way Interaction
• A Relaxed Matching Procedure for Unsupervised BLI