Filtered Inner Product Projection for Crosslingual Embedding Alignment

Abstract

Due to widespread interest in machine translation and transfer learning, there are numerous algorithms for mapping multiple embeddings to a shared representation space. Recently, these algorithms have been studied in the setting of bilingual dictionary induction where one seeks to align the embeddings of a source and a target language such that translated word pairs lie close to one another in a common representation space. In this paper, we propose a method, Filtered Inner Product Projection (FIPP), for mapping embeddings to a common representation space and evaluate FIPP in the context of bilingual dictionary induction. As semantic shifts are pervasive across languages and domains, FIPP first identifies the common geometric structure in both embeddings and then, only on the common structure, aligns the Gram matrices of these embeddings. Unlike previous approaches, FIPP is applicable even when the source and target embeddings are of differing dimensionalities. We show that our approach outperforms existing methods on the MUSE dataset for various language pairs. Furthermore, FIPP provides computational benefits both in ease of implementation and scalability.

arXiv commentICLR 2021

References in corpus (11)

• Distributed Representations of Words and Phrases and their Compositionality
• Exploiting Similarities among Languages for Machine Translation
• Enhanced LSTM for Natural Language Inference
• On the Cross-lingual Transferability of Monolingual Representations
• A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings
• Offline bilingual word vectors, orthogonal transformations and the inverted softmax
• Word Translation Without Parallel Data
• On the Dimensionality of Word Embedding
• Iterative Methods for Computing Eigenvalues and Eigenvectors
• A Novel Bilingual Word Embedding Method for Lexical Translation Using Bilingual Sense Clique
• The Global Anchor Method for Quantifying Linguistic Shifts and Domain Adaptation

Cited by in corpus (3)

• Improving Word Translation via Two-Stage Contrastive Learning
• Improving Bilingual Lexicon Induction with Cross-Encoder Reranking
• On Bilingual Lexicon Induction with Large Language Models