Are Girls Neko or Shōjo? Cross-Lingual Alignment of Non-Isomorphic Embeddings with Iterative Normalization
arXiv:1906.01622
Abstract
Cross-lingual word embeddings (CLWE) underlie many multilingual natural language processing systems, often through orthogonal transformations of pre-trained monolingual embeddings. However, orthogonal mapping only works on language pairs whose embeddings are naturally isomorphic. For non-isomorphic pairs, our method (Iterative Normalization) transforms monolingual embeddings to make orthogonal alignment easier by simultaneously enforcing that (1) individual word vectors are unit length, and (2) each language's average vector is zero. Iterative Normalization consistently improves word translation accuracy of three CLWE methods, with the largest improvement observed on English-Japanese (from 2% to 44% test accuracy).
arXiv commentACL 2019
References in corpus (7)
- Distributed Representations of Words and Phrases and their Compositionality
- Exploiting Similarities among Languages for Machine Translation
- A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings
- Offline bilingual word vectors, orthogonal transformations and the inverted softmax
- Towards Optimal Transport with Global Invariances
- A Resource-Free Evaluation Metric for Cross-Lingual Word Embeddings Based on Graph Modularity
- Convergence Analysis of Alternating Projection Method for Nonconvex Sets