Analyzing the Limitations of Cross-lingual Word Embedding Mappings
arXiv:1906.05407 · doi:10.18653/v1/P19-1492
Abstract
Recent research in cross-lingual word embeddings has almost exclusively focused on offline methods, which independently train word embeddings in different languages and map them to a shared space through linear transformations. While several authors have questioned the underlying isomorphism assumption, which states that word embeddings in different languages have approximately the same structure, it is not clear whether this is an inherent limitation of mapping approaches or a more general issue when learning cross-lingual embeddings. So as to answer this question, we experiment with parallel corpora, which allows us to compare offline mapping to an extension of skip-gram that jointly learns both embedding spaces. We observe that, under these ideal conditions, joint learning yields to more isomorphic embeddings, is less sensitive to hubness, and obtains stronger results in bilingual lexicon induction. We thus conclude that current mapping methods do have strong limitations, calling for further research to jointly learn cross-lingual embeddings with a weaker cross-lingual signal.
ACL 2019
References in corpus (1)
Cited by in corpus (12)
- A Benchmarking Study of Embedding-based Entity Alignment for Knowledge Graphs
- When Does Unsupervised Machine Translation Work?
- Emerging Cross-lingual Structure in Pretrained Language Models
- Cross-lingual alignments of ELMo contextual embeddings
- Robust Cross-lingual Embeddings from Parallel Sentences
- Transferring Knowledge Distillation for Multilingual Social Event Detection
- FOCUS: Effective Embedding Initialization for Monolingual Specialization of Multilingual Models
- Understanding Cross-Lingual Syntactic Transfer in Multilingual Recurrent Neural Networks
- Isomorphic Cross-lingual Embeddings for Low-Resource Languages
- Beyond Offline Mapping: Learning Cross Lingual Word Embeddings through Context Anchoring
- Hubness Reduction Improves Sentence-BERT Semantic Spaces
- An Analysis of Euclidean vs. Graph-Based Framing for Bilingual Lexicon Induction from Word Embedding Spaces