Revisiting the Context Window for Cross-lingual Word Embeddings

Abstract

Existing approaches to mapping-based cross-lingual word embeddings are based on the assumption that the source and target embedding spaces are structurally similar. The structures of embedding spaces largely depend on the co-occurrence statistics of each word, which the choice of context window determines. Despite this obvious connection between the context window and mapping-based cross-lingual embeddings, their relationship has been underexplored in prior work. In this work, we provide a thorough evaluation, in various languages, domains, and tasks, of bilingual embeddings trained with different context windows. The highlight of our findings is that increasing the size of both the source and target window sizes improves the performance of bilingual lexicon induction, especially the performance on frequent nouns.

arXiv commentACL2020

References in corpus (7)

• Efficient Estimation of Word Representations in Vector Space
• Distributed Representations of Words and Phrases and their Compositionality
• From Frequency to Meaning: Vector Space Models of Semantics
• Exploiting Similarities among Languages for Machine Translation
• A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings
• BilBOWA: Fast Bilingual Distributed Representations without Word Alignments
• Offline bilingual word vectors, orthogonal transformations and the inverted softmax

Cited by in corpus (1)

• Data Augmentation with Unsupervised Machine Translation Improves the Structural Similarity of Cross-lingual Word Embeddings