Multilingual Distributed Representations without Word Alignment

Abstract

Distributed representations of meaning are a natural way to encode covariance relationships between words and phrases in NLP. By overcoming data sparsity problems, as well as providing information about semantic relatedness which is not available in discrete representations, distributed representations have proven useful in many NLP tasks. Recent work has shown how compositional semantic representations can successfully be applied to a number of monolingual applications such as sentiment analysis. At the same time, there has been some initial success in work on learning shared word-level representations across languages. We combine these two approaches by proposing a method for learning distributed representations in a multilingual setup. Our model learns to assign similar embeddings to aligned sentences and dissimilar ones to sentence which are not aligned while not requiring word alignments. We show that our representations are semantically informative and apply them to a cross-lingual document classification task where we outperform the previous state of the art. Further, by employing parallel corpora of multiple language pairs we find that our model learns representations that capture semantic relationships across languages for which no parallel data was used.

To appear at ICLR 2014

References in corpus (2)

• From Frequency to Meaning: Vector Space Models of Semantics
• Learning Multilingual Word Representations using a Bag-of-Words Autoencoder

Cited by in corpus (12)

• An efficient framework for learning sentence representations
• A Survey Of Cross-lingual Word Embedding Models
• Computational linguistics and Natural Language Processing
• A Novel Bilingual Word Embedding Method for Lexical Translation Using Bilingual Sense Clique
• Learning to Represent Words in Context with Multilingual Supervision
• Cross-Lingual Sentiment Analysis Without (Good) Translation
• Learning to Understand Phrases by Embedding the Dictionary
• Semi Supervised Preposition-Sense Disambiguation using Multilingual Data
• Towards cross-lingual distributed representations without parallel text trained with adversarial autoencoders
• Learning Embodied Semantics via Music and Dance Semiotic Correlations
• The Geometry of Distributed Representations for Better Alignment, Attenuated Bias, and Improved Interpretability
• Char-RNN for Word Stress Detection in East Slavic Languages