Learning Multilingual Word Representations using a Bag-of-Words Autoencoder

Abstract

Recent work on learning multilingual word representations usually relies on the use of word-level alignements (e.g. infered with the help of GIZA++) between translated sentences, in order to align the word embeddings in different languages. In this workshop paper, we investigate an autoencoder model for learning multilingual word representations that does without such word-level alignements. The autoencoder is trained to reconstruct the bag-of-word representation of given sentence from an encoded representation extracted from its translation. We evaluate our approach on a multilingual document classification task, where labeled data is available only for one language (e.g. English) while classification must be performed in a different language (e.g. French). In our experiments, we observe that our method compares favorably with a previously proposed method that exploits word-level alignments to learn word representations.

This workshop paper was accepted on Octoble 30 2013 at the NIPS 2013 workshop on deep learning (https://sites.google.com/site/deeplearningworkshopnips2013/accepted-papers)

References in corpus (2)

• Natural Language Processing (almost) from Scratch
• Learning Semantic Representations for the Phrase Translation Model

Cited by in corpus (9)

• A Survey Of Cross-lingual Word Embedding Models
• Multilingual Models for Compositional Distributed Semantics
• Multilingual Distributed Representations without Word Alignment
• Separated by an Un-common Language: Towards Judgment Language Informed Vector Space Modeling
• Learning Bilingual Word Representations by Marginalizing Alignments
• Learning to Represent Words in Context with Multilingual Supervision
• A Deep Architecture for Semantic Parsing
• Distributed Representations for Compositional Semantics
• Feature extraction using Latent Dirichlet Allocation and Neural Networks: A case study on movie synopses