An Autoencoder Approach to Learning Bilingual Word Representations
arXiv:1402.1454
Abstract
Cross-language learning allows us to use training data from one language to build models for a different language. Many approaches to bilingual learning require that we have word-level alignment of sentences from parallel corpora. In this work we explore the use of autoencoder-based methods for cross-language learning of vectorial word representations that are aligned between two languages, while not relying on word-level alignments. We show that by simply learning to reconstruct the bag-of-words representations of aligned sentences, within and between languages, we can in fact learn high-quality representations and do without word alignments. Since training autoencoders on word observations presents certain computational issues, we propose and compare different variations adapted to this setting. We also propose an explicit correlation maximizing regularizer that leads to significant improvement in the performance. We empirically investigate the success of our approach on the problem of cross-language test classification, where a classifier trained on a given language (e.g., English) must learn to generalize to a different language (e.g., German). These experiments demonstrate that our approaches are competitive with the state-of-the-art, achieving up to 10-14 percentage point improvements over the best reported results on this task.
10 pages
References in corpus (3)
Cited by in corpus (49)
- Text Classification Algorithms: A Survey
- BilBOWA: Fast Bilingual Distributed Representations without Word Alignments
- A Survey Of Cross-lingual Word Embedding Models
- On the Feasibility of Transfer-learning Code Smells using Deep Learning
- Detection of Anomalies in Large Scale Accounting Data using Deep Autoencoder Networks
- Separated by an Un-common Language: Towards Judgment Language Informed Vector Space Modeling
- Cross-lingual Models of Word Embeddings: An Empirical Comparison
- A Multiplicative Model for Learning Distributed Text-Based Attribute Representations
- A Comprehensive Survey of Multilingual Neural Machine Translation
- A Novel Bilingual Word Embedding Method for Lexical Translation Using Bilingual Sense Clique
- Learning Translations via Matrix Completion
- Leveraging Monolingual Data for Crosslingual Compositional Word Representations
- On the Robustness of Unsupervised and Semi-supervised Cross-lingual Word Embedding Learning
- Stochastic Optimization for Deep CCA via Nonlinear Orthogonal Iterations
- Learning to Understand Phrases by Embedding the Dictionary
- Neural Cross-Lingual Named Entity Recognition with Minimal Resources
- Robust Cross-lingual Embeddings from Parallel Sentences
- Exploiting Deep Learning for Persian Sentiment Analysis
- Bilingual Learning of Multi-sense Embeddings with Discrete Autoencoders
- Multilingual Visual Sentiment Concept Matching
- Image search using multilingual texts: a cross-modal learning approach between image and text
- Topic-Preserving Synthetic News Generation: An Adversarial Deep Reinforcement Learning Approach
- Emoji-Powered Representation Learning for Cross-Lingual Sentiment Classification
- Multilingual Knowledge Graph Embeddings for Cross-lingual Knowledge Alignment
- A Variational Prosody Model for Mapping the Context-Sensitive Variation of Functional Prosodic Prototypes
- A Neural Network Approach for Mixing Language Models
- Zero and Few Shot Learning with Semantic Feature Synthesis and Competitive Learning
- A Brief Survey of Multilingual Neural Machine Translation
- Learning Dynamics of Linear Denoising Autoencoders
- Transformer based Multilingual document Embedding model
- Correlational Neural Networks
- Embedding Learning Through Multilingual Concept Induction
- Hierarchical Prototype Learning for Zero-Shot Recognition
- A Correlational Encoder Decoder Architecture for Pivot Based Sequence Generation
- Multilingual Embeddings Jointly Induced from Contexts and Concepts: Simple, Strong and Scalable
- Joint Representation Learning of Cross-lingual Words and Entities via Attentive Distant Supervision
- Duality Regularization for Unsupervised Bilingual Lexicon Induction
- Weakly-Supervised Concept-based Adversarial Learning for Cross-lingual Word Embeddings
- Characterizing Departures from Linearity in Word Translation
- Multi-lingual Common Semantic Space Construction via Cluster-consistent Word Embedding
- NMT-based Cross-lingual Document Embeddings
- Variational learning across domains with triplet information
- Diagnosis and Analysis of Celiac Disease and Environmental Enteropathy on Biopsy Images using Deep Learning Approaches
- Autoencoders for strategic decision support
- CLUSE: Cross-Lingual Unsupervised Sense Embeddings
- Local Deep-Feature Alignment for Unsupervised Dimension Reduction
- Bilingual Dictionary Induction for Bantu Languages
- ProLFA: Representative Prototype Selection for Local Feature Aggregation
- Learning to Represent Bilingual Dictionaries