Word Translation Without Parallel Data

Abstract

State-of-the-art methods for learning cross-lingual word embeddings have relied on bilingual dictionaries or parallel corpora. Recent studies showed that the need for parallel data supervision can be alleviated with character-level information. While these methods showed encouraging results, they are not on par with their supervised counterparts and are limited to pairs of languages sharing a common alphabet. In this work, we show that we can build a bilingual dictionary between two languages without using any parallel corpora, by aligning monolingual word embedding spaces in an unsupervised way. Without using any character information, our model even outperforms existing supervised methods on cross-lingual tasks for some language pairs. Our experiments demonstrate that our method works very well also for distant language pairs, like English-Russian or English-Chinese. We finally describe experiments on the English-Esperanto low-resource language pair, on which there only exists a limited amount of parallel data, to show the potential impact of our method in fully unsupervised machine translation. Our code, embeddings and dictionaries are publicly available.

ICLR 2018

References in corpus (7)

• Exploiting Similarities among Languages for Machine Translation
• NIPS 2016 Tutorial: Generative Adversarial Networks
• BilBOWA: Fast Bilingual Distributed Representations without Word Alignments
• Massively Multilingual Word Embeddings
• Parseval Networks: Improving Robustness to Adversarial Examples
• Learning Crosslingual Word Embeddings without Bilingual Corpora
• Identifying Word Translations in Non-Parallel Texts

Cited by in corpus (117)

• Multilingual Denoising Pre-training for Neural Machine Translation
• Unsupervised Machine Translation Using Monolingual Corpora Only
• Style Transfer as Unsupervised Machine Translation
• One-Shot Unsupervised Cross Domain Translation
• Does Object Recognition Work for Everyone?
• Multi-modal Sarcasm Detection and Humor Classification in Code-mixed Conversations
• Unsupervised Hyperalignment for Multilingual Word Embeddings
• XLDA: Cross-Lingual Data Augmentation for Natural Language Inference and Question Answering
• Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT
• Multiple-Attribute Text Style Transfer
• Cross-lingual Zero- and Few-shot Hate Speech Detection Utilising Frozen Transformer Language Models and AXEL
• Predicting Concreteness and Imageability of Words Within and Across Languages via Word Embeddings
• Deep Learning Models for Multilingual Hate Speech Detection
• Continuous Graph Neural Networks
• Unsupervised Alignment of Embeddings with Wasserstein Procrustes
• A Study of Neural Matching Models for Cross-lingual IR
• Extracting Parallel Sentences with Bidirectional Recurrent Neural Networks to Improve Machine Translation
• SpeechBERT: An Audio-and-text Jointly Learned Language Model for End-to-end Spoken Question Answering
• Low-Resource Text Classification using Domain-Adversarial Learning
• Unsupervised Multimodal Representation Learning across Medical Images and Reports
• Emerging Cross-lingual Structure in Pretrained Language Models
• Structural-analogy from a Single Image Pair
• Teaching a New Dog Old Tricks: Resurrecting Multilingual Retrieval Using Zero-shot Learning
• Loss in Translation: Learning Bilingual Word Mapping with a Retrieval Criterion
• Integrating Social Media into a Pan-European Flood Awareness System: A Multilingual Approach
• Unsupervised Cross-lingual Word Embedding by Multilingual Neural Language Models
• xSense: Learning Sense-Separated Sparse Representations and Textual Definitions for Explainable Word Sense Networks
• Exploring Fine-tuning Techniques for Pre-trained Cross-lingual Models via Continual Learning
• DICT-MLM: Improved Multilingual Pre-Training using Bilingual Dictionaries
• Cross-Lingual Transfer Learning for Multilingual Task Oriented Dialog
• Non-Autoregressive Neural Machine Translation with Enhanced Decoder Input
• Cross-Lingual Named Entity Recognition Using Parallel Corpus: A New Approach Using XLM-RoBERTa Alignment
• Non-Adversarial Unsupervised Word Translation
• Crosslingual Document Embedding as Reduced-Rank Ridge Regression
• QUEACO: Borrowing Treasures from Weakly-labeled Behavior Data for Query Attribute Value Extraction
• Towards Unsupervised Automatic Speech Recognition Trained by Unaligned Speech and Text only
• Deep Matching Autoencoders
• word2word: A Collection of Bilingual Lexicons for 3,564 Language Pairs
• Leveraging Subword Embeddings for Multinational Address Parsing
• Detecting Social Media Manipulation in Low-Resource Languages
• Unsupervised Multilingual Alignment using Wasserstein Barycenter
• Adversarial Removal of Demographic Attributes from Text Data
• Learning Unsupervised Word Mapping by Maximizing Mean Discrepancy
• Unsupervised Sentiment Analysis for Code-mixed Data
• Combining Pre-trained Word Embeddings and Linguistic Features for Sequential Metaphor Identification
• PidginUNMT: Unsupervised Neural Machine Translation from West African Pidgin to English
• Updating Pre-trained Word Vectors and Text Classifiers using Monolingual Alignment
• Neural Machine Translation: A Review of Methods, Resources, and Tools
• Linear-Time Gromov Wasserstein Distances using Low Rank Couplings and Costs
• Filtered Inner Product Projection for Crosslingual Embedding Alignment
• Leveraging Pretrained Word Embeddings for Part-of-Speech Tagging of Code Switching Data
• An Empirical Study on Post-processing Methods for Word Embeddings
• Cross-Lingual Transfer Learning for Question Answering
• Gender Bias in Multilingual Embeddings and Cross-Lingual Transfer
• Churn Intent Detection in Multilingual Chatbot Conversations and Social Media
• Zero-Resource Cross-Domain Named Entity Recognition
• A Robust Self-Learning Method for Fully Unsupervised Cross-Lingual Mappings of Word Embeddings: Making the Method Robustly Reproducible as Well
• Towards Zero-shot Cross-lingual Image Retrieval and Tagging
• Reference Language based Unsupervised Neural Machine Translation
• Contextual Lensing of Universal Sentence Representations
• Cross-Lingual Contextual Word Embeddings Mapping With Multi-Sense Words In Mind
• The 2021 Image Similarity Dataset and Challenge
• Unsupervised cross-lingual matching of product classifications
• Translating the Unseen? Yoruba-English MT in Low-Resource, Morphologically-Unmarked Settings
• Compass-aligned Distributional Embeddings for Studying Semantic Differences across Corpora
• Neural Decipherment via Minimum-Cost Flow: from Ugaritic to Linear B
• Towards Multi-Sense Cross-Lingual Alignment of Contextual Embeddings
• XL-NBT: A Cross-lingual Neural Belief Tracking Framework
• Towards Reducing Bias in Gender Classification
• Revisiting Language Encoding in Learning Multilingual Representations
• Discovering Bilingual Lexicons in Polyglot Word Embeddings
• Aligning Vector-spaces with Noisy Supervised Lexicons
• SemEval-2019 Task 1: Cross-lingual Semantic Parsing with UCCA
• A Survey on Low-Resource Neural Machine Translation
• Traceability Support for Multi-Lingual Software Projects
• Low-Resource Sequence Labeling via Unsupervised Multilingual Contextualized Representations
• SAR: Learning Cross-Language API Mappings with Little Knowledge
• Improving Candidate Generation for Low-resource Cross-lingual Entity Linking
• Worse WER, but Better BLEU? Leveraging Word Embedding as Intermediate in Multitask End-to-End Speech Translation
• Entity Projection via Machine Translation for Cross-Lingual NER
• BERT Goes Shopping: Comparing Distributional Models for Product Representations
• Density Matching for Bilingual Word Embedding
• A Case Study and Qualitative Analysis of Simple Cross-Lingual Opinion Mining
• The Typology of Polysemy: A Multilingual Distributional Framework
• An exploration of the encoding of grammatical gender in word embeddings
• Shared task: Lexical semantic change detection in German (Student Project Report)
• Bilingual Dictionary-based Language Model Pretraining for Neural Machine Translation
• MaskParse@Deskin at SemEval-2019 Task 1: Cross-lingual UCCA Semantic Parsing using Recursive Masked Sequence Tagging
• Unsupervised Object Matching for Relational Data
• Multilingual, Temporal and Sentimental Distant-Reading of City Events
• Unsupervised Transfer Learning in Multilingual Neural Machine Translation with Cross-Lingual Word Embeddings
• Word Embedding Transformation for Robust Unsupervised Bilingual Lexicon Induction
• Adversarial point set registration
• Multi-lingual Common Semantic Space Construction via Cluster-consistent Word Embedding
• Sentence transition matrix: An efficient approach that preserves sentence semantics
• Cross-lingual hate speech detection based on multilingual domain-specific word embeddings
• Contrastive Language Adaptation for Cross-Lingual Stance Detection
• DeepSubQE: Quality estimation for subtitle translations
• Fast Search with Poor OCR
• Explicit Cross-lingual Pre-training for Unsupervised Machine Translation
• Phonetic-and-Semantic Embedding of Spoken Words with Applications in Spoken Content Retrieval
• Cross-lingual transfer learning for spoken language understanding
• CLAR: A Cross-Lingual Argument Regularizer for Semantic Role Labeling
• Unsupervised Word Translation Pairing using Refinement based Point Set Registration
• Event-Driven Query Expansion
• Constrained Non-Affine Alignment of Embeddings
• Open Named Entity Modeling from Embedding Distribution
• A survey of cross-lingual features for zero-shot cross-lingual semantic parsing
• UniSent: Universal Adaptable Sentiment Lexica for 1000+ Languages
• Learning aligned embeddings for semi-supervised word translation using Maximum Mean Discrepancy
• CLUSE: Cross-Lingual Unsupervised Sense Embeddings
• Mapping Supervised Bilingual Word Embeddings from English to low-resource languages
• Language Model-Driven Unsupervised Neural Machine Translation
• Wasserstein distances for evaluating cross-lingual embeddings
• Cross-lingual Word Embeddings beyond Zero-shot Machine Translation
• Target-Oriented Fine-tuning for Zero-Resource Named Entity Recognition
• From Machine Translation to Code-Switching: Generating High-Quality Code-Switched Text