Exploiting Similarities among Languages for Machine Translation

Abstract

Dictionaries and phrase tables are the basis of modern statistical machine translation systems. This paper develops a method that can automate the process of generating and extending dictionaries and phrase tables. Our method can translate missing word and phrase entries by learning language structures based on large monolingual data and mapping between languages from small bilingual data. It uses distributed representation of words and learns a linear mapping between vector spaces of languages. Despite its simplicity, our method is surprisingly effective: we can achieve almost 90% precision@5 for translation of words between English and Spanish. This method makes little assumption about the languages, so it can be used to extend and refine dictionaries and translation tables for any language pairs.

References in corpus (2)

• Efficient Estimation of Word Representations in Vector Space
• Natural Language Processing (almost) from Scratch

Cited by in corpus (274)

• Distributed Representations of Sentences and Documents
• Recurrent Neural Network Regularization
• Cross-lingual Language Model Pretraining
• Skip-Thought Vectors
• Multilingual Denoising Pre-training for Neural Machine Translation
• A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings
• Text Understanding from Scratch
• Unsupervised Statistical Machine Translation
• An Autoencoder Approach to Learning Bilingual Word Representations
• BilBOWA: Fast Bilingual Distributed Representations without Word Alignments
• XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization
• Massively Multilingual Word Embeddings
• A Survey Of Cross-lingual Word Embedding Models
• Word Translation Without Parallel Data
• AutoExtend: Extending Word Embeddings to Embeddings for Synsets and Lexemes
• Multilingual Alignment of Contextual Word Representations
• Weakly Supervised Cross-Lingual Named Entity Recognition via Effective Annotation and Representation Projection
• XNLI: Evaluating Cross-lingual Sentence Representations
• sense2vec - A Fast and Accurate Method for Word Sense Disambiguation In Neural Word Embeddings
• Ultradense Word Embeddings by Orthogonal Transformation
• On the Dimensionality of Word Embedding
• Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learning
• How Neural Networks Extrapolate: From Feedforward to Graph Neural Networks
• Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Tasks
• Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond
• Unsupervised Neural Machine Translation
• Unsupervised Cross-Modal Alignment of Speech and Text Embedding Spaces
• Representation Learning for Natural Language Processing
• A Survey on Session-based Recommender Systems
• An Empirical Analysis of NMT-Derived Interlingual Embeddings and their Use in Parallel Sentence Identification
• Multilingual Models for Compositional Distributed Semantics
• Multilingual Distributed Representations without Word Alignment
• Distributed Representations of Signed Networks
• Unsupervised Hyperalignment for Multilingual Word Embeddings
• XLDA: Cross-Lingual Data Augmentation for Natural Language Inference and Question Answering
• Analyzing the Limitations of Cross-lingual Word Embedding Mappings
• A Call for More Rigor in Unsupervised Cross-lingual Learning
• Bilingual Lexicon Induction through Unsupervised Machine Translation
• Cross-lingual Retrieval for Iterative Self-Supervised Training
• Cross-lingual Models of Word Embeddings: An Empirical Comparison
• Learning Bilingual Word Representations by Marginalizing Alignments
• Ridge Regression, Hubness, and Zero-Shot Learning
• Learning Multilingual Word Representations using a Bag-of-Words Autoencoder
• Embedding Word Similarity with Neural Machine Translation
• Semantic Specialisation of Distributional Word Vector Spaces using Monolingual and Cross-Lingual Constraints
• Unsupervised Alignment of Embeddings with Wasserstein Procrustes
• From Zero to Hero: On the Limitations of Zero-Shot Cross-Lingual Transfer with Multilingual Transformers
• When Does Unsupervised Machine Translation Work?
• Transfer Learning for Speech and Language Processing
• Negative Sampling Improves Hypernymy Extraction Based on Projection Learning
• Confusion2Vec: Towards Enriching Vector Space Word Representations with Representational Ambiguities
• Improving Word Translation via Two-Stage Contrastive Learning
• Unsupervised Cross-lingual Adaptation for Sequence Tagging and Beyond
• Linear Ensembles of Word Embedding Models
• Emerging Cross-lingual Structure in Pretrained Language Models
• Cross-lingual alignments of ELMo contextual embeddings
• A Comprehensive Survey of Multilingual Neural Machine Translation
• High Quality ELMo Embeddings for Seven Less-Resourced Languages
• Word and Phrase Translation with word2vec
• A Resource-Free Evaluation Metric for Cross-Lingual Word Embeddings Based on Graph Modularity
• Identification of Biased Terms in News Articles by Comparison of Outlet-specific Word Embeddings
• Text Segmentation based on Semantic Word Embeddings
• Unsupervised Speech Recognition
• A Novel Bilingual Word Embedding Method for Lexical Translation Using Bilingual Sense Clique
• Towards Principled Unsupervised Learning
• Transductive Zero-Shot Hashing for Multilabel Image Retrieval
• Generalizing Procrustes Analysis for Better Bilingual Dictionary Induction
• Clustering Comparable Corpora of Russian and Ukrainian Academic Texts: Word Embeddings and Semantic Fingerprints
• Learning Translations via Matrix Completion
• Loss in Translation: Learning Bilingual Word Mapping with a Retrieval Criterion
• GT-SEER: Geo-Temporal SEquential Embedding Rank for Point-of-interest Recommendation
• Cognitive Database: A Step towards Endowing Relational Databases with Artificial Intelligence Capabilities
• Bilingual Embeddings with Random Walks over Multilingual Wordnets
• mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Dataset
• Word Mover's Embedding: From Word2Vec to Document Embedding
• Learning Backward Compatible Embeddings
• Are Emojis Predictable?
• MTOP: A Comprehensive Multilingual Task-Oriented Semantic Parsing Benchmark
• Embedding Semantic Relations into Word Representations
• MICE: Mining Idioms with Contextual Embeddings
• Cross-Lingual Sentiment Analysis Without (Good) Translation
• Words are Malleable: Computing Semantic Shifts in Political and Media Discourse
• USCORE: An Effective Approach to Fully Unsupervised Evaluation Metrics for Machine Translation
• Non-Adversarial Unsupervised Word Translation
• Leveraging Monolingual Data for Crosslingual Compositional Word Representations
• Crosslingual Document Embedding as Reduced-Rank Ridge Regression
• Modeling Musical Context with Word2vec
• Bridging the domain gap in cross-lingual document classification
• Distributed-Representation Based Hybrid Recommender System with Short Item Descriptions
• A Survey of Methods to Leverage Monolingual Data in Low-resource Neural Machine Translation
• word2word: A Collection of Bilingual Lexicons for 3,564 Language Pairs
• Revisiting the Context Window for Cross-lingual Word Embeddings
• Cross-Lingual BERT Transformation for Zero-Shot Dependency Parsing
• Fantastic Embeddings and How to Align Them: Zero-Shot Inference in a Multi-Shop Scenario
• Neural Cross-Lingual Named Entity Recognition with Minimal Resources
• Improving Cross-Lingual Word Embeddings by Meeting in the Middle
• Robust Cross-lingual Embeddings from Parallel Sentences
• Code-switched inspired losses for generic spoken dialog representations
• Cross-Lingual Alignment of Contextual Word Embeddings, with Applications to Zero-shot Dependency Parsing
• Acquiring Knowledge from Pre-trained Model to Neural Machine Translation
• How emoji and word embedding helps to unveil emotional transitions during online messaging
• Projecting Embeddings for Domain Adaptation: Joint Modeling of Sentiment Analysis in Diverse Domains
• Models for Capturing Temporal Smoothness in Evolving Networks for Learning Latent Representation of Nodes
• Enabling Cognitive Intelligence Queries in Relational Databases using Low-dimensional Word Embeddings
• Neural Machine Translation: Challenges, Progress and Future
• Distributed Representations for Compositional Semantics
• Unsupervised Multilingual Alignment using Wasserstein Barycenter
• Towards cross-lingual distributed representations without parallel text trained with adversarial autoencoders
• Hierarchical Learning of Cross-Language Mappings through Distributed Vector Representations for Code
• Cross-lingual Alignment vs Joint Training: A Comparative Study and A Simple Unified Framework
• A Study of Cross-Lingual Ability and Language-specific Information in Multilingual BERT
• Summarizing Unstructured Logs in Online Services
• Recurrent Neural Network based Part-of-Speech Tagger for Code-Mixed Social Media Text
• Learning Unsupervised Word Mapping by Maximizing Mean Discrepancy
• Word Embeddings for the Construction Domain
• Bilingual Dictionary Based Neural Machine Translation without Using Parallel Sentences
• Unsupervised Multilingual Word Embeddings
• Augmenting Statistical Machine Translation with Subword Translation of Out-of-Vocabulary Words
• Generalized Data Augmentation for Low-Resource Translation
• Filtered Inner Product Projection for Crosslingual Embedding Alignment
• Aligning Multilingual Word Embeddings for Cross-Modal Retrieval Task
• Improving Bilingual Lexicon Induction with Cross-Encoder Reranking
• Leveraging Pretrained Word Embeddings for Part-of-Speech Tagging of Code Switching Data
• Transferring Knowledge Distillation for Multilingual Social Event Detection
• Learning Continuous User Representations through Hybrid Filtering with doc2vec
• Updating Pre-trained Word Vectors and Text Classifiers using Monolingual Alignment
• PidginUNMT: Unsupervised Neural Machine Translation from West African Pidgin to English
• Cross-Lingual Word Embedding Refinement by ${\ell }_1$ Norm Optimisation
• Gender Bias in Multilingual Embeddings and Cross-Lingual Transfer
• Learning Multilingual Word Embeddings Using Image-Text Data
• Regula Sub-rosa: Latent Backdoor Attacks on Deep Neural Networks
• Multi-Source Cross-Lingual Model Transfer: Learning What to Share
• Classifying Syntactic Regularities for Hundreds of Languages
• What makes multilingual BERT multilingual?
• Cross-Lingual Relation Extraction with Transformers
• A Cross-Architecture Instruction Embedding Model for Natural Language Processing-Inspired Binary Code Analysis
• A Robust Self-Learning Method for Fully Unsupervised Cross-Lingual Mappings of Word Embeddings: Making the Method Robustly Reproducible as Well
• Semantic Classification of Tabular Datasets via Character-Level Convolutional Neural Networks
• Structured Embedding Models for Grouped Data
• Learning Representation Mapping for Relation Detection in Knowledge Base Question Answering
• Energy-based Self-attentive Learning of Abstractive Communities for Spoken Language Understanding
• Translating and Evolving: Towards a Model of Language Change in DisCoCat
• Bilingual Lexicon Induction with Semi-supervision in Non-Isometric Embedding Spaces
• Linear Transformations for Cross-lingual Sentiment Analysis
• Are Girls Neko or Shōjo? Cross-Lingual Alignment of Non-Isomorphic Embeddings with Iterative Normalization
• Combating the Curse of Multilinguality in Cross-Lingual WSD by Aligning Sparse Contextualized Word Representations
• Consistency Regularization for Cross-Lingual Fine-Tuning
• Efficient Non-parametric Estimation of Multiple Embeddings per Word in Vector Space
• Cross-Lingual BERT Contextual Embedding Space Mapping with Isotropic and Isometric Conditions
• Agreement-based Learning of Parallel Lexicons and Phrases from Non-Parallel Corpora
• Bilingual Alignment Pre-Training for Zero-Shot Cross-Lingual Transfer
• Multilingual Knowledge Graph Embeddings for Cross-lingual Knowledge Alignment
• Cross-Lingual Contextual Word Embeddings Mapping With Multi-Sense Words In Mind
• A Survey of Embedding Space Alignment Methods for Language and Knowledge Graphs
• Compass-aligned Distributional Embeddings for Studying Semantic Differences across Corpora
• Correlational Neural Networks
• Structural Correspondence Learning for Cross-lingual Sentiment Classification with One-to-many Mappings
• Shaping the Future of Endangered and Low-Resource Languages -- Our Role in the Age of LLMs: A Keynote at ECIR 2024
• Zero-Shot Cross-Lingual Dependency Parsing through Contextual Embedding Transformation
• Distributed representation of multi-sense words: A loss-driven approach
• Neural Adaptation Layers for Cross-domain Named Entity Recognition
• A Brief Survey of Multilingual Neural Machine Translation
• Learning Cross-lingual Embeddings from Twitter via Distant Supervision
• On the Limitations of Cross-lingual Encoders as Exposed by Reference-Free Machine Translation Evaluation
• Reproducing and learning new algebraic operations on word embeddings using genetic programming
• Cross-lingual Alignment Methods for Multilingual BERT: A Comparative Study
• Interactive Refinement of Cross-Lingual Word Embeddings
• Unsupervised Bilingual Lexicon Induction Across Writing Systems
• A Visual Embedding for the Unsupervised Extraction of Abstract Semantics
• Sinhala-English Parallel Word Dictionary Dataset
• Cell Type Identification from Single-Cell Transcriptomic Data via Semi-supervised Learning
• Aligning Vector-spaces with Noisy Supervised Lexicons
• Embedding Learning Through Multilingual Concept Induction
• Attention Focusing for Neural Machine Translation by Bridging Source and Target Embeddings
• Towards Multi-Sense Cross-Lingual Alignment of Contextual Embeddings
• Discovering Bilingual Lexicons in Polyglot Word Embeddings
• Team Alex at CLEF CheckThat! 2020: Identifying Check-Worthy Tweets With Transformer Models
• Modular Learning Component Attacks: Today's Reality, Tomorrow's Challenge
• Medical Knowledge Embedding Based on Recursive Neural Network for Multi-Disease Diagnosis
• Detecting Cross-Lingual Plagiarism Using Simulated Word Embeddings
• On the Convergent Properties of Word Embedding Methods
• The Geometry of Distributed Representations for Better Alignment, Attenuated Bias, and Improved Interpretability
• Combining Static Word Embeddings and Contextual Representations for Bilingual Lexicon Induction
• Isomorphic Cross-lingual Embeddings for Low-Resource Languages
• On Bilingual Lexicon Induction with Large Language Models
• PSSL: Self-supervised Learning for Personalized Search with Contrastive Sampling
• On a Novel Application of Wasserstein-Procrustes for Unsupervised Cross-Lingual Learning
• Neural Machine Translation: A Review and Survey
• Cross-Domain Ambiguity Detection using Linear Transformation of Word Embedding Spaces
• Learning Bilingual Word Embeddings Using Lexical Definitions
• Density Matching for Bilingual Word Embedding
• Multilingual Embeddings Jointly Induced from Contexts and Concepts: Simple, Strong and Scalable
• Semi-Supervised Translation with MMD Networks
• Spherical Regression under Mismatch Corruption with Application to Automated Knowledge Translation
• SAR: Learning Cross-Language API Mappings with Little Knowledge
• Learning User Representations in Online Social Networks using Temporal Dynamics of Information Diffusion
• Predicting Relevance Scores for Triples from Type-Like Relations using Neural Embedding - The Cabbage Triple Scorer at WSDM Cup 2017
• Dimension Projection among Languages based on Pseudo-relevant Documents for Query Translation
• A Strong Baseline for Learning Cross-Lingual Word Embeddings from Sentence Alignments
• Multilingual Word Embeddings using Multigraphs
• Weakly-Supervised Concept-based Adversarial Learning for Cross-lingual Word Embeddings
• Multilingual Music Genre Embeddings for Effective Cross-Lingual Music Item Annotation
• Multi-lingual Common Semantic Space Construction via Cluster-consistent Word Embedding
• Cross-lingual hate speech detection based on multilingual domain-specific word embeddings
• Characterizing Departures from Linearity in Word Translation
• Deep Learning Paradigm with Transformed Monolingual Word Embeddings for Multilingual Sentiment Analysis
• A Comparative Study on Structural and Semantic Properties of Sentence Embeddings
• Resolving Out-of-Vocabulary Words with Bilingual Embeddings in Machine Translation
• Analyzing Zero-shot Cross-lingual Transfer in Supervised NLP Tasks
• PESTS: Persian_English Cross Lingual Corpus for Semantic Textual Similarity
• Hyperpolyglot LLMs: Cross-Lingual Interpretability in Token Embeddings
• Refinement of Unsupervised Cross-Lingual Word Embeddings
• An Unsupervised Approach for Mapping between Vector Spaces
• Unsupervised Bilingual Lexicon Induction from Mono-lingual Multimodal Data
• Sentence transition matrix: An efficient approach that preserves sentence semantics
• Deep Structured Cross-Modal Anomaly Detection
• Bilingual Lexicon Induction via Unsupervised Bitext Construction and Word Alignment
• Why Overfitting Isn't Always Bad: Retrofitting Cross-Lingual Word Embeddings to Dictionaries
• Duality Regularization for Unsupervised Bilingual Lexicon Induction
• Cross-lingual Word Analogies using Linear Transformations between Semantic Spaces
• Generating Sense Embeddings for Syntactic and Semantic Analogy for Portuguese
• Exploiting Cross-Lingual Subword Similarities in Low-Resource Document Classification
• Are We Consistently Biased? Multidimensional Analysis of Biases in Distributional Word Vectors
• Gap Analysis of Natural Language Processing Systems with respect to Linguistic Modality
• Semi-Supervised Bilingual Lexicon Induction with Two-way Interaction
• A Relaxed Matching Procedure for Unsupervised BLI
• Cross-lingual Transfer of Abstractive Summarizer to Less-resource Language
• Sparse Coding of Neural Word Embeddings for Multilingual Sequence Labeling
• AM2iCo: Evaluating Word Meaning in Context across Low-Resource Languages with Adversarial Examples
• Cross-Attention is All You Need: Adapting Pretrained Transformers for Machine Translation
• Improving Zero-Shot Cross-Lingual Transfer Learning via Robust Training
• Unified Embedding and Metric Learning for Zero-Exemplar Event Detection
• Unsupervised Learning of Parsimonious General-Purpose Embeddings for User and Location Modelling
• A comprehensive empirical analysis on cross-domain semantic enrichment for detection of depressive language
• Cross-lingual Word Embeddings beyond Zero-shot Machine Translation
• Inverted Bilingual Topic Models for Lexicon Extraction from Non-parallel Data
• Building a robust sentiment lexicon with (almost) no resource
• Using Optimal Transport as Alignment Objective for fine-tuning Multilingual Contextualized Embeddings
• Cross-language Information Retrieval
• Group based Personalized Search by Integrating Search Behaviour and Friend Network
• Cross-lingual Transfer for Text Classification with Dictionary-based Heterogeneous Graph
• Examining Cross-lingual Contextual Embeddings with Orthogonal Structural Probes
• Bilingual Topic Models for Comparable Corpora
• UsingWord Embeddings for Query Translation for Hindi to English Cross Language Information Retrieval
• Contributions to Representation Learning with Graph Autoencoders and Applications to Music Recommendation
• Searching Personal Collections
• On Geometric Alignment in Low Doubling Dimension
• A Context-Aware Approach for Detecting Check-Worthy Claims in Political Debates
• Two Way Adversarial Unsupervised Word Translation
• Open Named Entity Modeling from Embedding Distribution
• Aligning Hotel Embeddings using Domain Adaptation for Next-Item Recommendation
• Balancing the composition of word embeddings across heterogenous data sets
• Towards conceptual generalization in the embedding space
• Document Network Embedding: Coping for Missing Content and Missing Links
• ABSent: Cross-Lingual Sentence Representation Mapping with Bidirectional GANs
• Bilingual Dictionary Induction for Bantu Languages
• Constrained Non-Affine Alignment of Embeddings
• Unsupervised Word Translation Pairing using Refinement based Point Set Registration
• Mapping Supervised Bilingual Word Embeddings from English to low-resource languages
• Multi-Fusion Chinese WordNet (MCW) : Compound of Machine Learning and Manual Correction
• Monolingually Derived Phrase Scores for Phrase Based SMT Using Neural Networks Vector Representations
• A Systematic Investigation of KB-Text Embedding Alignment at Scale
• Is Aligning Embedding Spaces a Challenging Task? A Study on Heterogeneous Embedding Alignment Methods
• Cross-language Citation Recommendation via Hierarchical Representation Learning on Heterogeneous Graph
• Revisit Systematic Generalization via Meaningful Learning
• Word Embeddings: Stability and Semantic Change
• Data Augmentation with Unsupervised Machine Translation Improves the Structural Similarity of Cross-lingual Word Embeddings
• DrugDBEmbed : Semantic Queries on Relational Database using Supervised Column Encodings
• Learning to Represent Bilingual Dictionaries
• Neural Cross-Lingual Coreference Resolution and its Application to Entity Linking
• Learning aligned embeddings for semi-supervised word translation using Maximum Mean Discrepancy
• Variational learning across domains with triplet information
• Revisiting Adversarial Autoencoder for Unsupervised Word Translation with Cycle Consistency and Improved Training
• Learning Contextualised Cross-lingual Word Embeddings and Alignments for Extremely Low-Resource Languages Using Parallel Corpora