Massively Multilingual Word Embeddings

Abstract

We introduce new methods for estimating and evaluating embeddings of words in more than fifty languages in a single shared embedding space. Our estimation methods, multiCluster and multiCCA, use dictionaries and monolingual data; they do not require parallel data. Our new evaluation method, multiQVEC-CCA, is shown to correlate better than previous ones with two downstream tasks (text categorization and parsing). We also describe a web portal for evaluation that will facilitate further research in this area, along with open-source releases of all our methods.

References in corpus (2)

• Transition-Based Dependency Parsing with Stack Long Short-Term Memory
• Learning Bilingual Word Representations by Marginalizing Alignments

Cited by in corpus (66)

• Cross-lingual Language Model Pretraining
• Word Translation Without Parallel Data
• XNLI: Evaluating Cross-lingual Sentence Representations
• Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Tasks
• Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond
• Zero-Shot Cross-lingual Classification Using Multilingual Neural Machine Translation
• Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT
• Multilingual Hierarchical Attention Networks for Document Classification
• Learning how to Active Learn: A Deep Reinforcement Learning Approach
• Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Information
• Cross-lingual Models of Word Embeddings: An Empirical Comparison
• What does it mean to be language-agnostic? Probing multilingual sentence encoders for typological properties
• Latin BERT: A Contextual Language Model for Classical Philology
• Low-Resource Text Classification using Domain-Adversarial Learning
• A Resource-Free Evaluation Metric for Cross-Lingual Word Embeddings Based on Graph Modularity
• Generalizing Procrustes Analysis for Better Bilingual Dictionary Induction
• mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Dataset
• MTOP: A Comprehensive Multilingual Task-Oriented Semantic Parsing Benchmark
• Crosslingual Document Embedding as Reduced-Rank Ridge Regression
• Neural Cross-Lingual Named Entity Recognition with Minimal Resources
• Improving Cross-Lingual Word Embeddings by Meeting in the Middle
• Word Alignment by Fine-tuning Embeddings on Parallel Corpora
• Towards cross-lingual distributed representations without parallel text trained with adversarial autoencoders
• Bilingual Dictionary Based Neural Machine Translation without Using Parallel Sentences
• Cross-lingual Alignment vs Joint Training: A Comparative Study and A Simple Unified Framework
• A Study of Cross-Lingual Ability and Language-specific Information in Multilingual BERT
• PidginUNMT: Unsupervised Neural Machine Translation from West African Pidgin to English
• Gender Bias in Multilingual Embeddings and Cross-Lingual Transfer
• What makes multilingual BERT multilingual?
• Learning Multilingual Word Embeddings Using Image-Text Data
• Structured Embedding Models for Grouped Data
• Linear Transformations for Cross-lingual Sentiment Analysis
• Transformer based Multilingual document Embedding model
• Multilingual Clustering of Streaming News
• Automatically Identifying Language Family from Acoustic Examples in Low Resource Scenarios
• Towards Multi-Sense Cross-Lingual Alignment of Contextual Embeddings
• Interactive Refinement of Cross-Lingual Word Embeddings
• Discovering Bilingual Lexicons in Polyglot Word Embeddings
• Embedding Learning Through Multilingual Concept Induction
• Detecting Cross-Lingual Plagiarism Using Simulated Word Embeddings
• Cross-neutralising: Probing for joint encoding of linguistic information in multilingual models
• GILE: A Generalized Input-Label Embedding for Text Classification
• Joint Multilingual Supervision for Cross-lingual Entity Linking
• Multilingual Embeddings Jointly Induced from Contexts and Concepts: Simple, Strong and Scalable
• Exploiting Cross-Lingual Subword Similarities in Low-Resource Document Classification
• Joint Representation Learning of Cross-lingual Words and Entities via Attentive Distant Supervision
• Why Overfitting Isn't Always Bad: Retrofitting Cross-Lingual Word Embeddings to Dictionaries
• Multi-lingual Common Semantic Space Construction via Cluster-consistent Word Embedding
• Cross-lingual hate speech detection based on multilingual domain-specific word embeddings
• Hyperpolyglot LLMs: Cross-Lingual Interpretability in Token Embeddings
• Expanding the Text Classification Toolbox with Cross-Lingual Embeddings
• Weakly-Supervised Concept-based Adversarial Learning for Cross-lingual Word Embeddings
• On the Feasibility of Automated Detection of Allusive Text Reuse
• Regularization Advantages of Multilingual Neural Language Models for Low Resource Domains
• ALL-IN-1: Short Text Classification with One Model for All Languages
• Characterizing Departures from Linearity in Word Translation
• Refinement of Unsupervised Cross-Lingual Word Embeddings
• Cross-lingual Word Embeddings beyond Zero-shot Machine Translation
• Addressee and Response Selection for Multilingual Conversation
• Towards Language Agnostic Universal Representations
• Bilingual Dictionary Induction for Bantu Languages
• Constrained Non-Affine Alignment of Embeddings
• Neural Cross-Lingual Coreference Resolution and its Application to Entity Linking
• Char-RNN for Word Stress Detection in East Slavic Languages
• Searching Personal Collections
• Cross-language Information Retrieval