XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation
arXiv:2004.01401
Abstract
In this paper, we introduce XGLUE, a new benchmark dataset that can be used to train large-scale cross-lingual pre-trained models using multilingual and bilingual corpora and evaluate their performance across a diverse set of cross-lingual tasks. Comparing to GLUE(Wang et al., 2019), which is labeled in English for natural language understanding tasks only, XGLUE has two main advantages: (1) it provides 11 diversified tasks that cover both natural language understanding and generation scenarios; (2) for each task, it provides labeled data in multiple languages. We extend a recent cross-lingual pre-trained model Unicoder(Huang et al., 2019) to cover both understanding and generation tasks, which is evaluated on XGLUE as a strong baseline. We also evaluate the base versions (12-layer) of Multilingual BERT, XLM and XLM-R for comparison.
References in corpus (6)
- Adam: A Method for Stochastic Optimization
- XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization
- CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data
- XNLI: Evaluating Cross-lingual Sentence Representations
- ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training
- PAWS: Paraphrase Adversaries from Word Scrambling
Cited by in corpus (11)
- CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation
- InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training
- M3P: Learning Universal Representations via Multitask Multilingual Multimodal Pre-training
- IndoLEM and IndoBERT: A Benchmark Dataset and Pre-trained Language Model for Indonesian NLP
- Exploring Fine-tuning Techniques for Pre-trained Cross-lingual Models via Continual Learning
- VECO: Variable and Flexible Cross-lingual Pre-training for Language Understanding and Generation
- ExplainaBoard: An Explainable Leaderboard for NLP
- ParsiNLU: A Suite of Language Understanding Challenges for Persian
- ProphetNet-X: Large-Scale Pre-training Models for English, Chinese, Multi-lingual, Dialog, and Code Generation
- Do Explicit Alignments Robustly Improve Multilingual Encoders?
- Bilingual Language Modeling, A transfer learning technique for Roman Urdu