KorNLI and KorSTS: New Benchmark Datasets for Korean Natural Language Understanding

Abstract

Natural language inference (NLI) and semantic textual similarity (STS) are key tasks in natural language understanding (NLU). Although several benchmark datasets for those tasks have been released in English and a few other languages, there are no publicly available NLI or STS datasets in the Korean language. Motivated by this, we construct and release new datasets for Korean NLI and STS, dubbed KorNLI and KorSTS, respectively. Following previous approaches, we machine-translate existing English training sets and manually translate development and test sets into Korean. To accelerate research on Korean NLU, we also establish baselines on KorNLI and KorSTS. Our datasets are publicly available at https://github.com/kakaobrain/KorNLUDatasets.

arXiv commentFindings of EMNLP 2020. Datasets available at https://github.com/kakaobrain/KorNLUDatasets

References in corpus (6)

• Cross-lingual Language Model Pretraining
• SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems
• Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Tasks
• XNLI: Evaluating Cross-lingual Sentence Representations
• Multilingual Universal Sentence Encoder for Semantic Retrieval
• KorQuAD1.0: Korean QA Dataset for Machine Reading Comprehension

Cited by in corpus (7)

• An Empirical Study of Tokenization Strategies for Various Korean NLP Tasks
• Open Korean Corpora: A Practical Report
• Accurate, yet inconsistent? Consistency Analysis on Language Understanding Models
• A novel hybrid methodology of measuring sentence similarity
• Analyzing Zero-shot Cross-lingual Transfer in Supervised NLP Tasks
• KoreALBERT: Pretraining a Lite BERT Model for Korean Language Understanding
• Reference and Document Aware Semantic Evaluation Methods for Korean Language Summarization