A Survey on Awesome Korean NLP Datasets
arXiv:2112.01624 · doi:10.1109/ICTC55196.2022.9952930
Abstract
English based datasets are commonly available from Kaggle, GitHub, or recently published papers. Although benchmark tests with English datasets are sufficient to show off the performances of new models and methods, still a researcher need to train and validate the models on Korean based datasets to produce a technology or product, suitable for Korean processing. This paper introduces 15 popular Korean based NLP datasets with summarized details such as volume, license, repositories, and other research results inspired by the datasets. Also, I provide high-resolution instructions with sample or statistics of datasets. The main characteristics of datasets are presented on a single table to provide a rapid summarization of datasets for researchers.
arXiv comment11 pages, 1 horizontal page for large table
References in corpus (16)
- HuggingFace's Transformers: State-of-the-art Natural Language Processing
- Language Modeling with Gated Convolutional Networks
- Deep Sentence Embedding Using Long Short-Term Memory Networks: Analysis and Application to Information Retrieval
- SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluation
- Revisiting Pre-Trained Models for Chinese Natural Language Processing
- KLUE: Korean Language Understanding Evaluation
- Semantic Modelling with Long-Short-Term Memory for Information Retrieval
- KorQuAD1.0: Korean QA Dataset for Machine Reading Comprehension
- An Empirical Study of Tokenization Strategies for Various Korean NLP Tasks
- Open Korean Corpora: A Practical Report
- CareCall: a Call-Based Active Monitoring Dialog Agent for Managing COVID-19 Pandemic
- Accurate, yet inconsistent? Consistency Analysis on Language Understanding Models
- LSTM-Based Predictions for Proactive Information Retrieval
- Investigating an Effective Character-level Embedding in Korean Sentence Classification
- Boosting Active Learning for Speech Recognition with Noisy Pseudo-labeled Samples
- StyleKQC: A Style-Variant Paraphrase Corpus for Korean Questions and Commands