FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation
arXiv:1810.10147
Abstract
We present a Few-Shot Relation Classification Dataset (FewRel), consisting of 70, 000 sentences on 100 relations derived from Wikipedia and annotated by crowdworkers. The relation of each sentence is first recognized by distant supervision methods, and then filtered by crowdworkers. We adapt the most recent state-of-the-art few-shot learning methods for relation classification and conduct a thorough evaluation of these methods. Empirical results show that even the most competitive few-shot learning models struggle on this task, especially as compared with humans. We also show that a range of different reasoning skills are needed to solve our task. These results indicate that few-shot relation classification remains an open problem and still requires further research. Our detailed analysis points multiple directions for future research. All details and resources about the dataset and baselines are released on http://zhuhao.me/fewrel.
arXiv commentEMNLP 2018. The first four authors contribute equally. The order is determined by dice rolling. Visit our website http://zhuhao.me/fewrel
References in corpus (5)
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Prototypical Networks for Few-shot Learning
- DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
- Matching Networks for One Shot Learning
- Learning with Noise: Enhance Distantly Supervised Relation Extraction with Dynamic Transition Matrix
Cited by in corpus (9)
- Few-shot Relation Extraction via Bayesian Meta-learning on Relation Graphs
- Deep Neural Network Based Relation Extraction: An Overview
- FLEX: Unifying Evaluation for Few-Shot NLP
- Entity Concept-enhanced Few-shot Relation Extraction
- Bridging Text and Knowledge with Multi-Prototype Embedding for Few-Shot Relational Triple Extraction
- Adaptive Prototypical Networks with Label Words and Joint Representation Learning for Few-Shot Relation Classification
- Web Question Answering with Neurosymbolic Program Synthesis
- Metadata Shaping: Natural Language Annotations for the Tail
- Open Rule Induction