Learning from Noisy Labels for Entity-Centric Information Extraction
arXiv:2104.08656 · doi:10.18653/v1/2021.emnlp-main.437
Abstract
Recent information extraction approaches have relied on training deep neural models. However, such models can easily overfit noisy labels and suffer from performance degradation. While it is very costly to filter noisy labels in large learning resources, recent studies show that such labels take more training steps to be memorized and are more frequently forgotten than clean labels, therefore are identifiable in training. Motivated by such properties, we propose a simple co-regularization framework for entity-centric information extraction, which consists of several neural models with identical structures but different parameter initialization. These models are jointly optimized with the task-specific losses and are regularized to generate similar predictions based on an agreement loss, which prevents overfitting on noisy labels. Extensive experiments on two widely used but noisy benchmarks for information extraction, TACRED and CoNLL03, demonstrate the effectiveness of our framework. We release our code to the community for future research.
arXiv commentAccepted at EMNLP 2021. Code available at https://github.com/wzhouad/NLL-IE
References in corpus (15)
- Adam: A Method for Stochastic Optimization
- Introduction to the CoNLL-2002 Shared Task: Language-Independent Named Entity Recognition
- Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels
- When Does Label Smoothing Help?
- MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels
- Training Convolutional Networks with Noisy Labels
- Text Chunking using Transformation-Based Learning
- A Closer Look at Memorization in Deep Networks
- Data Programming: Creating Large Training Sets, Quickly
- Simple BERT Models for Relation Extraction and Semantic Role Labeling
- Active Bias: Training More Accurate Neural Networks by Emphasizing High Variance Samples
- Decoupling "when to update" from "how to update"
- An Empirical Study of Example Forgetting during Deep Neural Network Learning
- Finding Influential Instances for Distantly Supervised Relation Extraction
- Robust Training with Ensemble Consensus