CalibreNet: Calibration Networks for Multilingual Sequence Labeling
arXiv:2011.05723
Abstract
Lack of training data in low-resource languages presents huge challenges to sequence labeling tasks such as named entity recognition (NER) and machine reading comprehension (MRC). One major obstacle is the errors on the boundary of predicted answers. To tackle this problem, we propose CalibreNet, which predicts answers in two steps. In the first step, any existing sequence labeling method can be adopted as a base model to generate an initial answer. In the second step, CalibreNet refines the boundary of the initial answer. To tackle the challenge of lack of training data in low-resource languages, we dedicatedly develop a novel unsupervised phrase boundary recovery pre-training task to enhance the multilingual boundary detection capability of CalibreNet. Experiments on two cross-lingual benchmark datasets show that the proposed approach achieves SOTA results on zero-shot cross-lingual NER and MRC tasks.
Long paper in WSDM 2021
References in corpus (7)
- Optimal Hyperparameters for Deep LSTM-Networks for Sequence Labeling Tasks
- How multilingual is Multilingual BERT?
- Dataset and Neural Recurrent Sequence Labeling Model for Open-Domain Factoid Question Answering
- Towards Lingua Franca Named Entity Recognition with BERT
- Template-Based Question Generation from Retrieved Sentences for Improved Unsupervised Question Answering
- Enhancing Answer Boundary Detection for Multilingual Machine Reading Comprehension
- Structure-Level Knowledge Distillation For Multilingual Sequence Labeling