CalibreNet: Calibration Networks for Multilingual Sequence Labeling

Abstract

Lack of training data in low-resource languages presents huge challenges to sequence labeling tasks such as named entity recognition (NER) and machine reading comprehension (MRC). One major obstacle is the errors on the boundary of predicted answers. To tackle this problem, we propose CalibreNet, which predicts answers in two steps. In the first step, any existing sequence labeling method can be adopted as a base model to generate an initial answer. In the second step, CalibreNet refines the boundary of the initial answer. To tackle the challenge of lack of training data in low-resource languages, we dedicatedly develop a novel unsupervised phrase boundary recovery pre-training task to enhance the multilingual boundary detection capability of CalibreNet. Experiments on two cross-lingual benchmark datasets show that the proposed approach achieves SOTA results on zero-shot cross-lingual NER and MRC tasks.

arXiv commentLong paper in WSDM 2021

References in corpus (14)

• On the Cross-lingual Transferability of Monolingual Representations
• Optimal Hyperparameters for Deep LSTM-Networks for Sequence Labeling Tasks
• Weakly Supervised Cross-Lingual Named Entity Recognition via Effective Annotation and Representation Projection
• How multilingual is Multilingual BERT?
• Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Tasks
• Dataset and Neural Recurrent Sequence Labeling Model for Open-Domain Factoid Question Answering
• Entity-Relation Extraction as Multi-Turn Question Answering
• Cross-Lingual Machine Reading Comprehension
• Towards Lingua Franca Named Entity Recognition with BERT
• Single-/Multi-Source Cross-Lingual NER via Teacher-Student Learning on Unlabeled Data in Target Language
• Neural Cross-Lingual Named Entity Recognition with Minimal Resources
• Template-Based Question Generation from Retrieved Sentences for Improved Unsupervised Question Answering
• Enhancing Answer Boundary Detection for Multilingual Machine Reading Comprehension
• Structure-Level Knowledge Distillation For Multilingual Sequence Labeling