Unsupervised Pronoun Resolution via Masked Noun-Phrase Prediction
arXiv:2105.12392
Abstract
In this work, we propose Masked Noun-Phrase Prediction (MNPP), a pre-training strategy to tackle pronoun resolution in a fully unsupervised setting. Firstly, We evaluate our pre-trained model on various pronoun resolution datasets without any finetuning. Our method outperforms all previous unsupervised methods on all datasets by large margins. Secondly, we proceed to a few-shot setting where we finetune our pre-trained model on WinoGrande-S and XS separately. Our method outperforms RoBERTa-large baseline with large margins, meanwhile, achieving a higher AUC score after further finetuning on the remaining three official splits of WinoGrande.
arXiv commentAccepted to ACL2021
References in corpus (9)
- SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems
- A Simple Method for Commonsense Reasoning
- Unsupervised Question Answering by Cloze Translation
- A Surprisingly Robust Trick for Winograd Schema Challenge
- Align, Mask and Select: A Simple Method for Incorporating Commonsense Knowledge into Language Representation Models
- A Review of Winograd Schema Challenge Datasets and Approaches
- Exploring Unsupervised Pretraining and Sentence Structure Modelling for Winograd Schema Challenge
- A Hybrid Neural Network Model for Commonsense Reasoning
- TTTTTackling WinoGrande Schemas