10 papers
Identifying the Limits of Cross-Domain Knowledge Transfer for Pretrained Models
Zhengxuan Wu, Nelson F. Liu, Christopher Potts
There is growing evidence that pretrained language models improve task-specific fine-tuning not just for the languages seen in pretraining, but also for new languages and even non-…
Evaluating Models' Local Decision Boundaries via Contrast Sets
Matt Gardner, Yoav Artzi, Victoria Basmova +23
Standard test sets for supervised learning evaluate in-distribution generalization. Unfortunately, when a dataset has systematic gaps (e.g., annotation artifacts), these evaluation…
Quoref: A Reading Comprehension Dataset with Questions Requiring Coreferential Reasoning
Pradeep Dasigi, Nelson F. Liu, Ana Marasović +2
Machine comprehension of texts longer than a single sentence often requires coreference resolution. However, most current reading comprehension benchmarks do not contain complex co…
Barack's Wife Hillary: Using Knowledge-Graphs for Fact-Aware Language Modeling
Robert L. Logan, Nelson F. Liu, Matthew E. Peters +2
Modeling human language requires the ability to not only generate fluent text but also encode factual knowledge. However, traditional language models are only capable of rememberin…
Inoculation by Fine-Tuning: A Method for Analyzing Challenge Datasets
Nelson F. Liu, Roy Schwartz, Noah A. Smith
Several datasets have recently been constructed to expose brittleness in models trained on existing benchmarks. While model performance on these challenge datasets is significantly…
Linguistic Knowledge and Transferability of Contextual Representations
Nelson F. Liu, Matt Gardner, Yonatan Belinkov +2
Contextual word representations derived from large-scale neural language models are successful across a diverse set of NLP tasks, suggesting that they encode useful and transferabl…