63 citations · 136 across the 11 of their papers we have counts for
17 papers
Lost in the Middle: How Language Models Use Long Contexts
Nelson F. Liu, Kevin Lin, John Hewitt +4
While recent language models have the ability to take long contexts as input, relatively little is known about how well they use longer context. We analyze the performance of langu…
Anchor Prediction: Automatic Refinement of Internet Links
Nelson F. Liu, Kenton Lee, Kristina Toutanova
Internet links enable users to deepen their understanding of a topic by providing convenient access to related information. However, the majority of links are unanchored -- they li…
Evaluating Verifiability in Generative Search Engines
Nelson F. Liu, Tianyi Zhang, Percy Liang
Generative search engines directly generate responses to user queries, along with in-line citations. A prerequisite trait of a trustworthy generative search engine is verifiability…
Are Sample-Efficient NLP Models More Robust?
Nelson F. Liu, Ananya Kumar, Percy Liang +1
Recent results in image classification and extractive question answering have observed that pre-trained models trained on less in-distribution data have better out-of-distribution…
Making Heads and Tails of Models with Marginal Calibration for Sparse Tagsets
Michael Kranzlein, Nelson F. Liu, Nathan Schneider
For interpreting the behavior of a probabilistic model, it is useful to measure a model's calibration--the extent to which it produces reliable confidence scores. We address the op…
Identifying the Limits of Cross-Domain Knowledge Transfer for Pretrained Models
Zhengxuan Wu, Nelson F. Liu, Christopher Potts
There is growing evidence that pretrained language models improve task-specific fine-tuning not just for the languages seen in pretraining, but also for new languages and even non-…
Do Question Answering Modeling Improvements Hold Across Benchmarks?
Nelson F. Liu, Tony Lee, Robin Jia +1
Do question answering (QA) modeling improvements (e.g., choice of architecture and training procedure) hold consistently across the diverse landscape of QA benchmarks? To study thi…
Lexical Semantic Recognition
Nelson F. Liu, Daniel Hershcovich, Michael Kranzlein +1
In lexical semantics, full-sentence segmentation and segment labeling of various phenomena are generally treated separately, despite their interdependence. We hypothesize that a un…
Evaluating Models' Local Decision Boundaries via Contrast Sets
Matt Gardner, Yoav Artzi, Victoria Basmova +23
Standard test sets for supervised learning evaluate in-distribution generalization. Unfortunately, when a dataset has systematic gaps (e.g., annotation artifacts), these evaluation…
Quoref: A Reading Comprehension Dataset with Questions Requiring Coreferential Reasoning
Pradeep Dasigi, Nelson F. Liu, Ana Marasović +2
Machine comprehension of texts longer than a single sentence often requires coreference resolution. However, most current reading comprehension benchmarks do not contain complex co…
Barack's Wife Hillary: Using Knowledge-Graphs for Fact-Aware Language Modeling
Robert L. Logan, Nelson F. Liu, Matthew E. Peters +2
Modeling human language requires the ability to not only generate fluent text but also encode factual knowledge. However, traditional language models are only capable of rememberin…
Inoculation by Fine-Tuning: A Method for Analyzing Challenge Datasets
Nelson F. Liu, Roy Schwartz, Noah A. Smith
Several datasets have recently been constructed to expose brittleness in models trained on existing benchmarks. While model performance on these challenge datasets is significantly…
Linguistic Knowledge and Transferability of Contextual Representations
Nelson F. Liu, Matt Gardner, Yonatan Belinkov +2
Contextual word representations derived from large-scale neural language models are successful across a diverse set of NLP tasks, suggesting that they encode useful and transferabl…
Augmenting Statistical Machine Translation with Subword Translation of Out-of-Vocabulary Words
Nelson F. Liu, Jonathan May, Michael Pust +1
Most statistical machine translation systems cannot translate words that are unseen in the training data. However, humans can translate many classes of out-of-vocabulary (OOV) word…
LSTMs Exploit Linguistic Attributes of Data
Nelson F. Liu, Omer Levy, Roy Schwartz +2
While recurrent neural networks have found success in a variety of natural language processing applications, they are general models of sequential data. We investigate how the prop…
AllenNLP: A Deep Semantic Natural Language Processing Platform
Matt Gardner, Joel Grus, Mark Neumann +6
This paper describes AllenNLP, a platform for research on deep learning methods in natural language understanding. AllenNLP is designed to support researchers who want to build nov…
Crowdsourcing Multiple Choice Science Questions
Johannes Welbl, Nelson F. Liu, Matt Gardner
We present a novel method for obtaining high-quality, domain-targeted multiple choice questions from crowd workers. Generating these questions can be difficult without trading away…