2.3k citations · 2.5k across the 15 of their papers we have counts for
18 papers
Instruction Following without Instruction Tuning
John Hewitt, Nelson F. Liu, Percy Liang +1
Instruction tuning commonly means finetuning a language model on instruction-response pairs. We discover two forms of adaptation (tuning) that are deficient compared to instruction…
Learning Translations via Matrix Completion
Derry Wijaya, Brendan Callahan, John Hewitt +4
Bilingual Lexicon Induction is the task of learning word translations without bilingual parallel corpora. We model this task as a matrix completion problem, and present an effectiv…
Model Editing with Canonical Examples
John Hewitt, Sarah Chen, Lanruo Lora Xie +3
We introduce model editing with canonical examples, a setting in which (1) a single learning example is provided per desired behavior, (2) evaluation is performed exclusively out-o…
Lost in the Middle: How Language Models Use Long Contexts
Nelson F. Liu, Kevin Lin, John Hewitt +4
While recent language models have the ability to take long contexts as input, relatively little is known about how well they use longer context. We analyze the performance of langu…
Backpack Language Models
John Hewitt, John Thickstun, Christopher D. Manning +1
We present Backpacks: a new neural architecture that marries strong modeling performance with an interface for interpretability and control. Backpacks learn multiple non-contextual…
JamPatoisNLI: A Jamaican Patois Natural Language Inference Dataset
Ruth-Ann Armstrong, John Hewitt, Christopher Manning
JamPatoisNLI provides the first dataset for natural language inference in a creole language, Jamaican Patois. Many of the most-spoken low-resource languages are creoles. These lang…
Truncation Sampling as Language Model Desmoothing
John Hewitt, Christopher D. Manning, Percy Liang
Long samples of text from neural language models can be of poor quality. Truncation sampling algorithms--like top- or top- -- address this by setting some words' probabilitie…
Leveraging Large (Visual) Language Models for Robot 3D Scene Understanding
William Chen, Siyi Hu, Rajat Talak +1
Abstract semantic 3D scene understanding is a problem of critical importance in robotics. As robots still lack the common-sense knowledge about household objects and locations of a…
Conditional probing: measuring usable information beyond a baseline
John Hewitt, Kawin Ethayarajh, Percy Liang +1
Probing experiments investigate the extent to which neural representations make properties -- like part-of-speech -- predictable. One suggests that a representation encodes a prope…
On the Opportunities and Risks of Foundation Models
Rishi Bommasani, Drew A. Hudson, Ehsan Adeli +111
AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks.…
Refining Targeted Syntactic Evaluation of Language Models
Benjamin Newman, Kai-Siang Ang, Julia Gong +1
Targeted syntactic evaluation of subject-verb number agreement in English (TSE) evaluates language models' syntactic knowledge using hand-crafted minimal pairs of sentences that di…
Probing artificial neural networks: insights from neuroscience
Anna A. Ivanova, John Hewitt, Noga Zaslavsky
A major challenge in both neuroscience and machine learning is the development of useful tools for understanding complex information processing systems. One such tool is probes, i.…
RNNs can generate bounded hierarchical languages with optimal memory
John Hewitt, Michael Hahn, Surya Ganguli +2
Recurrent neural networks empirically generate natural language with high syntactic fidelity. However, their success is not well-understood theoretically. We provide theoretical in…
The EOS Decision and Length Extrapolation
Benjamin Newman, John Hewitt, Percy Liang +1
Extrapolation to unseen sequence lengths is a challenge for neural generative models of language. In this work, we characterize the effect on length extrapolation of a modeling dec…
Finding Universal Grammatical Relations in Multilingual BERT
Ethan A. Chi, John Hewitt, Christopher D. Manning
Recent work has found evidence that Multilingual BERT (mBERT), a transformer-based multilingual masked language model, is capable of zero-shot cross-lingual transfer, suggesting th…
Designing and Interpreting Probes with Control Tasks
John Hewitt, Percy Liang
Probes, supervised models trained to predict properties (like parts-of-speech) from representations (like ELMo), have achieved high accuracy on a range of linguistic tasks. But doe…
Simple, Fast, Accurate Intent Classification and Slot Labeling for Goal-Oriented Dialogue Systems
Arshit Gupta, John Hewitt, Katrin Kirchhoff
With the advent of conversational assistants, like Amazon Alexa, Google Now, etc., dialogue systems are gaining a lot of traction, especially in industrial setting. These systems t…
XNMT: The eXtensible Neural Machine Translation Toolkit
Graham Neubig, Matthias Sperber, Xinyi Wang +10
This paper describes XNMT, the eXtensible Neural Machine Translation toolkit. XNMT distin- guishes itself from other open-source NMT toolkits by its focus on modular code design, w…