9 citations · 18 across the 7 of their papers we have counts for
11 papers · 1 filter
Instruction Following without Instruction Tuning
John Hewitt, Nelson F. Liu, Percy Liang +1
Instruction tuning commonly means finetuning a language model on instruction-response pairs. We discover two forms of adaptation (tuning) that are deficient compared to instruction…
JamPatoisNLI: A Jamaican Patois Natural Language Inference Dataset
Ruth-Ann Armstrong, John Hewitt, Christopher Manning
JamPatoisNLI provides the first dataset for natural language inference in a creole language, Jamaican Patois. Many of the most-spoken low-resource languages are creoles. These lang…
Truncation Sampling as Language Model Desmoothing
John Hewitt, Christopher D. Manning, Percy Liang
Long samples of text from neural language models can be of poor quality. Truncation sampling algorithms--like top- or top- -- address this by setting some words' probabilitie…
Conditional probing: measuring usable information beyond a baseline
John Hewitt, Kawin Ethayarajh, Percy Liang +1
Probing experiments investigate the extent to which neural representations make properties -- like part-of-speech -- predictable. One suggests that a representation encodes a prope…
Refining Targeted Syntactic Evaluation of Language Models
Benjamin Newman, Kai-Siang Ang, Julia Gong +1
Targeted syntactic evaluation of subject-verb number agreement in English (TSE) evaluates language models' syntactic knowledge using hand-crafted minimal pairs of sentences that di…
RNNs can generate bounded hierarchical languages with optimal memory
John Hewitt, Michael Hahn, Surya Ganguli +2
Recurrent neural networks empirically generate natural language with high syntactic fidelity. However, their success is not well-understood theoretically. We provide theoretical in…