337 citations · 2.2k across the 46 of their papers we have counts for
40 papers · 1 filter
Truncation Sampling as Language Model Desmoothing
John Hewitt, Christopher D. Manning, Percy Liang
Long samples of text from neural language models can be of poor quality. Truncation sampling algorithms--like top- or top- -- address this by setting some words' probabilitie…
Deep Bidirectional Language-Knowledge Graph Pretraining
Michihiro Yasunaga, Antoine Bosselut, Hongyu Ren +4
Pretraining a language model (LM) on text has been shown to help various downstream NLP tasks. Recent works show that a knowledge graph (KG) can complement text data, offering stru…
Diffusion-LM Improves Controllable Text Generation
Xiang Lisa Li, John Thickstun, Ishaan Gulrajani +2
Controlling the behavior of language models (LMs) without re-training is a major open problem in natural language generation. While recent works have demonstrated successes on cont…
LinkBERT: Pretraining Language Models with Document Links
Michihiro Yasunaga, Jure Leskovec, Percy Liang
Language model (LM) pretraining can learn various knowledge from text corpora, helping downstream tasks. However, existing methods such as BERT model a single document, and do not…
GreaseLM: Graph REASoning Enhanced Language Models for Question Answering
Xikun Zhang, Antoine Bosselut, Michihiro Yasunaga +4
Answering complex questions about textual narratives requires reasoning over both stated context and the world knowledge that underlies it. However, pretrained language models (LM)…
LM-Critic: Language Models for Unsupervised Grammatical Error Correction
Michihiro Yasunaga, Jure Leskovec, Percy Liang
Training a model for grammatical error correction (GEC) requires a set of labeled ungrammatical / grammatical sentence pairs, but manually annotating such pairs can be expensive. R…