56 citations · 197 across the 41 of their papers we have counts for
7 papers · 1 filter
Do pretrained Transformers Learn In-Context by Gradient Descent?
Lingfeng Shen, Aayush Mishra, Daniel Khashabi
The emergence of In-Context Learning (ICL) in LLMs remains a remarkable phenomenon that is partially understood. To explain ICL, recent studies have created theoretical connections…
SemStamp: A Semantic Watermark with Paraphrastic Robustness for Text Generation
Abe Bohan Hou, Jingyu Zhang, Tianxing He +7
Existing watermarking algorithms are vulnerable to paraphrase attacks because of their token-level design. To address this issue, we propose SemStamp, a robust sentence-level seman…
Error Norm Truncation: Robust Training in the Presence of Data Noise for Text Generation Models
Tianjian Li, Haoran Xu, Philipp Koehn +2
Text generation models are notoriously vulnerable to errors in the training data. With the wide-spread availability of massive amounts of web-crawled data becoming more commonplace…
The Trickle-down Impact of Reward (In-)consistency on RLHF
Lingfeng Shen, Sihao Chen, Linfeng Song +5
Standard practice within Reinforcement Learning from Human Feedback (RLHF) involves optimizing against a Reward Model (RM), which itself is trained to reflect human preferences for…
GEAR: Augmenting Language Models with Generalizable and Efficient Tool Resolution
Yining Lu, Haoping Yu, Daniel Khashabi
Augmenting large language models (LLM) to use external tools enhances their performance across a variety of tasks. However, prior works over-rely on task-specific demonstration of…
"According to ...": Prompting Language Models Improves Quoting from Pre-Training Data
Orion Weller, Marc Marone, Nathaniel Weir +3
Large Language Models (LLMs) may hallucinate and generate fake information, despite pre-training on factual data. Inspired by the journalistic device of "according to sources", we…