Base Models Can Reason By Taking a Cue From Training Data

Sophie L. Wang, Amil Dravid, Rulin Shao +3

In this paper, we study how training data creates associations between the tokens at the start of a base model's response and the reasoning behavior that follows. First, we demonst…