Linear representations in language models can change dramatically over a conversation

Andrew Kyle Lampinen, Yuxuan Li, Eghbal Hosseini +2

Language model representations often contain linear directions that correspond to high-level concepts. Here, we study the dynamics of these representations: how representations evo…