As X, Do Y: How Persona and Task Combine in Instruction-Tuned LLMs

Eric Xu

Role prompts of the form As X, do Y admit a clean linear decomposition at one specific site in the residual stream: the prompt-to-answer transition -- the last prompt token togethe…