Multiple Token Divergence: Measuring and Steering In-Context Computation Density

Vincent Herrmann, Eric Alcaide, Michael Wand +1

Measuring the in-context computational effort of language models is a key challenge, as metrics like next-token loss fail to capture reasoning complexity. Prior methods based on la…