Uncovering Latent Reasoning Strategies in Language Models

Awni Altabaa, John Lafferty

A language model $p_{\hat{I}}¸(y\mid x)$ trained on reasoning tasks learns to solve problems via multiple distinct strategies, yet these strategies are implicit and entangled within the m…