4 papers · 1 filter
Beyond the Training Horizon: Mechanisms and Limits of Length Generalization in Looped Transformers
Jia Liang, Xi Jin, Liangming Pan
Looped Transformers can generalize to reasoning chains longer than those encountered during training, but the computations enabling this behavior and limiting its extent remain unc…
Where Steering Signals Come From: Activation Source Selection in Activation Steering
Jiaran Ye, Lingxu Ran, Zijun Yao +5
Activation steering controls language models by adding vectors or features to hidden states at inference time, but the upstream source of these steering signals is often treated as…
ASA: Backbone-Training-Free Representation Engineering for Tool-Calling Agents
Youjin Wang, Run Zhou, Yingjie Ma +6
Adapting LLM agents to domain-specific tool calling remains notably brittle under evolving interfaces. Prompt and schema engineering is easy to deploy but often fragile under distr…
Do Latent-CoT Models Think Step-by-Step? A Mechanistic Study on Sequential Reasoning Tasks
Jia Liang, Liangming Pan
Latent Chain-of-Thought aims to enable step-by-step computation without emitting long rationales, yet its internal mechanisms remain unclear. We study CODI, a continuous-thought te…