Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
Task Vector Geometry Underlies Dual Modes of Task Inference in Transformers
Hao Yan, Haolin Yang, Yiqiao Zhong
Transformers are effective at inferring the latent task from context via two inference modes: recognizing a task seen during training, and adapting to a novel one. Recent interpret…
cs.LG2023
Uncovering hidden geometry in Transformers via disentangling position and context
Jiajun Song, Yiqiao Zhong
Transformers are widely used to extract semantic meanings from input tokens, yet they usually operate as black-box models. In this paper, we present a simple yet informative decomp…