2 papers
cs.CV2025
Artifacts and Attention Sinks: Structured Approximations for Efficient Vision Transformers
Andrew Lu, Wentinn Liao, Liuhui Wang +2
Vision transformers have emerged as a powerful tool across a wide range of applications, yet their inner workings remain only partially understood. In this work, we examine the phe…
cs.LG2025
Decomposing Prediction Mechanisms for In-Context Recall
Sultan Daniels, Dylan Davis, Dhruv Gautam +3
We introduce a new family of toy problems that combine features of linear-regression-style continuous in-context learning (ICL) with discrete associative recall. We pretrain transf…