2 papers
cs.CV2026
SpIn-ViT: Designing a Sparsity-Induced Vision Transformer That Is Mechanistically Interpretable
Philip H. Lee, Parth Padalkar
Mechanistic interpretability has recently expanded to Vision Transformers (ViTs), with Sparse Autoencoders (SAEs) increasingly used as post-hoc tools to decompose internal represen…
cs.LG2025
Position as Probability: Self-Supervised Transformers that Think Past Their Training for Length Extrapolation
Philip Heejun Lee
Deep sequence models typically degrade in accuracy when test sequences significantly exceed their training lengths, yet many critical tasks--such as algorithmic reasoning, multi-st…