2 papers
cs.LG2026
From Sparsity to Simplicity: Enabling Simpler Sequential Replacements via Sparse Attention Distillation
Yuxin Ren, Maxwell D Collins, Miao Hu +1
Self-attention serves as the core foundation of large-scale transformer pretraining, but its quadratic token interaction cost makes inference expensive. Replacing attention with si…
cs.CV2026
FAR: Function-preserving Attention Replacement for IMC-friendly Inference
Yuxin Ren, Maxwell D Collins, Miao Hu +1
While transformers dominate modern vision and language models, their attention mechanism remains poorly suited for in-memory computing (IMC) devices due to intensive activation-to-…