Showing 2025 · cs.LGShow all
2 papers · 2 filters
cs.LG2025
Exact Causal Attention with 10% Fewer Operations
Dmitry Rybin, Yushun Zhang, Ding Tian +2
We present Exact Causal Attention (ECA), a Strassen-style algorithm that computes exact Causal Attention using 10\% fewer operations. ECA improves a special class of matrix multipl…
cs.LG2025
Towards Quantifying the Hessian Structure of Neural Networks
Zhaorui Dong, Yushun Zhang, Jianfeng Yao +1
Empirical studies reported that the Hessian matrix of neural networks (NNs) exhibits a near-block-diagonal structure, yet its theoretical foundation remains unclear. In this work,…