Exact Causal Attention with 10% Fewer Operations

Dmitry Rybin, Yushun Zhang, Ding Tian +2

We present Exact Causal Attention (ECA), a Strassen-style algorithm that computes exact Causal Attention using 10\% fewer operations. ECA improves a special class of matrix multipl…