FON: A Draft-Revise Dual-Organ Language Model without Sequence-Level Quadratic Attention

Helong Hu, Hongdan Pan, Shuiqing Hu

Self-attention assigns two functions to one mechanism-where to read and how to mix-at the price of O(L^2) training, an O(T) decode state, and position signals bound to the training…