MemoryFormer: Minimize Transformer Computation by Removing Fully-Connected Layers

Ning Ding, Yehui Tang, Haochen Qin +6

In order to reduce the computational complexity of large language models, great efforts have been made to to improve the efficiency of transformer models such as linear attention a…