collaborators

1 paper

cs.LG2025

Multi-matrix Factorization Attention

Jingcheng Hu, Houyi Li, Yinmin Zhang +5

We propose novel attention architectures, Multi-matrix Factorization Attention (MFA) and MFA-Key-Reuse (MFA-KR). Existing variants for standard Multi-Head Attention (MHA), includin…