Interpretable and Sparse Linear Attention with Decoupled Membership-Subspace Modeling via MCR2 Objective

Tianyuan Liu, Libin Hou, Linyuan Wang +1

Maximal Coding Rate Reduction (MCR2)-driven white-box transformer, grounded in structured representation learning, unifies interpretability and efficiency, providing a reliable whi…