Accelerating Birkhoff Projection for Manifold-Constrained Hyper-Connections

Chenrui Wang, Yixuan Qiu

Manifold-constrained hyper-connections (mHCs) have recently been proposed as a principled extension of hyper-connections, where the residual mixing matrices are constrained to be d…