Attention Projection Mixing with Exogenous Anchors

Jonathan Su

Cross-layer reuse of early attention projections can improve optimization and data efficiency, but it creates a structural conflict: the first layer must simultaneously act as a st…