Role-Decoupled Attention Residuals: Separating Matching and Content Retrieval Across Depth

Kehan Wang

Depth-routing residual architectures allow Transformer layers to retrieve earlier representations instead of inheriting only the immediately preceding state. Existing Block Attenti…