Showing 2026Show all
2 papers · 1 filter
cs.AI2026
Role-Decoupled Attention Residuals: Separating Matching and Content Retrieval Across Depth
Kehan Wang
Depth-routing residual architectures allow Transformer layers to retrieve earlier representations instead of inheriting only the immediately preceding state. Existing Block Attenti…
cs.LG2026
HAARES Half-Split Residual Basis Routing for Deep Transformers
Kehan Wang
Block-level residual routing makes learned residual aggregation practical by routing over block summaries, but each summary compresses an ordered sequence of attention and MLP upda…