1 citations · 1 across the 4 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
Prism Transformer: Progressive Head Schedules for Hierarchical Attention Processing
Shubham Aggarwal
Multi-head attention conventionally partitions the hidden dimension equally across all heads at every layer, enforcing an identical representational subspace dimension (dh = dmodel…
cs.LG2026
Rethinking Attention Output Projection: Structured Hadamard Transforms for Efficient Transformers
Shubham Aggarwal, Lokendra Kumar
The dense output projection in multi head attention scales quadratically with model dimension, contributing significantly to parameter count, memory footprint, and inference cost.…