Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
Exact Fusion of Softmax Attention Heads Can Require Exponentially Many Atoms
Hiroyasu Usami, Keisuke Hara, Ayato Tsuboi +1
Fusing independently normalized attention heads can increase the number of explicit key-value entries, or atoms, needed to preserve their output. We prove a sharp separation for $W…
cs.LG2026
Invertible Graph Aggregation Can Increase ReLU Readout Width
Hiroyasu Usami, Masanori Tsukamoto, Kohei Hashiguchi
An invertible graph aggregation preserves the complete feature table, but can increase the size of a classifier that reads one node at a time. For every even , we construct…