Effective Does Not Mean Useful: Conditional Functional Substitutability for Redundancy and Scaling in Transformers

Jiaheng Chen, Jiaxing Li, Yucheng Xiao +4

Modern neural networks scale predictably, yet the mechanisms behind these regularities remain unclear. Neural redundancy is typically characterized by component importance or repre…