An In-depth Investigation of Sparse Rate Reduction in Transformer-like Models

Yunzhe Hu, Difan Zou, Dong Xu

Deep neural networks have long been criticized for being black-box. To unveil the inner workings of modern neural architectures, a recent work \cite{yu2024white} proposed an inform…