Hyper-SET: Designing Transformers via Hyperspherical Energy Minimization

Yunzhe Hu, Difan Zou, Dong Xu

Transformer-based models have achieved remarkable success, but their core components, Transformer layers, are largely heuristics-driven and engineered from the bottom up, calling f…