Global Convergence in Training Large-Scale Transformers

Cheng Gao, Yuan Cao, Zihao Li +5

Despite the widespread success of Transformers across various domains, their optimization guarantees in large-scale model settings are not well-understood. This paper rigorously an…