Training Infinitely Deep and Wide Transformers

Raphaël Barboni, Maarten V. de Hoop, Takashi Furuya +1

Transformers have become the dominant architecture in modern machine learning, yet the theoretical understanding of their training dynamics remains limited. This paper develops a r…