Why "classic" Transformers are shallow and how to make them go deep

Yueyao Yu, Yin Zhang

Since its introduction in 2017, Transformer has emerged as the leading neural network architecture, catalyzing revolutionary advancements in many AI disciplines. The key innovation…