Non-asymptotic Convergence of Training Transformers for Next-token Prediction

Ruiquan Huang, Yingbin Liang, Jing Yang

Transformers have achieved extraordinary success in modern machine learning due to their excellent ability to handle sequential data, especially in next-token prediction (NTP) task…