Early-stopping for Transformer model training

Jing He, Hua Jiang, Cheng Li +2

This work, based on Random Matrix Theory (RMT), introduces a novel early-stopping strategy for Transformer training dynamics. Utilizing the Power Law (PL) fit to tansformer attenti…