AdaMTP: An Adaptive Training Paradigm for Multi-Token Prediction

Ziqiang Cui, Han Shi, Bowei He +8

Multi-Token Prediction (MTP) has emerged as an effective paradigm that augments a shared Large Language Model backbone with auxiliary heads, training the model to predict several f…